The dragons are speaking Dovahul which is a language anyone can hear without being Dragonborn (Greybeards, Dragon Priests, etc.). What makes the Dragonborn special is they can the language as a shout almost immediately. Our character can't shout so no lore is being broken.
Did you just think that throughout Skyrim whenever you shouted, other people just heard garbled nonsense coming from you?
Just because you can hear the words doesn't mean you know what they mean, and it wouldn't break lore even if non-dragonborn or greybeard characters did, it's just a definition. It's being able to use those words in a shout that takes people a lot of time and practice - not being able to hear it
Dovahzul is an actual Language. Anyone can learn the language. What's special are the shouts. Sure you can learn and speak the language, but can you "shout" it? Most people no. The Greybeards even say they spent their entire lives learning how to Shout but only the Dragonborn can do it almost immediately without training.
A funny example is Katia from the Webcomic Prequel: Making a Cat Cry the Adventure. She's a Khajiit that knows "Dragonish".
CP1,700+Master Explorer - AvA Legate - Console Peasant - The Clan