It's just you, all the mounts run at the same speed. But remember it does depend on your riding skill - if you're comparing mounts on different characters, or comparing your mount to other people's you need to factor that in.
The different animations can also make it seem like one is slower than another, but it's just an optical illusion.
PC EU player | She/her/hers | PAWS (Positively Against Wrip-off Stuff) - Say No to Crown Crates!
"Remember in this game we call life that no one said it's fair"
As mentioned, it's an optical illusion. You see the same effect in Lord Of The Rings Online. Dwarf and Hobbit race both use ponies instead of horses. The ponies are shorter/lower to the ground and visually it looks and feels like they are going twice the speed.