Mournhold has the shops, though I suppose not everyone is a frequent buyer of things.
If not for that, I'd actually go with Elden Root. I know others may disagree, but I like the main area layout and never once had any loading screen issues there.
Actually, if the question weren't limited to faction capitals I'd say that for everything other than traders and Undaunted I prefer to hang around Vulkhel's Guard (with either of the other two starting locations close behind). Again, it's just the layout of the various services.
Mournhold. Everything is close and easy to get to. Wayrest is a maze and Elden Root you have to deal with a loading screen to get to the bank/crafting area from the wayshrine.
They all have their problems and none really stand out for me.
Elden Root, while interesting as far as aesthetics goes, has the issue of needing to travel from the wayshrine into the "City" proper to use most of the facilities.
Wayrest is pretty generic looking. It looks like a proper city sure but the issue comes from how far away everything is from each other. Crafting stations in the N, Bank in the West, Dye Station/Enchanting/Alchemy tables in the S, Undaunted Enclave, Traders and Respec Shrine in the SW. Just a pain to traverse.
Mournhold has most the facilities right beside the wayshrine, making it convenient, with only the Bank and Undaunted Enclave being a small trek to get to but probably the most dull of the 3 in aesthetics. Wayrest may be a generic looking city but at least it LOOKS like a proper city instead of a handful of large buildings in close proximity to one another.
Of the 3, I say Mournhold is my favorite merely for convenience purposes alone but again, they all SUCK (IMO)