I was going to suggest that it might be related to the fact that dungeons are separate instances. I'm not sure whether public dungeons are separate instances from the zones they're in.

But then I considered that we can teleport directly to wayshrines (for a fee) and to the exterior of player houses, so now I'm not sure it has anything to do with instances.

Although maybe it does after all, because if you queue for a dungeon or teleport directly to one from the map, exiting the dungeon puts you back where you were before.

The same is true when you queue for a battleground, or for a campaign in Cyrodiil or the Imperial City-- exiting the instance places you back at whatever location you were at when you accepted the queue.

In contrast, if you were to teleport to a public dungeon and then exit, it would put you outside the entrance to the public dungeon. I don't see why that should be a problem, since that's comparable to teleporting to the interior of a player house you own and then exiting the player house, but maybe there's some obscure issue that would need to be addressed to make it so-- aside from any UI changes, that is.

Why not? Definitely a QoL feature.