Experienced this weirdness too.
It seems to go by the last place you fast traveled or transitioned FROM outside of a player home, whether it's a wayshrine, fast travel, etc.
Here's a test example:

Last night when I finished up I fast traveled from a beach in Auridon to my home (HotLC)
- So today just now I FT to Arx Corinium from HotLC
- Exited using the doors, it put me back on that beach in Auridon where I was last night before I FT home last night.
- Next I RODE to Arx, entered the front doors.
- On exit I ended up outside the doors of Arx where I'd expect to be.

So if you port to a dungeon, when you leave it will place you at the last place you fast traveled from.
Exception is if you are inside a player home instance it will choose the last fast travel spot you jumped from BEFORE you entered you home.
If you walk in the doors normally, when you leave it will put you back outside like "normal"

Hope that makes sense? It was less confusing in my brain.