I've had this, mostly if I'm crossing a lake they despawn and come back but I've had a few occasions where they didn't reappear.
Also is it intentional that they despawn when I use a banker or merchant? Cos that's just irritating if I'm in the middle of a dungeon or delve. I keep missing that they're gone after an inventory purge and then missing out on XP for them.
There is one specific spot in craglorn where she will despawn without fail and you have to manually call her again, very easily repeated