.......you have a point.
The wall, is just a wall. Sure they say the seas are particularly rough around, maybe the only "safe" approach is on the west. Portals could also compromised or hijacked, and portal rituals need to be more complex to move large numbers.