When the game freezes and your toon can still move around, it is your connectivity. Try disconnecting or pulling the power on your router mid-game, you'll see the same behaviour. This is different from the widespread lagging and disconnect that happens for everybody in Cyrodiil.

It may seem to have related timing, because pvp combat is complex and results in a large number of activity/messages/packets being sent back and forth. ESO is a bandwidth hog, even more so than most games. Make sure that you have adequate down and upload speeds, and ensure that your isp doesn't throttle you (a particular risk during peak traffic times). Avoid playing while anybody else on your network is streaming or downloading things.

If all of these things are beyond your control, consider a router reboot and/or dns flush periodically during long play sessions.