I think it's supposed to be dangerous to fish in IC. That's part of the fun in it.
The same goes for Cyrodiil PVE. I didn't start playing PVP until the end of August 2014. Completing the Cyrodiil PVE achievements on Thornblade at its peak as a noob was extremely entertaining. It provided a sense of danger that this game otherwise lacks most of the time.
I have some admiration for the opponents who ignored the fisherman, but I don't fault those who attacked him. We're all fair game in IC.
I kill every AD and DC on sight anyday any level any time. We are at war, the minute your people stop attacking our castles and give us the throne I will let you fish in peace. Until then I'll have your head.