I don't worry about it. I waited until it had quieted down last year, found a quiet place to fish, then picked up 30-40 of each fish. Stored them in bank so I can get an instant turn-in.

That said, I will admit the flappy birds have been the bane of my existence when trying to interact with anything. I really wish there was a "hide other players' pets when not in combat" option. Or that the pets would only appear while there is active combat, period.