It is a simple issue. There is one pavilion with one NPC in one zone for the Festival. Causing a few issues as you would imagine with everyone congregating in one place.
If you stay away from Eastmarch, I would imagine the game performs just fine. Hopefully they will either expand the locations this NPC is at or create more instances for them to appear.