Its a matter of lag around the dolmens, usually due to a large number of players around the dolmens. Its especially prevalent during the festival and zones where dolmen-farming groups run.

Suggestions that have worked for me:
1. Group with other dolmen farmers - you'll get credit for kills.
2. Go to a less populated zone and fight at those dolmens - not always a good solution since sometimes it has to be that dolmen or you want gear from a particular zone. However, if you want confirmation that its the players and not your system, try dolmens in Eastmarch, the Rift, or a less popular zone.
3. Equip non-targeted skills like ground-based AOEs. Wall of Elements and Endless Hail are available to any magicka or stamina characters very quickly. Run around the dolmen spamming your AOE to damage enemies and lay down the AOE where the boss will land. If you have a non-targeted or AOE ultimate, switch to that for damage on the boss even if he doesn't load in properly for you.

Number 3 worked pretty well for me last year during the festival or any time when dolmens are loading properly, which is almost always when there is a large population of players at the dolmen.