Skill lag in group play

Khyllar

it seems when ever I have more then 2 people around me I get some minor to severe skill lag. Even in 4 man group dungeons I can notice a slight delay in my abilities and hit registration. Does anyone else have this issue or is it something I can fix? Its pretty frustrating when the combat is so smooth when im by myself but really sluggish when I play with other people.