Not sure in your case, but I play a 100% nightblade with Elemental Susceptibility slotted. If I hit someone with
Elemental Susceptibility I can forget about staying cloaked until that wears off the enemy. Elemental Susceptibility applies direct damage over time (DoT) effects, such as Chilled and Concussed, which are considered direct damage. I suppose if it's not damage on you, then it could be a single target "sticky" damage skill that you applied to the enemy which has periodic DoTs that do direct damage per tick.

I dont slot any dots or poison, but maybe it was frost staff chilled or something else. I can only think of frost and shock staffs i was using but even then, as you can see it took really long time, i reapplied cloak 5 or 6 times, walking inbetween. And wasnt really actively attacking beforehand. Maybe casted one or two light attacks but idk. It was literally up to 20 seconds that my cloak was breaking and i got the miss text on me. I was thinking maybe some dot that registers as direct hit, or something like that? Also my enemies on the screen werent visually affected by anything from me. Strange. There was a hit effect every time i got off cloak, either a combustion or lightning but cant really tell. I was thinking about elemental susceptibility that was on me but i tested it and it doesnt break cloak for my enemies, when i cast it on someone cloaking.