It applies to damage-over-time effects, including sets. Poison Arrow: no to the first hit, yes to the poison damage over time. Charge: no to the direct damage (charge part), yes to the AoE DoT from Stampede
There are always some skills that SHOULD be considered DoT or direct damage, but aren't. For the most part, it's any damage that has a duration component