Everything that hits immediately. Initial ticks of dots are also counted as direct damage, only the ones that follow up are not. You can test everything by using scathing mage. It doesn't have anything to do with aoe. The initial hit of damage over time effects and spammable skills (may it be aoe like deep fissure or single target like the cliffracer) are counted as direct damage.
The most prominent exception is twisting path, an aoe damage over time ability where all ticks count as direct damage AND dot at the same time.