AFAIK, it does. ESO has "movement speed" and that base number gets augmented and/or penalized, depending. Swift buffs movement speed while being in stealth penalizes it, so the trick is to load on sufficient buffs to offset or even cancel the penalty.