People will just find a way to sustain on blue food more likely.
From my experience losing max stamina/magicka is less critical then losing SD/WD enchants or optimal mundus. Just look at what effect bloodthirsty trait on jewelry did. It notable boost 2-3k for overall dps without notably crippling dps in non-execute range.