Since each armor piece has a unique armor value I feel like that larger pieces should be more affected by the armor passive.
The reason I suggest this is because there is very little value in using a light armor chest piece. It offers a significant decrease in resistance but no more magicka recovery or increase in crit damage.
As a Magicka user I'm almost handcuffed into wearing heavy armor on a larger armor piece.
I propose that the larger the piece of armor is, the more it's affected by the passive. This way there is a trade off of survivability vs dps.