I use the Resto Ult ALOT.
While I didn't check for the numbers, I'm sure the specific Weapon Tree passives will be lost.
For example the last Resto passive (+5% healing) is lost when switching to Destro.
Not to mention, while the Ult lasts on swap, that your higher/lower Max magicka and Spelldmg also change on bar swap(changing amount of healing in case of Resto ult).