I am actually planning on switching away from destruction staff. I was forced to switch weapons for a quest in order to kill an NPC that had healing orbs because the staff just could not attack fast enough (I was doing a fast attack as well).
The destruction staff just doesn't seem to do a ton of damage, and the first skill you can get is really nice as fire, but the lightning was really bad and I never got to try the frost one.