As the above said, weapon damage doesn't increase with stamina, only abilities that use stamina have their damage increased with more stamina.
Also the offhand weapon only adds %25 of it's weapon damage to your total (each hit then goes off your total instead of alternating), though if you take the duel-wield expert passive it increases to 40% then 50% of the weapon damage (lvl 1 and 2).
leandro.800ub17_ESO wrote: »So when you up stamina you only affect weapon skills and not mouse button?