I once had similar problem. My charater had both home and guest campaign assigned to Wabbajack. This made me lose the home buffs. If that is the same issue I would contact the customer service.
A is for Atronach.
B is for Bungler's Bane.
C is for Comberry.
Just to make sure-
You do know you only get buffs for the enemy scrolls held right? Holding your own gets u nothing but the ability to grab enemy scrolls.