I would love to be able to join the local enemy - like Winterborn in Wrothgar or Order of the Hour in Gold Coast or just some bandit groups or even those small funny goblin groups.
How much it would change the game and what would have to be dealt with? And would it be worthy of all the effort?

1. If it wouldn't have to offer a full main story or lead to destruction of the opposite (main) faction, then the original quests wouldn't have to be changed, only I would be able to do some minor skirmishes, that would appear because of such quest.
2. all the human (almost human) groups would have to be non aggresive to players as default, with some leader, who would offer some small stupid dailies (like kidnapping, searching for dark brotherhood headquarters, or taking bottles from Dunmers) - and only if I would agree with a quest, that would lead to conflict with them, they would became aggressive to me.
3. Because it would be so minor, there could be even some prerequisities like the player would have to be certain race(s), or some gifts present in his inventory and some locals could give you tips, what you could bring to gain their trust.
4. some quests would have to have 2 variations - the peaceful and the aggresive, but some do have it already anyway, and if the original quest was meant to destroy them, they would became aggressive to the player after recieving such quest and this kind of relationship would be disabled forever.
5. what next it should have?

Such world would be much more believable to me and I miss such features a lot in this game, so would it be possible and what would have to be changed to make this happen?