Be a nightblade and stealth away. Then hide till he de-aggros

If you can just dump through a door, the guard should be able to continue chase outside.

Just do it with an alt in the Mournhold bank, as long as you don't go downstairs you never have to deal with guards. I have an alt planted in that bank that doesn't even sneak around anymore. The master safebox shows up, I brazenly walk up and pick it whether or not a bank teller is looking not even in stealth. Grab all the desks, go to the next room grab the cabinet, desk, and the other two chests; logout. Come back another time when the safeboxes respawn to do it all over again. When I return the next day, all that bounty has cleared and I can prance my way over to sell the stolen loot.

Occasionally the Fighter guild NPC wants to fight, but I have prevent attack innocent NPC turned ON. In this case if its my Sorc alt I just summon pets who auto-attack for me, Twilight Tormentor does a lot of damage, and generally kill the NPC. As long as I am not in line of sight and in sneak when the NPC dies I won't get additional bounties even though my pets made the kill. Alternate option, if you have at least 30 points invested in Atronach CP you get Riposte passive which turns your Block into an attack, with Prevent attack Innocent On this becomes a viable counterattack without actually hitting back. As the NPC hits you while blocking, you do return damage eventually killing the NPC.