There unfortunately aren't many options, so it has been a constant request for them on the forums.
There is an Ansei sword from Alik'r:
The Dark Brotherhood dagger from Gold Coast:
An Antiquity from Hew's Bane, a Yokudan scabbard:
Then there are the Alliance shields from any Global Achievement vendor. Sometimes you can work with a statue to have a shield on the wall if you cleverly position them, like with the Orc statue.
There are some knives, like the Common Kitchen Knives, Butcher Knives, and the Orcish Kitchen Knife.
Otherwise... I don't think there's much else. You could make some custom weapons. I've made halberds using Common Flagpoles and Cheese Cutters from the Mage's Guild, or spears by attaching a knife at the end of the poles. If you want to get reaaal technical with it then there are poison bottles too. You could also probably make your own staffs.
As far as armor, some monster heads and busts can work as facades, or certain statues. There are Factotum body parts as well.