I'm looking to roll a thief just for fun and to explore other aspects of the game. I'm guessing the best class is Nightblade and I'm thinking a stamina build but would like to get input from more experienced players than myself.
A Khajiit Nightblade would indeed be a great fit for you, in my opinion Stamina would be the most "lore-friendly", though Magicka can work as well! Most of the sets for decreasing your sneak detection range are Stamina. Though, the new Darloc Brae set introduced in Elsweyr seems to be okay for either Stamina or Magicka.
Its night blade, going stamina is recommended, Khajiit has bonus to sneak distance so an bonus.
Vampire for sneaking at higher speed
Nothing stop you from using this character for other stuff like dungeons or pvp, just use special gear for sneaking and stealing.
If you're only stealing, and not trying to kill anything, then using the armor sets suggested by @Ildun , combined with high Mag and Mag Regen as suggested by @hakan will not only make you hard to detect, but allow you to spam Shadowy Disguise. Stage 4 Vampire with Dark Stalker passive to remove the stealth speed penalty, and Concealed Weapon slotted for 25% stealth speed increase. Add in a skill for snare removal, and you'll never be caught.
8-hr/day casual gamer on Xbox NA. 20 Characters, all DC, all Level 50. +2400CP