Nah, have no shame in your roleplay characters. I feel bad for the people who don't get immersed like this!

Goblin bands are actually pretty rare in the AD zones, so you're not going to run into many NPCs who have a strong reaction to them, since most of the population hasn't encountered them in the wild. There is the fact that, historically as well as currently, High Elves are known to occasionally keep goblin slaves. So the idea that a goblin might be "domesticated" isn't too far a reach for the Altmer, though it's assumed you'd be treated much like one of the "slave races" is treated in Morrowind.

Story-wise... hm. The character who pulls you into the faction questline is an out-of-the-box thinker, so I can see this being pulled off. The role you play in the storyline may be a bit too heroic, though, given the above way the Altmer will generally look at a goblin. Then again, given the nature of your role in the storyline, that could very well work in your character's favor.

My advice? Give it a shot. Play through the Auridon zone and see if it gels. Auridon is going to be likely where your character would meet the most resistance (again, because of the Altmer), and it's the first zone after the starter island, so it wouldn't take too long to tell whether it works or not. After that, you'd be in the clear, and it would certainly be an interesting twist to have *cough spoiler cough* be a goblin!

Edit: And have fun! The AD questline has my favorite character in the game, so if you like a good story with interesting characters, you'll probably like this one a lot!

Something you may want to consider is whether a goblin would be considered sufficiently sapient to be able to act with rational judgement in the context of receiving and completing quests. Different tribes around Tamriel appear to have different levels of intelligence, with some being able to make minimal contracts with other races, while others are little more than beasts.