Not a good idea. Each race is "itself" - which includes passives.

The main reason I dislike this is that racial passives are a narrative tool and serve a purpose. If you don't know anything about the lore, you can still read a description of each of the races and immediately get a feel for them. Without that, you can still put in a description, but since it doesn't affect you in any way at all, that's just more text that people are going to click "skip" on. Ideally playing a Redguard should make you feel like you are a master of martial weapons, etc. That's what racial passives are good for in my opinion.

Come to think of it, now that everything scales with your highest stat and all sets are hybrid, I am wondering if there is even still a point to having Altmer increase weapon damage, because that makes very little sense from a lore perspective, Orcs having spell damage even less so.