The tough part about racials, for me, is when the community decides that a particular race is better at some particular role. Players playing that role, but the "wrong" race become somewhat ostracized in groups. (See: troll warriors in classic WoW)
The trouble is that race is a decision (often) made at the very start of their play with the game. If they "choose wrong", they're (lightly) punished - or they experience anxiety surrounding the choice.
If players could change these racials after the fact, it wouldn't be a problem, even if it were difficult to do. They probably wouldn't be called racials at that point, though.
WoW's racials especially irk me due to the paid race changes. Min/maxers feel compelled to follow the FOTM. I know quite a few folk that spent an unreasonable amount of money on that, just so they could be "optimal" :-/