Maybe the majority of people driving FWD might not know or care, but I believe the majority driving RWD do know, and do care.I think the vast majority of people have no idea which wheels are driving the car, and therefore don't care.
So I'd disagree with you when you say people will go out of their way to buy RWD. BUT, that said, it depends on the segment. Some segments, FWD would be unacceptable. Others (most vehicles, as evidenced by what sells) it's not a big deal, and people don't care.