I don't think you're crazy, but I also don't think that most people are very aware of their own prejudices or even that those prejudices are coming through in their actions.

I'm not a fan of Roseanne, but I did think that this episode dealt with the topic pretty well, especially given the time it was released:

Basically the son is in a school play and is excited that he's required to kiss a girl until he finds out that the role will be played by a black girl, but he doesn't tell his parents that, only that his teacher has given him the ultimatum of playing the role or leaving the play. Roseanne thinks this is unfair and goes to talk to the teacher about it, who believes that Roseanne is upset because of the girl's race. The family then examines their own prejudices and how that affects their relationships with people in their lives.

I don't blame people for having prejudices (because we all have them) but I do think that the responsible thing to do is to examine the motivations behind those prejudices and to try to move past them. And of course, I don't expect others to see things that way, though I hope that more people will.