I think this is not a racist reaction but rather a hidden cost of having those programs to begin with. When we use different acceptance criteria for women or minorities, it comes with a racist/sexist assumption that they need extra help or special treatment. In a real sense that undermines the legitimate work and contributions they make later. It's tough to have it both ways because if there are different admission criteria for minority groups, then it is correct to say that some got in only because they were non-white/Asian/Indian or female. If there are not different admission criteria, that distinction no longer exists. I have heard the arguments for affirmative action too, however, and they are good. I don't know the best way to fix it though.