Outside of indie games, it seems that for the most part companies have gotten it in to their heads that if they let the player be a girl then anyone touching it will think it's gross.

Lara Croft and Samus would disagree. the last installment of Tomb Raider was even about Lara saving her best friend from the clutches of an apocalyptic cult of sexists... who were woshipping a woman.

The issue for Red Dead specifically is likely one of time rather than malicious intent.

Right. An error of omission. That makes it a casual problem in a $100m game. So in other words, we can believe on of two scenarios:

1) Rock Star spent $100m on a game and didn't bother to go through their press materials to see if there was anything that might be deeply offensive.

2) Rock Star spent $100m on a game and leveraged the press controversy that always follows Rock Star releases despite perpetuating negative gender stereotypes.

In other words, they're so offensive they didn't realize they were being offensive, or they're so offensive they decided to profit from being offensive. Either way, this isn't a "sweep it under the rug" problem.