Personal biases against hard sci-fi aside, this is a very interesting train of thought. Edit: Also, I think I might be one of these jerks.A point Harry Harrison made in just about every book he wrote (notably the Stainless Steel Rat series and especially the Deathworld trilogy) is that interstellar warfare is a null concept. The resources necessary are too vast and the plunder possible is too limited to bother. David Brin, when he writes "malevolent aliens" does not write of conquerers, he writes of assassins extinguishing whole species long distance as an afterthought.
http://writebadlywell.blogspot.com/2010/02/sci-fi-week-1-replace-real-life-items.html?m=1 The study is stupid. Replacing interchangeable words with sci fi terms is the hallmark of bad science fiction. This is like explaining that garlic is bad because when you add it to things that don't need garlic they taste needlessly of garlic.
To pick up on a very old thread... I've been reading David Abram's The Spell of the Sensuous lately. He has a chapter in there about how in oral cultures stories are very closely bound up with particular places, so that it makes no sense to tell a story without saying where it happened. The powers of the place are actively involved in the story. Only in cultures with alphabetical writing, he says (roughly), do we find being treated as a neutral setting (space) for action. I wonder if part of the issue isn't a prejudice against science fiction, but that we naturally need to understand stories in terms of relatable places - that we relate better to the story when the setting is relatable and naturally absorb it more deeply? Presumably most of us don't have much familiarity with spaceships and airlocks.
It sounds to me that the key is "oral cultures." When your body of knowledge exists as folklore shared amongst those you only know face-to-face, speculation is the same thing as lying. I'm unaware of any non-writing culture that thought hypothetically and thinking hypothetically is the crux of science fiction.