a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment by thundara

All these "scandals" are scientists examining the work of other scientists and pointing out their biases, errors, and scams.

Open access journals that are free to the public sound great. Until you remember that literally anybody can set them up and profit on the publication fees. Suddenly openness must be balanced with reputability. The result? PLOS / eLife / open-access options on traditional journals.

Meta-analysis of many clinical trials sounds great. Until you remember that garbage in == garbage out and small trials with small biases that turn into large biases upon aggregation. Suddenly trial number must be balanced with trial size and power.

Statistical analysis of quantitative data sounds great. Until you remember those functions are built on a rigid framework of mathematical assumptions. Suddenly significance must be balanced with human behavior and uncertainty in newly developed methods or untrained researchers.

I would argue that in addition to being "harder than we give it credit for", the challenges brought up here are partially one of size. Science has grown into a massive sector full of universities, institutes, private labs, etc. And as the number of people involved goes up, you get: (1) a greater chance of any one researcher committing fraud, and (2) a greater chance that a bias will be uncovered by any one scientist with extra time or significant concern with some issue.