A regular correspondent sent this to me, asking did it sound familiar. And it does. You’re allowed a guess or two as to what field the writer is talking about. It isn’t ‘climate change’, but it could be, because some of the same fundamental weaknesses in the funding and carrying out of science are present there too. The author is American and he is plainly talking about the USA — yet how much of what he points it is not also evident here? A lot of it, I think.
The responsibility for this unfortunate state of affairs rests squarely on the leaders of XXXX research. Rather than training graduate students in the scientific method, and allowing their research to serve the needs of society, the field’s leaders choose to train their mentees to serve only their own professional needs—namely, to obtain grant funding and publish their research. I have experienced these practices myself as I transitioned from student to graduate research assistant to research fellow, and colleagues continue to emphasize that this is how it must be, lest they fail to get funding and ‘feed’ their graduate students and families. But by not training mentees in the basics of science and skepticism, the XXXX field has fostered the use of measures that are so profoundly dissonant with scientific principles that they will never yield a definitive conclusion. As such, we now have multiple generations of XXXX researchers who dominate federal XXXX research and the peer review of that work, but lack the critical thinking skills necessary to critique or conduct sound scientific research.
The apparent self-interest that is driving research in this field is not limited to raising students to merely follow the herd. The subjective data yielded by poorly formulated XXXX studies are also the perfect vehicle to perpetuate a never-ending cycle of ambiguous findings leading to ever-more federal funding… an estimated $2.2 billion on XXXX research in the 2012 fiscal year, a significant proportion of which was spent on research that used … pseudoscientific methods … The fact that XXXX researchers have known for decades that these techniques are invalid implies that the field has been perpetrating fraud against the US taxpayers for more than 40 years—far greater than any fraud perpetrated in the private sector (e.g., the Enron and Madoff scandals).
When anti-science rhetoric occurs at a Kansas school-board fight over creationism, we can nod our educated heads in silent amusement, but if multiple generations of XXXX researchers have been trained to ignore contrary evidence, to continue writing and receiving grants, and to keep publishing specious results, the scientific community as a whole has a major credibility issue. Perhaps more importantly, to waste finite … research resources on pseudo-quantitative methods and then attempt to base public … policy on these anecdotal “data” is not only inane, it is willfully fraudulent.
The solution to this dilemma is quite simple: funding agencies must stop funding flawed XXXX research, and the editors of … journals need to stop publishing the results. Given the immense amount of money invested in this field each year, this goal is much easier to state than to accomplish.
OK. The field is nutrition, and the lament comes from a paper that you can read here. The lead author, Edward Archer, and his colleagues looked at all the data from the National Health and Nutrition Examination Survey (NHANES) from 1971 to 2010, covering 29,000 men and 35,000 women. Archer zeroes in on the basic weakness: each respondent’s food intake was estimated from a 24-hour retrospective recall of his or her actual diet over that day. Converting the reported diet into energy values led to ‘caloric energy intake’ figures that Archer says are just ‘physiologically implausible’. In other words, respondents systematically under-reported what they had eaten.
His conclusion: the ability to estimate population trends in caloric intake and generate empirically supported public policy relevant to diet-health relationships from U.S. nutritional surveillance is extremely limited. In his view researchers are, or should be knowledgeable about the problem of self-reporting, and no less aware that the other constituents are basal energy expenditure and physical activity: it’s not only what you take in, but what is needed to keep you going plus how much physical activity you engage in.
The failure of nutrition studies to deal with such obvious weaknesses means, as he sees it, that after about 40 years and many millions of dollars of US nutritional surveillance data were fatally flawed. In most research domains, such a finding might be monumental; yet in nutrition epidemiology—the study of the impact of diet on health, here(in)after referred to simply as “nutrition”—these results are commonplace. In fact, there is a large body of evidence demonstrating that the systematic misreporting of energy and macronutrient intake renders the results and conclusions of the vast majority of federally funded nutrition studies invalid.
I can see obvious parallels with climate science: the failure of researchers to look critically at their own work and that of their colleagues, the stern defence of what they have published, the ignoring of really basic problems in the methodology, the production of results that please the paymasters, and so on.
There is a temptation to argue that it is only climate science where these glaring weaknesses exist, and to provide good reasons why this might be true. But this article, and other work by John Ioannidis that I have mentioned in the past, suggest that there may well be systemic problems in the way contemporary science is funded and carried out.