Last week, PolicyMic pundit Cameron English discussed some recent studies that have found associations between diet soda intake and weight gain, cardiac disease, and other adverse health outcomes. In his article, English cast well-deserved scorn on media outlets that twisted the researchers’ findings. Reports suggested that the studies found diet soda to “cause” X, Y, or Z health outcome, when in reality the studies were only designed to establish correlations. This irresponsible coverage, while no doubt effective in generating more clicks, subscriptions, and ad sales, suggests that many in the American media are out of touch with the realities of biomedical research.
In his critique, however, English unfortunately falls into the same trap. In response to the researchers’ claims, he writes, “There's one problem, however: they're not true.” This language has no place in a scientific discussion. In writing up their findings, the researchers suggested potential mechanisms through which diet sodas could lead to adverse health outcomes, but nowhere did they claim that their studies had established a causal relationship. Such a relationship, they were clear to state, will have to be explored in future research. If the researchers were merely presenting their data and highlighting avenues for further investigation, what exactly could be called “not true?”
The language used in the diet soda debate highlights a shortcoming in the way the public talks about research. The wording we use when discussing science is important. Studies don’t “prove,” nor are they “true” or “false.” Rather, they “suggest” or they “find”; they are well designed and executed or they are flawed. The language isn’t sexy, and it may not make headlines, but it’s responsible. If we are to improve the tone of the debate, America needs to bear in mind the fundamentals of scientific investigation.
Let’s think about a basic statistical concept in scientific inquiry, the null hypothesis. When planning an experiment, a researcher envisions a reasonably expected result. This default outcome is the null hypothesis, and the result of the experiment can yield one of two conclusions: the researcher can reject the null hypothesis or fail to reject the null hypothesis. Importantly, one cannot prove the null hypothesis. When we use inductive reasoning a claim can be shown to be almost certainly true, but it cannot be proven. This basic principle has important implications for the way we interpret scientific findings.
We can apply the idea to any number of the scientific issues that periodically crop up in the media. If we are investigating a new medicine’s cardiac side effects, for example, the null hypothesis may anticipate that the drug has no impact on heart function. We can either reject the null hypothesis and demonstrate that the drug is harmful or we can fail to reject it. We cannot, however, prove that the new drug is safe. MMR vaccines and autism? A responsible scientist is unlikely to say that vaccines have been “proven” safe. Rather, an overwhelming amount of evidence has been gathered that fails to associate them with autism spectrum disorders. This type of semantic hair splitting is of no medical relevance given the wealth of data in support of vaccination, but it is enough to scare some parents into avoiding routine inoculation.
So where do we go wrong when we consume or write about scientific information? Science has fallen victim to the same forces that are at work across the media. Americans, bombarded by ever increasing streams of information, no longer have time nor interest for issues that cannot be tidily summed up in a headline or tweet. We want our science quickly served and easily digestible. We also grow frustrated with the ebbs and flows that typify scientific progress: “one day they’re telling us to switch to diet soda, the next day they’re telling us it makes you fat.” Whether it’s climate change or cancer screening, these issues can cut right to the core of our existences. Of course we want clear-cut answers. Is it any surprise that this is what the media then offer?
But science does not deal in absolutes. The research process yields conclusions in varying shades of grey, conclusions that must then be filtered through our own value systems to be given meaning and offer us guidance. Americans must respect the fact that science is a painstaking process that does not conform to the demands of the news cycle. We can be more confident or less about a given finding, but we can never be sure, and we can never stop questioning. We should incorporate the scientific method into our own lives, approaching issues with a critical eye and an open mind. We must accept the complexity and ambiguity of scientific findings, employing our best objective judgment as we wade through the muck. This is not an argument for left or right thinking; there is room for all comers in an intellectual forum. There is no room, however, for those who give in to the desire to make things quick and easy; for those who refuse to earnestly employ the tools of critical, scientific, thought.
Photo Credit: Wikimedia Commons