In Part I of this series, we elaborated a bit on the philosophical and methodological flaws of mainstream Western medicine. In this ensuing Part II of our series, we will continue our exploration with our current medical system — albeit from a vastly different perspective.
It’s one thing to claim that a house is built upon unsustainable pillars, but it’s another thing to claim that the construction process has been carried out improperly, leaving countless tiny holes unattended here and there. In what follows, we’ll focus on how our medical system can fail us, solely due to the fact that it’s managed by individuals with a diverse range of beliefs and interests.
Every experimental/observational study on the effects of a medical treatment invariably requires an elaborated eligibility criteria — a criteria based on which potential subjects are either included in, or excluded from a study.
While seemingly methodologically reasonable at first sight, eligibility criteria can be tightened (somewhat arbitrarily) by the researchers to exclude certain key subpopulations (e.g., minors, seniors, patients with certain medical or genetic conditions) from participating in a study — even if the examined treatment specifically targets these key subpopulations as its primary or secondary audience. Result? A study outcome that does not necessarily reflect the one experienced by the general public.
In a similar spirit, a clinical study typically involves a washout period preceding the actual trial, whereby researchers are allowed to exclude clinical data of subjects who experienced side effects during the treatment. To put things into perspective, such exclusions can amount to clinical data distortion, but can also be easily justified by simply citing safety concerns.
While methodologically distinct, both eligibility criteria and washout period can be taken advantage of, to maximize the reported effectiveness of a treatment — all the while minimizing the treatment’s reported side effects. What’s worse, such “tamperings”, intentional or otherwise, are usually carried out rather subtly, in a way that even if there is malicious intent involved, it would be extremely difficult to prove it in court.
While reading conclusions from a study certainly doesn’t seem too difficult, constructing a robust experimental design does require a significant amount of brainstorming and planning. For example. if experimental subjects become aware of the treatments they will be receiving, their beliefs on these treatments can very well alter the experimental outcomes (e.g., placebo effect). On the other hand, if the investigators become aware of the treatments the patients will be receiving, there will also be a risk of them cherry-picking the data. For these reasons, well-designed clinical trials tend to involve an aspect of separation between investigators and data gatherers. and are usually double-blind, that is, neither the treatment prescribers nor the patients are aware of the kind of treatments being prescribed by the experimenters.
And then, we also have methodological biases of a more statistical nature, which, in all likelihood, are more of a concern for the peer-reviewers than for the non-academics. Such biases include, among others, certain forms of statistical favouritism (e.g., experimental subjects not randomly chosen from the targeted population; experimental subjects not randomly assigned to the treatment groups) and other statistical abuses (e.g., preconditions of confidence interval not met; employing methods of hypothesis testing which has low power).
When the conclusions drawn in a study does not reasonably reflect the gathered data, the study is said to contain instances of reporting bias. What’s more interesting, however, is that reporting biases can themselves be examined scientifically in another study — a sort of meta-analysis of reporting bias, so to speak.
For example, a 2013 study published in Annals of Oncology analyzes reporting bias in 164 randomized clinical trials for breast cancer between 1995 and 2011. The study found, among others, that:
- Among the 72 clinical trials showing positive treatment results, the primary end point (i.e., targeted biomarker) tends to be emphasized and the toxicity under-reported.
- On the other hand, among the remaining 92 trials showing negative treatment results, 59% of them used secondary end point to suggest possible treatment benefits. In other words, ineffective treatments tend to get a positive spin when reported.
- In overall, only 32% of trials report the frequency of grade 3 and grade 4 toxicities in the abstract of the trial.
While we are on the topic of toxicity, between 2007 and February 2013, Health Canada, Canada’s federal health regulator, received reports from doctors and pharmacists documenting 600 adverse reactions and 23 deaths (mostly due to blood clots) linked to the “newer-generation” birth control pills Yaz and Yasmin. These pills, manufactured by Bayer, are known to contain a hormone known as drospirenone, which, according to a Health Canada safety review, increases the risk of blood clots by 150% to 300%.
In fact, a Canadian class-action lawsuit representing 1000 women outside of Ontario has since then been filed against Bayer. Unsurprisingly, the lawyer behind this lawsuit suspected rampant under-reporting of side effects caused by these pills:
“There are about 30 or 40 deaths that we think are known, but that is usually just the tip of the iceberg because all sorts of people will have died.”
Occasionally though, the outcomes of clinical trials get a negative spin instead, claiming that treatments found to be barely statistically significant are of little clinical values. This, along with the previously mentioned cases, suggests that there are many hidden factors at play here. In particular,
Studies can be spinned from one way to the other, when there are vested interests to do so.
In any case, reporting biases are usually a form of confirmation bias — an attempt to selectively extract only the facts that are consistent with our worldview and belief.
Research publishers tend to be run by groups of people with similar interests and vision. Consequently, it’s not all uncommon for them to favor one study over another.
However, this situation can turn stickier when there is a significant amount of studies struggling to even get their results published and disseminated to the wider community. In fact, we know for one that at least 4 large-scale papers collectively concluded, that around 50% or more of results from clinical trials don’t even make it through the publication stage — and this is regardless of whether they were funded by the industries or not.
But then, it would be unfair to point the finger squarely at the publishers, for in some occasions, it is actually the researchers themselves who decide not to publish the outcome of their own clinical trials — sometimes because they find the outcomes disappointing; other times because the trials abruptly come to a stop…
One obvious implication of these missing clinical trials, is that a treatment could be no more and no less effective than a placebo, yet still appears in the literature mostly as a viable alternative. What’s more, the practitioners and the institutions following the literature could then be tricked into believing the treatment’s effectiveness, and proceed to prescribe it to their patients or integrate such “evidence” into their manuals or medical policies.
For more, here is Dr. Ben Goldacre, the author of Bad Pharma, elaborating on the extent of publication bias in medicine — in particular, how he was misled by the studies on the effectiveness of the antidepressant Reboxetine:
And here’s an audio clip from NPR about rethinking dietary fat, where Dr. Walter Willett, nutrition researcher at Harvard School of Public Health, reminisced about one of his studies which found that replacing saturated fat with carbohydrates does not reduce the risk of coronary heart disease — a study which had a hard time getting accepted by certain medical journals, due to the mainstream bias against meat and saturated fat at that time.
The Catalyst, an ABC (Australia’s public broadcaster) investigative health series, was forced in May 2014 to ban two of their episodes (which explores misconceptions about saturated fats, dietary cholesterol and the cholesterol-lowering class of drugs Statin). The banning occurred in part due to intense pressure from the lobbyists (e.g., mainstream medical community, pharmaceutical industry, National Heart Foundation), who were concerned about raising “controversial” matters which challenge the very conventional health recommendations they themselves have been advocating over the years.
(although no longer officially available, these two episodes can be found here — The viewers are hence free to judge the impartiality of these episodes on their own.)
“All over-diagnosis results in unnecessary surgery, unnecessary chemotherapy, unnecessary radiotherapy and unnecessary hormonal therapy.”
— Dr. Cornelia Baine, a coauthor of the Canadian National Breast Screening study
Let’s move on to another case of resistance in the medical community. In February 2014, the British Medical Journal published the result of a 25-year breast cancer follow-up study (a.k.a. the Canadian National Breast Screening Study) involving 89,835 women from 6 Canadian provinces, who either underwent 5 consecutive years of screening (mammography arm), or didn’t undergo any mammography screening at all (control arm). Surprisingly, the study found that:
- The cumulative breast cancer mortality rates were not significantly different between the two arms (both hazard ratios ~0.99).
- Among the 484 women diagnosed with invasive breast cancer, 106 of them (22%) were being over-diagnosed, resulting in unnecessary surgeries, chemotherapies, radiotherapies and hormonal therapies.
While this study called for a reassessment of breast cancer screening policy in Canada, cancer agencies in the provinces of British Colombia and Ontario have already decided to ignore that advice. In fact, similar advice was also met with fierce resistance in the States in the past, all of which suggest that changing screening policy is indeed as difficult as Dr. Mette Kalager (the author behind the editorial published along with the Canadian National Breast Screening study) predicted:
“This is not an easy task, because governments, research funders, scientists, and medical practitioners may have vested interests in continuing activities that are well-established.”
As with all human endeavours, there are countless ways in which our judgments and decisions can infiltrate the research publications and medical manuals, which in turn could influence our public health policies and standard protocols.
As such, if there is a lesson to be learnt here, it would be the importance of having a system that actually keeps our biases and intents in check. While these human errors are not necessarily undesirable, they could be transmitted from one source to another, such that if we were way of the real targets, they could contaminate our medical science like new strains of viruses, or gear us towards the cliff of looming public health disasters.
While the functional integrity of the system is paramount to our public health, the integrity itself shouldn’t be used as an excuse to abstain from due diligence and self-education. Medical science — as with any other field of knowledge — should be taken with a grain of salt:
- If a study finds a treatment effective, what do we know about its researchers’ background?
- Could the same outcome be replicated by other researchers — especially those from a different school of thought?
- Is the conclusion of the study consistent with our own experience? Is there a way to corroborate the study outcomes directly via repeated self-experimentation? Or indirectly via other means?
- What do we know about the organizations responsible for supplying materials to the researchers?
- Through which channels does the study’s funding come by?
Remember, an attentive guard is fundamentally better than a burglar alarm! While asking poignant questions is not infallible, it still has great potential in helping us uncover some hidden aspects of medicine — aspects that we might have easily overlooked; aspects which could prove to be deadly.
In Part III of this series, we will continue our exploration with mainstream Western medicine — from a human-relationship perspective. 😉
Enjoy this article? Consider liking or sharing it! And if you like our work, consider following or supporting us!