CLOSE
Kim J, Matthews NL, Park S via Wikimedia Commons // CC BY 2.5
Kim J, Matthews NL, Park S via Wikimedia Commons // CC BY 2.5

Software Flaws May Invalidate 40,000 Neuroscience Studies

Kim J, Matthews NL, Park S via Wikimedia Commons // CC BY 2.5
Kim J, Matthews NL, Park S via Wikimedia Commons // CC BY 2.5

The human fallibility of science has been in the news a lot lately. The latest hit? Researchers say the three most popular programs for interpreting functional magnetic resonance imaging (fMRI) results all produced shockingly high rates of false positives—a finding that could invalidate tens of thousands of studies. The report by researchers from Sweden's Linköping University and the UK's University of Warwick was published in the Proceedings of the National Academy of Sciences.

Over the last two and a half decades, fMRI has become a crucial tool for investigating the brain and its inner workings. Because our bodies supply blood to the parts that need it most, fMRI’s ability to measure blood flow in the brain allows us to track brain activity. But just how good of a job is it doing? Scientists weren’t really sure.

“Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data,” the authors write. 

To test it, they pulled data on 499 healthy people from the 1000 Functional Connectomes Project, a database of baseline brain scans. They then split their subjects into different subgroups and compared the scans in different permutations, over and over, until they had 2,880,000 different group analyses.

For a long time, neuroscientists have believed that scanning the brain of a healthy person in a resting state should yield no signs of activity. Consequently, any sign of activity could be read as a positive result.

But the authors of the current study say that assumption could be wrong, and following it could lead to a lot of false positives.

Looking at the results of their almost 3 million comparisons, the researchers expected to find a false positive rate of about 5 percent. Instead, they found that the top three fMRI analysis programs—Statistical Parametric Mapping, the fMRIB Software Library, and Analysis of Functional Neuroimages—yielded false positive rates up to 70 percent. That’s right: seven out of 10 scans analyzed using these methods came to inaccurate conclusions. And the authors say that may be a conservative estimate.

If these findings are correct, it has a huge impact on neuroscience research. In a literature search, the authors found more than 40,000 neuroscience studies that relied on fMRI results. Still, there may be no point in going back and re-doing these studies. It’s better to look to the future, the authors say: "Considering it is now possible to evaluate common statistical methods using real fMRI data, the fMRI community should, in our opinion, focus on validation of existing methods.”

Know of something you think we should cover? Email us at tips@mentalfloss.com.

nextArticle.image_alt|e
iStock
arrow
Health
The First Shot to Stop Chronic Migraines Just Secured FDA Approval
iStock
iStock

Migraine sufferers unhappy with current treatments will soon have a new option to consider. Aimovig, a monthly shot, just received approval from the Food and Drug Administration and is now eligible for sale, CBS News reports. The shot is the first FDA-approved drug of its kind designed to stop migraines before they start and prevent them over the long term.

As Mental Floss reported back in February before the drug was cleared, the new therapy is designed to tackle a key component of migraine pain. Past studies have shown that levels of a protein called calcitonin gene–related peptide (CGRP) spike in chronic sufferers when they're experiencing the splitting headaches. In clinical trials, patients injected with the CGRP-blocking medicine in Aimovig saw their monthly migraine episodes cut in half (from eight a month to just four). Some subjects reported no migraines at all in the month after receiving the shot.

Researchers have only recently begun to untangle the mysteries of chronic migraine treatment. Until this point, some of the best options patients had were medications that weren't even developed to treat the condition, like antidepressants, epilepsy drugs, and Botox. In addition to yielding spotty results, many of these treatments also come with severe side effects. The most serious side effects observed in the Aimovig studies were colds and respiratory infections.

Monthly Aimovig shots will cost $6900 a year without insurance. Now that the drug has been approved, a flood of competitors will likely follow: This year alone, three similar shots are expected to receive FDA clearance.

[h/t CBS News]

nextArticle.image_alt|e
iStock
arrow
Medicine
The 98.6℉ Myth: Why Everything You Think You Know About Body Temperature Is a Lie
iStock
iStock

When you were kid, you probably knew that to score a magical sick day home from school, you needed to have a fever. When the thermometer came out of your mouth, it had to read higher than 98.6℉—the long-accepted "normal" human body temperature. (If you wanted to really seal the deal, you may have hoped to hit 100℉.) Since then, you may have used a temperature above 98.6℉ as a metric to work from home (or call out sick entirely).

But here's the thing: The average body temperature isn't actually 98.6℉—a fact that we've known for more than 25 years. The myth originated in the 19th century with a single doctor, and despite evidence to the contrary, it's persisted ever since.

THE GIANT—AND FAULTY—ARMPIT THERMOMETER

In 1851, Carl Wunderlich, the director of the hospital at Leipzig University, began going from room to room with a comically large thermometer in tow. He wanted to understand how body temperature is affected by different diseases, so in each room, he would hold the foot-long device in patients' armpits for a full 20 minutes, waiting for a temperature to register. Once it did, he'd note the temperature on the patient's chart (Wunderlich is thought to be the first physician to do so). He and his staff did this for years, repeatedly taking the temperatures of some 25,000 patients and logging them on their charts, until he had millions of readings. In 1868, he finally published this data in Das Verhalten der Eigenwarme in Krankheiten (On the Temperature in Diseases: A Manual of Medical Thermometry). He concluded that the average human body temperature was 98.6℉, underscoring the idea that fever is a symptom of illness, not a cause.

No one questioned Wunderlich's methods, or his average, for about 140 years. Then, in the early 1990s, internist Philip Mackowiak—a professor of medicine at the University of Maryland, a medical historian, and, apparently, a clinical thermometer junkie—saw one of the physician's instruments at the Mutter Museum in Philadelphia. He told the Freakonomics podcast that he'd always had doubts about the 98.6℉ standard. "I am by nature a skeptic," he said. "And it occurred to me very early in my career that this idea that 98.6 was normal, and then if you didn't have a temperature of 98.6, you were somehow abnormal, just didn't sit right."

Getting his hands on Wunderlich's thermometer—which the museum let him borrow—only deepened his doubts. The huge thermometer was unwieldy and non-registering, meaning, Mackowiak explained, "that it has to be read while it's in place." Not only that, but Wunderlich had used the device to measure temperatures in the armpit, which is less reliable than temperatures taken in the mouth or rectum. The instrument itself also wasn't terribly precise: It measured up to 2 degrees Centigrade higher than both ancient and modern instruments.

In 1992, Mackowiak decided to test Wunderlich's average. Using normal-sized oral thermometers and a group of volunteers, he determined that the average human body temperature actually hovers around 98.2℉. Mackowiak found that body temperature tends to vary over the course of the day, with its lowest point around 6 a.m. and its highest in the early evening. Body temperature can also fluctuate monthly (with the menstrual cycle) and over a lifetime (declining decade by decade with age), and may even be differentially linked to sex and race assignments. He concluded that normal body temperature is so unique to each person that it's almost like a fingerprint and, given that wide variation, not actually a very reliable indicator of illness.

As a result of his study, Mackowiak proposed raising the threshold for fever to 98.9℉ for temperatures taken in the morning (and 99.9℉ at other times). While it's a relatively minor change in terms of actual degrees, this fever threshold is actually lower than the CDC's, which is a temperature of 100.4℉ or higher.

There are potential real-life consequences in this gap, for everyone from students (who'd have to attend school with what would be considered a low-grade fever by Wunderlich's 98.6℉ standard) to employers and daycares (who use temperature to set attendance policies). What's more, anyone who is actually sick but ignores a low-grade fever—one that meets Mackowiak's threshold but still falls under the CDC's—could pose a risk to people with compromised immune systems trying to avoid unnecessary exposure to illness in public places.

THE BALANCING POINT

There's a reason the average trends near 98℉ instead of 92℉ or 106℉. As endotherms, mammals expend a great deal of energy maintaining body temperature when compared with cold-blooded creatures. To find and conserve a just-right body temperature, central nervous system sensors gather data (too warm? too cold? just right, Goldilocks?) and send that information to the pebble-sized hypothalamus near the base of the brain. There, the data is converted into action: releasing sweat and widening the blood vessels if too warm; raising metabolism, constricting the blood vessels, and inducing shivering if too cold.

According to a study by Aviv Bergman and Arturo Casadevall in the journal mBio, the precise balancing point for ideal body temperature is the sweet spot where the metabolic cost for all this thermoregulation balances with the evolutionary advantage of warding off fungal disease. (While warm-blooded animals are prone to bacterial or viral infections, they rarely experience fungal infections because most fungi can't withstand temperatures above 86℉. Cold-blooded animals, on the other hand, are prone to all three.) For Bergman and Casadevall, this benefit even explains what tipped Darwin's scales in favor of mammals, allowing them to edge out other vertebrates for dominance after the Cretaceous-Tertiary mass extinction wiped out the dinosaurs.

Of course, rules call for exceptions, and the one place where human body temperature demonstrates sustained elevation is outer space. Astronauts on prolonged missions clock significantly higher average body temperatures than they do when terrestrial—even up to 104℉. This so-called "space fever" is probably a product of some combination of radiation exposure, psychological stress, and immune response to weightlessness. Researchers believe this phenomenon could yield crucial information about thermoregulation—and may even offer insight into how humans might adapt to climate change.

WHY THE MYTH PERSISTS

It's been 26 years since Mackowiak's study, yet the newer data has not taken hold among medical professionals or the public. What gives?

Mackowiak tells Mental Floss that he finds it a bit mystifying that the myth persists, especially since many people, when pressed, know that the so-called "average" temperature varies. Part of the problem may be psychological: We cling to beliefs despite evidence to the contrary—a phenomenon called belief perseverance [PDF]. It's a significant force upholding a surprising number of medical myths. The idea humans should drink eight glasses of water a day? Not science. Sugar causes hyperactive behavior? Nope. Reading in dim light harms eyesight? Not really.

Unlearning persistent myths—especially ones loaded with the weight of medical authority—is difficult. "Deep down, under it all," Mackowiak says, "people want simple answers for things."

SECTIONS

arrow
LIVE SMARTER
More from mental floss studios