The life cycle of unicorns
« previous post | next post »
Maybe the tide is turning against "Gene for X" thinking — Ed Yong, "A Waste of 1,000 Research Papers", 5/17/2019:
Decades of early research on the genetics of depression were built on nonexistent foundations. How did that happen?
In 1996, a group of European researchers found that a certain gene, called SLC6A4, might influence a person’s risk of depression.
[…]
But a new study—the biggest and most comprehensive of its kind yet—shows that this seemingly sturdy mountain of research is actually a house of cards, built on nonexistent foundations.
That "new study" is R. Border et al., "No Support for Historical Candidate Gene or Candidate Gene-by-Interaction Hypotheses for Major Depression Across Multiple Large Samples", Am J Psychiatry May 2019:
OBJECTIVE: Interest in candidate gene and candidate gene-by-environment interaction hypotheses regarding major depressive disorder remains strong despite controversy surrounding the validity of previous findings. In response to this controversy, the present investigation empirically identified 18 candidate genes for depression that have been studied 10 or more times and examined evidence for their relevance to depression phenotypes.
METHODS: Utilizing data from large population-based and case-control samples (Ns ranging from 62,138 to 443,264 across subsamples), the authors conducted a series of preregistered analyses examining candidate gene polymorphism main effects, polymorphism-by-environment interactions, and gene-level effects across a number of operational definitions of depression (e.g., lifetime diagnosis, current severity, episode recurrence) and environmental moderators (e.g., sexual or physical abuse during childhood, socioeconomic adversity).
RESULTS: No clear evidence was found for any candidate gene polymorphism associations with depression phenotypes or any polymorphism-by-environment moderator effects. As a set, depression candidate genes were no more associated with depression phenotypes than noncandidate genes. The authors demonstrate that phenotypic measurement error is unlikely to account for these null findings.
CONCLUSIONS: The study results do not support previous depression candidate gene findings, in which large genetic effects are frequently reported in samples orders of magnitude smaller than those examined here. Instead, the results suggest that early hypotheses about depression candidate genes were incorrect and that the large number of associations reported in the depression candidate gene literature are likely to be false positives.
The Atlantic article emphasizes the scandalous amount of effort wasted on this false positive:
“What bothers me isn’t just that people said [the gene] mattered and it didn’t,” wrote the psychiatrist Scott Alexander in a widely shared blog post. “It’s that we built whole imaginary edifices on top of this idea of [it] mattering.” Researchers studied how SLC6A4 affects emotion centers in the brain, how its influence varies in different countries and demographics, and how it interacts with other genes. It’s as if they’d been “describing the life cycle of unicorns, what unicorns eat, all the different subspecies of unicorn, which cuts of unicorn meat are tastiest, and a blow-by-blow account of a wrestling match between unicorns and Bigfoot,” Alexander wrote.
Yong suggests the "reproducibility crisis" as the wider perspective:
Many fields of science, from psychology to cancer biology, have been dealing with similar problems: Entire lines of research may be based on faulty results. The reasons for this so-called “reproducibility crisis” are manifold. Sometimes, researchers futz with their data until they get something interesting, or retrofit their questions to match their answers. Other times, they selectively publish positive results while sweeping negative ones under the rug, creating a false impression of building evidence.
Beyond a few cases of outright misconduct, these practices are rarely done to deceive. They’re an almost inevitable product of an academic world that rewards scientists, above all else, for publishing papers in high-profile journals—journals that prefer flashy studies that make new discoveries over duller ones that check existing work. People are rewarded for being productive rather than being right, for building ever upward instead of checking the foundations. These incentives allow weak studies to be published. And once enough have amassed, they create a collective perception of strength that can be hard to pierce.
That's all good, but the author should also have slammed the whole "gene for X" ideology, which remains an influential but spectacularly false idea about how genes work. The "gene for X" ideology is one of the factors making journal editors receptive to questionable research, and motivating the data-dredging and p-hacking and HARKing and drawer-filing that generate papers fitting the editors' memetic receptors.
A sample of previous LLOG discussion of this issue, as it relates to the infamous "language gene":
"Finch phrase structure?", 10/1/2007
"Mice with the 'language gene' stay mum", 6/4/2009
"More on FOXP2", 6/5/2009
"The hunt for the Hat Gene", 11/15/2009
"An invented statistic returns", 2/22/2013
"Sex and FOXP2: Preservation of endangered stereotypes", 2/28/2013
"Gene/culture co-evolution", 6/13/2013
And as it happens, last year there was an essentially parallel corrective result, demonstrating that the FOXP2 hoo-ha (more than 15,000 Google Scholar hits) also depended on a false positive based on over-interpretation of a small and unrepresentative sample. Elizabeth Grace Atkinson et al., "No Evidence for Recent Selection at FOXP2 among Diverse Human Populations", Cell 2018:
FOXP2, initially identified for its role in human speech, contains two nonsynonymous substitutions derived in the human lineage. Evidence for a recent selective sweep in Homo sapiens, however, is at odds with the presence of these substitutions in archaic hominins. Here, we comprehensively reanalyze FOXP2 in hundreds of globally distributed genomes to test for recent selection. We do not find evidence of recent positive or balancing selection at FOXP2. Instead, the original signal appears to have been due to sample composition. […] Our results represent a substantial revision to the adaptive history of FOXP2, a gene regarded as vital to human evolution.
Needless to say, this result has not gotten nearly as much media play as the earlier theories that it falsifies.
Tom S. Fox said,
May 20, 2019 @ 5:51 am
So this paper is the genetics counterpart of the Mueller Report?
Tom S. Fox said,
May 20, 2019 @ 6:00 am
Also, maybe now you will understand why people aren’t convinced when you tell them, “Global warming must be true because of muh research papers!”
(I’m not saying global warming isn’t happening. I’m saying the argument is shit.)
Mark P said,
May 20, 2019 @ 7:38 am
@Tom S. Fox — I'm sure there are reasons that some people question the existence of global warming, but this is not one of them.
David Marjanović said,
May 20, 2019 @ 8:17 am
It did get published in Cell, though, which is one of the most prestigious journals = one of the most prone to rejecting manuscripts as not mind-blowingly groundbreaking enough.
What the vertical gene transfer, man.
Nobody even says "global warming must be true". Global warming is true, as a measured fact. All measurements of the last several decades, and most before that, agree.
The question is how to explain that fact. That's where science comes in. Frankly, though, before you even read any of the resulting papers, consider the following facts:
– We now have 415 ppm of CO₂ in the air, which happens to be the upper end of the confidence intervals reconstructed for the entire Pliocene (y'know, before Greenland and West Antarctica iced over) and contrasts rather starkly with the 400 ppm of 2013, let alone the preindustrial value of 280 (stable for ten thousand years) or even the values of the warmest interglacials (around 320) which we can measure because we have air from those times in bubbles in Antarctic inland ice.
– CO₂ is a greenhouse gas, meaning it absorbs infrared radiation from the Earth at certain wavelengths, heats up in the process and conducts this heat to the air around it. The relationship isn't linear, for a number of reasons that are very well understood by now.
– There is no ¹⁴C in this additional CO₂. That means it comes either from volcanoes, or from fossil fuels. The rate of volcanic outgassing has not increased. The rate of burning fossil fuels has.
– Solar activity can increase the global mean temperature. It hasn't increased since 1940.
– Over deep time, CO₂ has mostly been a feedback on temperature changes rather than simply a cause, because there aren't many natural sources of large amounts of CO₂. But consider the Paleocene-Eocene Thermal Maximum. And then consider the fact that CO₂ is increasing 10 times as fast now as it was then.
Now consider the motivations of scientists. In short: job security, so we* can work on our pet projects in peace. "Gene for X" research promises exciting knowledge with practical applications; that means a grant that would pay a researcher's bills for a few years has a reasonable chance of being funded. There's no exciting knowledge to be had in climatology these days, and "welp, we need to do something" is not a practical application anybody could make money from.
Money, instead, is to be had from contradicting the consensus. That's where the excitement lies. It's also where the interest of oil companies lies, and indeed oil companies have funded the last two or three researchers who, up until 10 years ago, tried to blame the sun (or cosmic rays and their supposed effects on cloud formation).
The conclusion is obvious: it's us. If you'd really like to discuss this further, find me in Google Scholar and drop me an e-mail.
* Disclaimer: I'm a scientist, but not a climatologist.
mikegrubb said,
May 20, 2019 @ 8:49 am
"[…] a blow-by-blow account of a wrestling match between unicorns and Bigfoot." This is clearly nonsense. Everyone knows that unicorns have hooves and can't grapple an opponent. What they need are genes for hands with opposable thumbs.
Mike said,
May 20, 2019 @ 9:15 am
John Horgan refers to "gene-for-x" as "gene-whiz science":
https://blogs.scientificamerican.com/cross-check/dear-skeptics-bash-homeopathy-and-bigfoot-less-mammograms-and-war-more/?redirect=1
Mark P said,
May 20, 2019 @ 10:00 am
Thank you David Marjanović.
I have pointed out on a number of occasions in other places that global warming is not a theory, but an observation. I was too weary to go into it here. My degree is actually in atmospheric sciences, but I am not a climatologist. I have also pointed out in other places that not just anyone is qualified to question the research and conclusions of climatologists, even if they have a degree in a related area. If denialists want a skeptic's view, I sometimes point them to the work of Richard Muller at UC Berkeley, who was a well-known global warming skeptic. He was funded by Charles Koch to redo the global warming research, in my opinion hoping that he would debunk the whole area of research. However, he found that global warming was, indeed, taking place. I suspect that result was not the desired one.
BZ said,
May 20, 2019 @ 11:47 am
I hope you're not discounting every case in which a disease is said to have a single gene mutation as a cause, since I happen to have one that is exactly that.
[(myl) Of course there are disorders caused by a single genomic variant. What don't exist, in general, are individual genes for natural phenotypic features, whether physical (like height) or behavioral (like depression or language).]
Gabriel Holbrow said,
May 20, 2019 @ 12:30 pm
Professor Liberman had a post here on Language Log earlier this month pondering what "trolling" is:
https://languagelog.ldc.upenn.edu/nll/?p=42718
And now, in the comments to this thread, Tom S. Fox has provided (what I consider to be) a textbook demonstration of trolling, and David Marjanović has provided a textbook reaction.
Mr. Fox posted two comments, only glancingly related to Professor Liberman's post here but certainly irrelevant to it, that stated opinions well known to be political controversial. Note that Mr. Fox's posts provide him deniability that the started opinions are actually his own opinions. Mr. Marjanović then took Mr. Fox's bait and exerted considerable effort to argue against one of those opinions. If Mr. Fox thinks his posts and the reaction they got were funny (and I cannot know if he does), then we what we have here is a definitive example of trolling.
Mr. Fox, stop it, please.
Mr. Marjanović, I respectfully suggest that not responding would have been a better reaction.
Darla Weatherford said,
May 20, 2019 @ 1:02 pm
Maybe I'm the 1 misreading Tom S. Fox, but I found both of his statements funny, the first because the Mueller report apparently says things The Powers That Be don't want us to know and is therefore being kept away from us (and the folks who wrote and published those unicorn papers likely feel the same way about this report), and the second because what he said was, "why people aren't convinced when you tell them" about all the papers out there. I guess I didn't think about it much until I started following LL, but I've developed a huge skepticism for "peer-reviewed publication," so I'm among those who don't want to hear the argument that global warming must be true b/c so many papers have been published about it. Instead, I want the kind of argument that David Marjanovic gave us, which sort of makes me think he missed Fox's point but does give me good, solid points (as opposed to "somebody has published a lot of papers!", which *isn't* "solid") for the next time I want to discuss the issue.
Dan said,
May 20, 2019 @ 1:39 pm
@BZ: yes, there are effects caused by single gene mutations. But it turns out they’re all crazy edge cases, not the way genes normally work. It’s just like what happened with nutrition; we figured out that scurvy was caused by lack of vitamin C and rickets was caused by lack of vitamin D and assumed that this could be extended into a large set of micronutrients each with very specific (and entirely independent) effects. And then a century of terrible food science followed.
Kristian said,
May 20, 2019 @ 1:48 pm
Just because people thought that SLC6A4 and some other genes had a correlation with depression (did anyone actually say they were the genes for depression?) and it turns out it probably doesn't, that doesn't make all gene-disease correlation research somehow theoretically invalid, even if a lot of it is exaggerated, especially when it trickles into the popular press. Some diseases have more complicated genetics, others have simpler genetics.
Depression isn't a very easy thing to study, anyway. Consider that hundreds of millions of people have taken anti-depressants (serotonin uptake inhibitors) and there isn't any consensus on whether they have much useful effect or not.
Since SLC6A4 is a gene for a serotonin transporter, researchers were evidently far too keen to believe that it has some key role in depression.
In any case, far from claiming that this kind of research is misguided, the Atlantic article quotes —
"Nor, he says, should his work be taken to mean that genes don’t affect depression. They do, and with newer, bigger studies, researchers are finally working out which ones do."
That does sound a bit funny in the context of the article.
David Marjanović said,
May 20, 2019 @ 3:40 pm
Nah. I wrote the whole comment off the top of my head, nonstop. It hardly took longer than the typing alone did. :-)
The common advice not to feed the trolls comes with a few disadvantages. First, it is easily overapplied to people who actually mean what they say and really came here for an argument (off-topic though it may be) – there's not enough evidence for me to decide, but perhaps Mr. Fox is such a person. Second, it is generally not good to let the claims of trolls stand uncontradicted for posterity. Third, if you ignore actual trolls, they simply ramp up their efforts and become ever more obnoxious till they're either banned or destroy the place. They don't just stop on their own. It works the same way as with bullies in meatspace: we've all been told "ignore them, they just want your attention, don't give it to them" – but if you don't give bullies your attention, they come and grab it.
So, I prefer to feed the trolls till they explode. It has worked on occasion.
I thought there was a consensus: they work differently on different people, at one extreme spectacularly well on some and at the other extreme not at all on others?
Thank you.
Jen in Edinburgh said,
May 20, 2019 @ 4:41 pm
I thought Tom S Fox had a point, and I thought David Marjanović missed it, although his comment was interesting and informative in itself.
Things like this, and the Voynich manuscript case, show that being published, or being peer reviewed, or being published separately by lots of different people, aren't guarantees of truth, or even of reliability. Some of that is just science, zigzagging its way towards an answer by trying the dead ends. Some of it is people – working scientists and publishers – twisting science (however gently) to their own ends, whether to make a name or to make a living.
In this case, lots of people have said the same thing, and we are asked to believe that the one who comes along afterwards and says something different is telling the real truth.
In the case of global warming, lots of people have said the same thing, and we're expected to believe that anyone who comes along and says different is a fool or a charlatan or a liar.
Why? As Darla Weatherford said, 'lots of people say this' or 'lots of people have published papers that say this' are obviously not good arguments in themselves.
The true difference between the two cases, I suppose, is that given the measurements, or the means of taking the measurements, you – if you knew how – would come to the same conclusions yourselves in the case of global warming, and probably wouldn't in the case of the gene.
But if you don't know how, and the only people who could teach you are the same people whose conclusions you are trying to test, where do you start?
[(myl) At least in the case of FOXP2, there were many sensible people who warned against some of the various forms of over-interpretation from the beginning — including Simon Fisher, the discoverer of the gene.]
Christopher J. Henrich said,
May 20, 2019 @ 4:50 pm
I found David Marjanovic's posting interesting and useful.
Bathrobe said,
May 20, 2019 @ 5:25 pm
I suspect this post applies to 'scientific claims' that pop up with monotonous regularity, such as:
Altruism has a genetic basis.
People have a gene that makes them sensitive to cheating (they are genetically disposed to identify behaviour whereby someone takes more than their fair share).
Infidelity has a genetic basis.
[(myl) It's important to distinguish two quite different ideas here:
(1) might well be true when (2) is false, because "the genetic basis for X" might be a complex interaction among hundreds of genes and many environmental factors. The relevant notions of "heritability" are tangled at best, but they're less trivially and transparently false than (nearly all instances of) the "gene for X" meme.]
J.W. Brewer said,
May 20, 2019 @ 8:38 pm
Multiple generations of schoolchildren (at least in the US and probably elsewhere as well) have been taught a simplified model of Mendelian genetics, where the paradigm case is always trait X is caused by a single gene and if it's recessive who you need to get that gene from both parents but if it's dominant only from one (meaning you can't tell from exterior appearances whether someone who lacks the trait is or isn't a carrier of the gene). This was recently reiterated to me by my 9th-grader, who is the only one of my kids who exhibits my (recessive, someone told her) trait of attached rather than detached earlobes.
Unless/until basic K-12 education does a better job of explaining how lots of things that do have a hereditary component are so complex and polygenic that that simplified Mendelian model is more likely to confuse than illuminate, "gene for X" stories will continue to gain traction. According to this https://udel.edu/~mcdonald/mythearlobe.html, even earlobes are not the result of a "gene for X," but whoever has been teaching science to future K-12 science teachers off in the schools of education may not be aware of that.
The said,
May 20, 2019 @ 9:34 pm
The question is how to explain that fact. That's where science comes in. Frankly, though, before you even read any of the resulting papers, consider the following facts:
You appear to be struggling with the concept that an apparently solid structure can be built on what turns out to be fundamentally flawed premises. As the original poster pointed out, he didn't deny. He just isn't impressed that a lot of people say the same thing. Listing a whole host of facts, even correct ones, does not make a theory true of itself. You're kind of proving the original statement true, in fact.
Marxist economics was *the* most scientific theory ever to those behind the Iron Curtain. They could wheel out any number of facts to back it up. And many believed in the theory passionately. Some still do, in fact. (They conversely, of course argue that Capitalist economists are in the position I accuse the Marxist ones of being in.)
Andrew Usher said,
May 20, 2019 @ 9:43 pm
I have no doubt that school concept does play a role; however, if people thought scientifically they'd use it in reverse and realise that if there's a 'gene for X' then X should almost always obey simple Mendelian inheritance. Of course the things being talked about here don't even come close.
Everything that involves humans can be expected to 'have a genetic basis' in that sense, so I think we're maybe too broad to be meaningful there. I don't think any bright-line definition would please everyone that assigned certain things to 'genetic' and others not; even being perfectly rational you're losing information badly by doing that.
In the case of global warming, ther problem is again that lay sceptics don't think like a scientist in understanding the issue; they know scientists can be wrong, even by consensus on occasion, they know there is money in the issue (there's money in literally every science) and that at least some of it is on the pro-climate change side, and it's not too difficult to come to the unfortunate conclusion without being too irrational. It doesn't help that hardly anyone seems able to distinguish the science from policy issues. Result: global warming in the US becomes another political 'deadlock' issue, no matter how ridiculous it may seem (and frankly, they all are).
k_over_hbarc at yahoo.com
KeithB said,
May 21, 2019 @ 4:35 pm
The list of scientific concepts that are simplified to the point of wrongness are legion. (Quantum Mechanics, anyone?)
I had a lady at work who said that the Big Bang could not have created the Earth because all the explosives she had worked with cause things to blow apart, not come together.
David Marjanović said,
May 23, 2019 @ 5:54 am
That's what I was trying to quickly explain and demonstrate. Evidently too quickly. :-)
Scientific ideas don't "list" facts, they explain facts. A theory is better than another if it explains more facts in a simpler way, "simple" here meaning "parsimonious", which in turn means "with the smallest number of additional assumptions".
Oil-funded "skeptical" scientists have indeed tried to explain some of the facts I listed. They have never come close to explaining the whole package, let alone in a simpler way than I outlined.
Nobody has claimed that fervent belief is a criterion we should use. Marxist economics, AFAIK, follows logically from a few premises, some of which are inaccurate…
Few people appreciate how little money there is in any science except those that promise marketable practical applications (medical research in the widest sense, petroleum geology, and not much else) and those that promise to promote the goals of somebody with a lot of money (research that promises to show that the product of a big corporation is safe).
Few people understand how a grant works. The number of dollars in a grant, particularly in the US, can be quite staggering, but almost all goes into the research expenses themselves or into meager salaries for ramen-eating grad students and desperate postdocs, leaving quite little for the Principal Investigator who submitted the grant. There's a great video on YouTube by a climatologist that breaks down the numbers; unfortunately I don't have time to look for it right now.
The kind of fame that leads to a bit more money comes from publishing in the most prestigious journals (as measured by their impact factor), and those journals reject all manuscripts that aren't mind-blowingly groundbreaking enough. Of two manuscripts whose research quality is equal, one that contradicts the current consensus has much better chances of being published than one that merely confirms it.
Indeed it didn't :-)
Mark Meckes said,
May 23, 2019 @ 8:52 am
This SMBC ("The Satan Gene") is relevant here.