LSA solicits ClinicalTrials.gov responses
« previous post | next post »
[Below is a guest post by Matt Goldrick on behalf of the Linguistic Society of America]
Over the past decade the lack of transparency in research – and its implications for the reproducibility of research findings – has been a major focus of scientists and funding agencies (see previous discussions on LanguageLog here, here, here). This has led to many exciting developments in the social sciences including the founding of no-fee platforms for sharing and pre-registering studies (e.g., the Center for Open Science, AsPredicted) and new professional sciences promoting transparent, open research practices (e.g., the Society for the Improvement of Psychological Science). These positive developments have, unfortunately, lead to an overzealous reaction from the US National Institutes of Health (NIH) which is likely to hamper basic science (more info below). The LSA, in partnership with other research organizations, is asking for your help in pushing back against policies that could hamper the work of NIH-funded linguists.
Background: The NIH decided to address issues of transparency by expanding the use of ClinicalTrials.gov, a mechanism for reporting the design and results of medical research studies. The definition of a “clinical trial” was revised as follows:
“A research study in which one or more human subjects are prospectively assigned to one or more interventions … to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.”
These terms are so vague (“behavioral outcome”) as to potentially apply to most if not all basic science research. This overapplication of a medical model provoked strong pushback from individual scientists and professional societies fearing a greatly increased bureaucratic burden and, most critically, potential roadblocks to supporting basic science research (by restricting the types of funding opportunities these “clinical” studies can be supported by).
The feedback from basic scientists lead to a seemingly productive dialogue with the NIH and, importantly, a provision in the 2018 omnibus spending bill that directed the NIH to delay implementation of the new rule and consult with the research community. (The Federation of Behavioral and Brain Sciences [FABBS] has a comprehensive overview of the debate.)
Where the LSA needs your help: As directed by Congress, the NIH has delayed enforcement of the rule and is requesting input from the community. However, they have also made it clear that they are not backing down from their approach to these issues. We want to make sure the NIH hears from all the scientists that might be impacted. For example, you might want to review the interventional study protocol template to see what you’d be expected to enter when registering you basic science study.
To be clear, our community embraces transparent and open research practices, but there is real concern ClincalTrials.gov is not the proper mechanism for implementing these principles in basic science research. The window for such comments is relatively short (ends November 12), so please respond quickly. You can find information about how to do this here. Thanks for your help!
[Above is a guest post by Matt Goldrick on behalf of the Linguistic Society of America]
DWalker07 said,
August 20, 2018 @ 9:53 am
Typo: "These positive developments have, unfortunately, lead to…"
A said,
August 20, 2018 @ 5:57 pm
@DWalker07
Maybe it's selection bias, but I feel like I've been seeing past-tense "lead" (by analogy with "read"?) more often lately. At what point does this shift from a mere error to a new standard?
tangent said,
August 20, 2018 @ 11:22 pm
Syntax question: doesn't "health-related" restrict "… behavioral outcomes"? If so, how is linguistics colorably health-related?
Barbara Phillips Long said,
August 20, 2018 @ 11:39 pm
@tangent: I read the wording as ambiguous. Is it "(health-related biomedical) or (behavioral) outcomes" or is it "health-related (biomedical or behavioral) outcomes"? As an editor, I'd be asking the writer for clearer wording.
My concern would be that all behavioral studies would be impacted, or that proving that a behavioral study was not health-related would impose significant burdens on researchers. Since I don't do any research, I am sure actual researchers can see any number of other pitfalls I have not.
Rodger C said,
August 21, 2018 @ 7:00 am
Maybe it's selection bias, but I feel like I've been seeing past-tense "lead" (by analogy with "read"?) more often lately.
I can't speak to changes in frequency, but I've been seeing it all my life. It's motivated, surely, not only by analogy with "read/read" but by interference from "lead" the metal.
Trogluddite said,
August 21, 2018 @ 5:25 pm
> Tangent: "If so, how is linguistics colorably health-related?"
The first paragraph of the AAAS article (linked at "strong pushback") suggests one area; research into developmental conditions and learning disabilities. For example; language delays are a common characteristic of autism, and can take a huge variety of forms, as the semantic, prosodic, pragmatic, verbal, written etc. aspects of language are often affected independently. There is much yet to be discovered about the neurological basis for these delays, how they interact with other autistic traits and each other, their genetics and heritability, their social effects, the influence of environment, what they can tell us about typical language development, and so on.
Given a broad enough reading of the guidelines, any of these could be deemed to be researching diagnostic markers or therapeutic interventions, when the intent is merely to test a hypothesis. A small exploratory study into a new hypothesis might then be held to a standard which requires more resources than is economic.
Improving the quality and accountability of health research is surely something that none of us would argue with. But it would be unfortunate, and rather ironic, if novel health interventions were stifled because the guidelines hamper the small-scale studies seeking evidence with which to justify the clinical trials which the guidelines intend to improve. It is not just theoretical researchers who might be disadvantaged.
Andrew Usher said,
August 21, 2018 @ 6:37 pm
On past-tense 'lead':
I assume it's been around a long time, but I have noticed it more, too. It seems to be on its way to becoming the 'Internet standard', just like spacing paragraphs instead of indenting is. Traditional editing will continue to demand 'led' as they continue in other obsolescent style conventions (like indenting paragraphs, just mentioned). But 'lead' is not merely a mistake (and almost certainly never one of typing) but a legitimate analogy with other spellings in the language. Indeed, 'led' is an exception, probably standardised specifically to provide an orthographic distinction from the metal, to this general rule: the spelling 'ea' is always preserved when the vowel is shortened, while the spelling 'ee' is changed to 'e'. I've noticed past-tense 'plead' as well, following the same rule (Of course, there the prescriptive standard is 'pleaded' – but nearly everyone _says_ /plɛd/.)
This is not only seen in 'read' but the irregular verb forms 'leapt' and 'dreamt' (never *'lept', *'dremt'), and there used to be more (contrast 'kept', 'sped' – the 'ee' spelling is not preserved). E.g. 'eat' and 'beat' followed the pattern of 'lead', but past tense 'eat' has been replaced with a different variant (the pron. took a long time to follow in many dialects, though), while 'beat' has changed (back) to the long vowel.
As much importance as I place on writing and speaking 'properly', I don't mind changes that make the language more regular and don't really cause any problems, and past-tense 'lead' (and 'plead') seem to fall into that category; though not as clearly as 'alright' which one has to be ignorant or stupid to oppose (it's a different word than 'all right', folks, with a different pronunciation for many including me).
On the subject of regulation:
I think you err if you have much confidence in the government's policies being a result of being insufficiently informed about your field. Knowing what I know about regulatory bureaucrats, they lose their sense of perspective when it comes to their field of regulation (as do some others that interact with them, unfortunately) – they can't see the burden of their regulation, while they see the (theoretical) benefit magnified out of proportion. Most people would like to be exempt from regulations, and they normally do have at least some good arguments – but those arguments don't work if the only people to use them on are the regulators themselves.
Now I truly believe that academics (except medicine) really ought to be generally exempted from regulations, but that's hopeless given how academia works now. You can't except any support from administration, nor from faculty in other fields, nor from the general public which has too little interest. I suppose you have nothing to lose by trying to argue that linguistics is different and ought not to fall under these regulations – I see no other hope. But don't be surprise if, given the history of regulatory extension to other academic fields, in the next generation not only have these regulations been imposed but new faculty actually believe in them, or pretend to.
k_over_hbarc at yahoo.com
Matt Goldrick said,
August 22, 2018 @ 11:37 am
Hello all —
I'm pleased that my usage triggered some interesting conversations!
To make the context of the regulations clearer, these are not placing restrictions on *any* type of research in linguistics. The regulations are only placing restrictions on research that is funded by the National Institutes of Health. This includes (some) research in linguistics as well as some of the research in many other fields: behavioral sciences (e.g., psychology, communication sciences), neurosciences, etc. You can see an example of my own NIH funded work here.
The argument is not that linguistics alone is different, or that psychology alone is different, or that neuroscience alone is different; it's that all basic science is different from clinical studies. Specifically, the argument is that the regulations developed for clinical studies should not be uniformly applied to all research funded by the NIH.
There are many regulations that already apply to basic science. For example, all studies of humans must be approved by an Institutional Review Board to ensure that the rights of participants in research studies are protected. I believe many behavioral scientists believe, like me, that there could be substantial benefits from additional regulations that required more reporting of research results outside of the traditional academic paper format. But there are many ways of meeting that goal that do not require fitting all basic science research into a clinical straightjacket.
mg said,
August 22, 2018 @ 12:32 pm
I'm a medical researcher and I fully support LSA's position. I think the requirement for transparency in clinical research is very important, and clinicaltrials.gov is an important step in fighting the effects of publication bias (positive findings are much more likely to be published than negative ones, which is a major problem).
However, entering studies into ct.gov is onerous and time consuming (therefore expensive). We spend a large number of person-hours entering our studies when they start, and then the results after they conclude. The deadlines are very strict and late penalties punitive. The site is not set up to accommodate non-standard designs.
I would not want us to not have to enter clinical trials into ct.gov, but agree that extending it to basic science research is definitely not a good idea and will add needless cost and difficulty to studies that are already underfunded.
Andrew Usher said,
August 23, 2018 @ 9:55 pm
OK. Their hearing from scientists in other fields would be definitely a good thing, and certainly publication bias is much more a concern is clinical trials than anywhere else (not least because of the possibility of influence by drug companies). Truly, the conventional publication model has become a problem in all academics but it's certainly not going to be solved this way.
Of course I know about IRBs, but I imagine they're more flexible than government regulations normally are, e.g. for linguistics they probably mainly ensure standard rules for consent and privacy, and refrain from scrutinising the experimental procedure as they perhaps might in psychology.
In any case I posted, as always, to support a general argument and am therefore not overly concerned with details; naturally, those directly affected are likely to differ in that respect and that's fine.