When is research "deemed not research", and why?
« previous post | next post »
This post is going to delve into one aspect of a recently-published article from the Centers for Disease Contol: DeCuir J, Payne AB, Self WH, et al. Interim Effectiveness of Updated 2023–2024 (Monovalent XBB.1.5) COVID-19 Vaccines Against COVID-19–Associated Emergency Department and Urgent Care Encounters and Hospitalization Among Immunocompetent Adults Aged ≥18 Years — VISION and IVY Networks, September 2023–January 2024. MMWR Morb Mortal Wkly Rep 2024;73:180–188.
That's the "Suggested citation" given within the linked article. To get past the citation, you'll need to expand some abbreviations:
- "Morb Mortal Wkly Rep" means "Morbidity and Mortality Weekly Report".
- "VISION" is a backronym for "Virtual SARS-CoV-2, Influenza, and Other respiratory viruses Network", and references a "a multisite, electronic health records (EHR)–based network including 369 EDs and UCs and 229 hospitals in eight states that uses a test-negative, case-control design to estimate COVID-19 VE".
- "IVY" stands for "Investigating Respiratory Viruses in the Acutely Ill', and references "a multisite, inpatient network including 26 hospitals in 20 U.S. states that uses a test-negative, case-control design to prospectively enroll patients with COVID-19–like illness (CLI) who receive testing for SARS-CoV-2 within 10 days of illness onset and 3 days of hospital admission".
There's a somewhat de-jargonized version of the article's content by Ian Ingram at MedPage Today — "Latest COVID Shots Protect Against Serious Outcomes — Effectiveness against COVID-related hospitalization ranged from 43% to 52%" — which explains that the article means you probably want to consider getting one of the "recently updated COVID vaccines".
But what I've been trying to decode is only indirectly about the article's content.
The last bit of the article's Data Analysis section tell us:
Analyses were conducted using R software (version 4.3.2; R Foundation) for the VISION analysis and SAS software (version 9.4; SAS Institute) for the IVY analysis. This activity was reviewed by CDC, deemed not research, and was conducted consistent with applicable federal law and CDC policy.**** This activity was reviewed and approved as a research activity by one VISION site.
And the four asterisks take us to the following endnote [with links added to the two cited C.F.R. sections and three U.S.C. sections]:
**** 45 C.F.R. part 46.102(l)(2), 21 C.F.R. part 56; 42 U.S.C. Sect. 241(d); 5 U.S.C. Sect. 552a; 44 U.S.C. Sect. 3501 et seq.
The cited regulations and laws are meant to explain how "this activity … was conducted consistent with applicable federal law and CDC policy". The specific point of interest to me was how and why the described activity is "deemed not research".
We can learn about this in 45 C.F.R. part 46, 102(l), which starts by defining "research" in a way that actually seems to cover the described activity: "Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge."
However, there are four subsections defining specific exceptions to this definition — "For purposes of this part, the following activities are deemed not to be research", and the second of them is
(2) Public health surveillance activities, including the collection and testing of information or biospecimens, conducted, supported, requested, ordered, required, or authorized by a public health authority. Such activities are limited to those necessary to allow a public health authority to identify, monitor, assess, or investigate potential public health signals, onsets of disease outbreaks, or conditions of public health importance (including trends, signals, risk factors, patterns in diseases, or increases in injuries from using consumer products). Such activities include those associated with providing timely situational awareness and priority setting during the course of an event or crisis that threatens public health (including natural or man-made disasters).
As normal lexicography, this is weird — it's like defining mammal in something like the usual way ("A warm-blooded animal that has hair and produces milk to feed its young") and then specifying that certain kinds of mammals, like "Angora goats with brown coat color", are specifically excluded from the category.
But as legal lexicography, this seems to be normal. The motivation, I think, is to avoid more complex re-writing of a law or policy.
In this case, 45 C.F.R part 46 defines policies for the "protection of human subjects", including various sorts of procedures for approving and constraining various stages of various sorts of activities involving those "human subjects". These (evolving) procedures are highly (if variously) complex, and specify elaborate chains of documentation and approval, as well as complex constraints on what sorts of information can be released to whom and when. In that context, the motivation for the four exemptions is clear.
Consider the first exception, covering "Scholarly and journalistic activities (e.g., oral history, journalism, biography, literary criticism, legal research, and historical scholarship)", where recordings and transcripts are often published along with the names of subjects. When the humanities and social sciences first came under the purview of academic Institutional Review Boards, there were several cases where IRBs blindly applied the normal rules for interviews in clinical studies, insisting that the names of oral history subjects could not be published, and that interview recordings and transcripts had to be entirely deleted after studies presenting general conclusions about the "research" in question were completed. (See this 2015 post…)
It would be very complicated to re-write the laws and regulations so that oral history is subject only to appropriate constraints in all of the many places where constraints and procedures are specified — instead, the responsible authorities apparently just decided that oral history is actually just not "research", for the purposes of the regulations in question. (At least they didn't decide that oral history interviewees are to be deemed not human — though they could plausibly have stipulated that they are not "human subjects"…)
As for exemption number 2, covering the cited C.D.C. article, the reasoning seems to be that such work is important and needs to be done and published quickly, without the potentially time-consuming approval process. (This one is the "Morbidity and Mortality Weekly Report"…) It also happens to be true that the collection and submission of the underlying data was covered by multiple institutional approvals, though the regulation doesn't seem to require that.
And of course, court cases often involve complex lexicographical arguments, in ways that sometimes seem to defy common sense. (See e.g. "Is a fish a "tangible object'?", 4/30/2014; "A result that no sensible person could have intended", 12/8/2005; etc.)
I assume the laws and policies in other traditions (e.g. Napoleonic) use similar techniques, for similar reasons — readers will no doubt be able to enlighten us.
Dan Romer said,
March 12, 2024 @ 6:32 am
Hi Mark,
You are justified in expressing confusion about the use of the term research. But this is mainly a way that regulatory bodies, such as IRBs, can make exceptions for certain kinds of "research" so that those studies do not have to be reviewed in order to proceed. This happens all the time and I assume that CDC took note of this in order to not have to include a statement about ethical review.
But for the lay audience, this has to be a source of confusion :)
Cervantes said,
March 12, 2024 @ 6:58 am
Yes, this is an odd locution, but the point is that it is exempt from oversight by an Institutional Review Board. Normally they would say it isn't "human subjects research," because it doesn't use data from individually identifiable people. In this case, although the people were identifiable at some point along the chain of data collection, by the time the data made it to this analysis they were not. It was probably easier to define the activity as public health surveillance and "not research," than to declare it exempt on other grounds. It's just a technicality. Legal language is often different from vernacular, which tends to annoy people, but it's common.
Mark Liberman said,
March 12, 2024 @ 7:02 am
@Dan Romer: "This happens all the time and I assume that CDC took note of this in order to not have to include a statement about ethical review."
I recognized immediately that the phrase "deemed not research" meant essentially that the activity was not subject to human-subjects restrictions. The goal of my investigation was a hermeneutical one, to find chapter and verse for how and why that wording was the appropriate one.
And it's worth noting that the alternative would have involved multiple documents and layers of approval, for several stages of the process.
I've known for some time that oral history research is exempt from the IRB approval process, but I didn't know that this is because it's officially "deemed not research"…
J.W. Brewer said,
March 12, 2024 @ 8:42 am
A very basic principal of the legal-jargon register of American English is that a defined term means whatever it is defined to mean, which, if it's a common everyday word, almost always means something different in scope than the common word otherwise does.
Here's a random example from a federal statute that's currently in another open window on my computer: 'The word “States”, as used in this section, includes the Territories, the District of Columbia, and the Commonwealth of Puerto Rico.' That doesn't mean that Puerto Rico is really a State of the Union in some ontological/constitutional sense, only that it is to be treated the same way as e.g. Rhode Island is for purposes of the operation of the particular statute and this is a convenient way of wording things.
Sometimes it's better practice in a situation like that of the original post (where the ordinary meaning is being contracted rather than expanded) to come up with a vaguely arbitrary term, like "Covered Research" or "Applicable Research" or maybe (euphemistically!) "Eligible Research." That has the benefit of reminding the reader that the relevant meaning is somehow different from the ordinary meaning of small-r research and, precisely because the meaning of the phrase is not intuitive, you've gotta to look up exactly what it means.
Leslie Katz said,
March 12, 2024 @ 10:15 am
Perhaps you'll like a (fictional) quotation from an English judge: "There is too much of this damned deeming". It's from AP Herbert's More Uncommon Law. (As a judge, I inserted the quotation into my reasons for judgment once.)
AntC said,
March 12, 2024 @ 3:00 pm
A very basic principal of the legal-jargon register of American English is that a defined term means whatever it is defined to mean, which, if it's a common everyday word, almost always means something different in scope than the common word otherwise does.
"whatever it is defined to mean" is also in words, right? So how do we tell in reading a complex passage which words are the definiens (not to be taken in their everyday sense); which are the definition (must be taken in their everyday sense)?
Or is the ultimate purpose to reduce everything to semantic mush that only a lawyer is allowed to understand?
Yes, AP Herbert and Mr Justice Cocklecarrot seem close at hand.
Mark Liberman said,
March 12, 2024 @ 4:26 pm
@J.W. Brewster: "A very basic principal of the legal-jargon register of American English is that a defined term means whatever it is defined to mean, which, if it's a common everyday word, almost always means something different in scope than the common word otherwise does."
I'm aware — but it wasn't obvious in advance to me how the "Morbidity and Mortality Weekly Report" was "deemed not to be research". Presumably an exception clause in the definition of "research" is how it was done, but I wondered how and where it happened.
And I thought the answer was a fun lesson in legal lexicography.
J.W. Brewer said,
March 12, 2024 @ 4:46 pm
@AntC: Two ways, at least.
1. the definition will accompany or precede the text's very first use of the defined term, and/or there will be a separate section of the document labeled as containing all of the definitions.
2. The defined term may be consistently capitalized or otherwise marked in its written presentation to signal (at least to a reader who is attuned to the meaning of the signal …) that it is, in fact, a defined term. That's very common practice in court filings and contract-type transactional documents, but for whatever historical reasons does not seem to be very common in statutes or regulations, which may be unfortunate.
If used well, defined terms make the resultant prose less wordy and convoluted, because they use a single word or short phrase to stand in for what would be a lengthier stretch of prose if you had to essentially write out the whole definition again each time. They also sometimes enable prior documents to be adapted for a new situation with less editing – if you have a form/template of a contract for let's say the sale of a house that repetitively refers to the various rights/duties of the Buyer and Seller, with both as defined terms, you just have to edit the part at the beginning that says who those persons are for the sake of this particular contract, rather than changing the names at dozens of separate places throughout the document.
As with a lot of things, they are not always used well and sometimes are used poorly. And sometimes things go awry. At one point in my career I worked on a dispute (ultimately amicably resolved) concerning inter alia a provision in a 100-page-plus commercial office lease where the provision was clearly using a defined term, except, due to some sort of editing glitch when the document was being finalized some years earlier, the actual definition of the defined term did not actually appear anywhere, having been inadvertently left on the cutting-room floor.
AntC said,
March 12, 2024 @ 9:33 pm
Thanks @JWB
I had in mind a discussion in Another Place of the legal phrase 'beyond a reasonable doubt'; which in British jurisprudence is now regarded as so technically-imbued/overused/bleached of its everyday sense juries are "no longer customarily directed to consider".
But the phrase does appear regularly in legal commentary of a current rash of U.S. cases, where it seems to me the defendents' (plural) modus operandi is to raise all sorts of unreasonable doubts; clearly with the aim of making the juries' heads spin to the point they can no longer reason.
David Hilbert said,
March 12, 2024 @ 11:02 pm
I enjoyed this footnote to the list of countries whose citizens are eligible for the US Visa Waiver Program:
* With respect to all references to “country” or “countries” on this page, it should be noted that the Taiwan Relations Act of 1979, Pub. L. No. 96-8, Section 4(b)(1), provides that “[w]henever the laws of the United States refer or relate to foreign countries, nations, states, governments, or similar entities, such terms shall include and such laws shall apply with respect to Taiwan.” 22 U.S.C. § 3303(b)(1). Accordingly, all references to “country” or “countries” in the Visa Waiver Program authorizing legislation, Section 217 of the Immigration and Nationality Act, 8 U.S.C. 1187, are read to include Taiwan. This is consistent with the United States’ one-China policy, under which the United States has maintained unofficial relations with Taiwan since 1979.
If I understand it correctly, it says that Taiwan, although not a recognized country, is in the extension of "country" and "countries" wherever those words occur in the authorizing legislation for the visa waiver program. It seems to have much the same logic and function as Mark pointed out for the exclusions in the definition of research.
J.W. Brewer said,
March 12, 2024 @ 11:44 pm
@AntC: I have only once served on an criminal petit jury, and it was some decades ago, but we acquitted the defendant of certain charges where it was pretty obviously >50% likely he'd done it. From which it follows that American jurors do, at least sometimes, take the "beyond a reasonable doubt" standard of proof in criminal cases as seriously as the judges instruct them to. If juries in England, whence we derived the concept, no longer do so, the more fool them for abandoning their cultural patrimony.
Obviously defense counsel (and/or, these days, freelance internet advocates and blowhards) may seek to raise unreasonable doubts as to factual guilt in a particular case, and it is the job of competent prosecutors (in the actual courtroom rather than in online banter) to convince jurors why these proffered doubts are not, in fact, reasonable.