The Golden Fleece redivivus
« previous post | next post »
"The National Science Foundation: Under the Microscope: A Report by Tom A. Coburn, M.D., U.S. Senator, Oklahoma", April 2011:
This report is the first comprehensive overview of NSF. It examines the management of the agency, recognizes many of its accomplishments and successes, identifies some areas for improvements, and questions some of its priorities and funding decisions.
The good news for taxpayers is there is no question NSF has contributed significantly to scientific discovery.
The bad news is a significant percentage of your money is going to what most Americans will consider fraud, waste and abuse, and there are many areas where NSF could contribute far more with better management and smarter targeting of resources.
This report identifies over $3 billion in mismanagement at NSF. This includes tens of millions of dollars spent on questionable studies, excessive amounts of expired funds that have not been returned to the Treasury, inadequate contracting practices that unnecessarily increase costs, and a lack of metrics to demonstrate results. Additionally, a significant portion of the agency’s budget is spent on efforts duplicating missions performed by other government agencies and a number of NSF officials and grantees have been caught engaging in inappropriate behaviors, but face little or no consequences.
Aside from the introductory nod in the direction of NSF's positive contributions, and a page and a half on "Transformative Research Funded by NSF", essentially all of this 73-page report is negative. There's an explicit assumption that non-"transformative" research is a waste of money; but of course most of the report focuses on research that is not only not "transformative", but is depicted as "questionable" or worse.
Some of this is straightforward scandal: alleged peculation, porn, fraud, and so forth. Some of it strikes me as just a tad hypocritical, coming from a U.S. Senator: complaints about duplication of effort, missed deadlines, accounting oddities, and other things that have been developed into legislative-bureaucratic art forms on Capital Hill. But the core of the report is the 25 pages on "Questionable NSF Projects", which leads with a study on “The Role of Social Network Sites in Facilitating Collaborative Processes” — billed as "Does playing FarmVille on Facebook help people to make friends and keep them?" — and ends with a project on a "3-D, multi-user virtual environment (MUVE) of the 1964/65 New York World's Fair".
A few of these projects really are questionable, in my opinion. If that weren't true, NSF wouldn't be doing its job.
Opinions naturally differ about the relative priority of various scientific questions, and about the likely pay-off of different approaches to answering them. NSF tries hard to get good consensus advice about all this, and to strike an appropriate balance between caution and risk in placing its funding bets, just as any other investment portfolio manager should do. In this situation, if I weren't skeptical in advance about some of their choices, and if in the end some of their choices didn't fail, that would be striking evidence that their choices were way too cautious. And the moderately low correlation between my own tastes and the quality of research-project outcomes is just more evidence that there's no magic route to rewards without risks.
But what Sen. Coburn's report does is something radically different than just expressing an individual's opinions about research portfolio alternatives. Nor is he making an argument that NSF has its risk/return goals set wrong. Instead, he's arguing that NSF's procedures for assessing projects are fundamentally flawed; and his rhetorical method is to highlight particular projects that are argued to be silly and useless. However, those of his examples that I've checked rely on misunderstood or half-understood project descriptions, or results taken out of context and otherwise misrepresented.
This kind of research-project gotcha! is a game that's easy to play in a (sincerely or cynically) stupid way, as Senator William Proxmire used to demonstrate with his Golden Fleece awards. (See here, here, here for some details of specific awards).
At least one of Sen. Coburn's "questionable projects" involves speech and language:
Do twitter users “tweet” in regional slang? Researchers at Carnegie Mellon University received an NSF grant to perform a study of tens of thousands of tweets. A “tweet” refers to a 140-character or less post on the popular social networking site known as “twitter.” The conclusion was that, “regional slang and dialects are as evident in tweets as they are in everyday conversations.”
As CMU's response indicates, this passage takes work out of context ("the Twitter study was just one of more than 30 research papers" connected with the cited grant, which partly funded a broad range of work on social media), ignores the real and practical applications of such work ("Whether to promote positive uses or to counter negative uses, it’s important to understand the nature of interactions via social networking and how they evolve"), and fails to note that the research in question was also partly funded by the U.S. Defense Department and by Google.
My main reaction to Sen. Coburn's report is to note that the FY2010 budget appropriation for NSF's "Research and Related Activities" account was a bit under $7 billion. A significant portion of that was for large on-going projects like the polar programs (funding U.S. Antarctic research stations, for example) which take up half a billion dollars or so; as I understand it, about $5 billion went for research grants. This is a significant amount of money — but it's about a quarter of annual federal expenditures on agricultural subsidies, and a bit less than the current monthly cost of the Afghan war.
There's been a certain amount of press response to Sen. Coburn's report. My favorite, so far, is Erico Guizzo, "U.S. Senator Calls Robot Projects Wasteful. Robots Call Senator Wasteful", IEEE Spectrum 6/14/2011. Among other things, this article asserts that the report mischaracterizes the research, gets the grant references and amounts wrong, makes false claims (for instance that NSF paid $1.5 million for a robot that in fact was obtained for free), and never contacted the scientists involved to check facts or get comments.
Not far behind on my list is a letter to the Muskogee Phoenix from one of Sen. Coburn's constituents, "Antarctic gelatin wrestling no big deal", 6/9/2011:
While in Antarctica working for Raytheon and the National Science Foundation, you are subject to working conditions that you cannot imagine. To say that it is harsh is an extreme understatement. You work six days a week anywhere from eight to 12 hours per day. Much of it outside, walking much of the time because of the high cost of transporting fuel. It is cold and windy, and nearly everything you do during your stay there is in support of the science. […]
Yes, we had some unusual entertainment activities. It’s an unusual place. There are no movie theaters, bowling alleys, restaurants or golf courses Mr. Coburn. To ridicule and deny the dedicated and hardworking people there any form of recreation is unrealistic; it was a cheap shot.
I would gladly pay for the 30 or 40 boxes of jello that were used that Sunday if Mr. Coburn thinks that a little recreation for hard working people, isolated in a desolate place, is the correct area to save money.
A quick web search indicates that the cost for 30 to 40 standard-sized boxes of jello would be $30 to $35 plus shipping. I'd be happy to underwrite that myself, in return for naming rights ("The Annual Language Log Polar Jello Wrestling Challenge"?). And I wonder how many questionable expenditures of $35 or more could be found in a careful scrutiny of travel and entertainment expenditures by U.S. Senators.
Mary Kuhner said,
June 16, 2011 @ 10:22 am
In my experience this kind of outsider perspective, though it *ought* to be able to bring real waste and fraud to light, tends to degenerate into stupid (but eye-catching) attacks on researchers whose research "sounds funny" to the outsider.
Three cases in which "sounding funny" led to a lot of annoying and unfounded criticisms:
–Research on "nude mice," which have a congenital defect in the immune system and are used as a model of human immune deficiency diseases
–Research on the cellular effects of Prozac on nematode worms, trying to establish what the drug does at the level of cell/cell interactions in the nervous system
–Research on altitude adaptation in ducks–I was involved with this one, and my co-author stressed that we should keep the word "duck" out of any material meant to be seen by the public, as apparently a lot of people find ducks inherently silly
There are also whole fields of research, especially sexual behavior (human and animal), that find it difficult to get both funding and respect because the topic is seen as snigger-worthy. I don't think that the practice of science should be hampered by having to consider whether your topic makes for a comedic cheap shot or not.
Disclaimer: I am funded by NSF.
D.O. said,
June 16, 2011 @ 10:41 am
What if somebody submits an NSF proposal to research (scientifically!) waste and abuse by the US Senate? Would Sen. Coburn look at this research approvingly?
Stephen Nicholson said,
June 16, 2011 @ 10:42 am
I'm currently working through the Sherlock Holmes cannon and the Twitter bit caught my eye because it's the kind of thing Holmes would say.
"Watson, I see this tweet was made by a woman from Canada."
"That's amazing Holmes."
"Surely you can see it for yourself, it's obvious by the way she ends her tweets."
Spell Me Jeff said,
June 16, 2011 @ 10:43 am
Pandering as usual, with the by-now-socialized subtext that academics are liberals, and liberals engage in elite practices that waste tax dollars, etc.
What we SHOULD take away from moments like this is that education in general (higher or otherwise) has FAILED to communicate to the citizenry a sense of how and why knowledge is made. A bit of numeracy with respect the the federal budget wouldn't hurt either (e.g., compared to a single sortie over Libya (just one, mind you) how much money was spent studying tweets?).
(I know I'm not the first one to say all this, but it bears repeating.)
fev said,
June 16, 2011 @ 11:59 am
Not surprisingly, the reaction in (ahem) some press sectors was a little more gleeful, and "Millions of Your Tax Dollars to Exercise Shrimp" rivaled the nekkid polar Jell-O wrestling as the punching bag of the day. (headsuptheblog.blogspot.com/2011/05/science-bad.html, if I can beg the Log's indulgence)
According to the APA, Proxmire ended up paying $10,000 to the scientist who accused him of libel in the case linked in your post. That certainly doesn't seem to have had a chilling effect on the War on Science.
[(myl) The jello-wrestling seems to have been non-nekkid, though there was apparently some skinny dipping nearby in time and space. From Sen. Coburn's report (which alas is probably not a reliable source):
It's unclear whether or not Aqua Buddha put in an appearance.]
Blerg said,
June 16, 2011 @ 12:21 pm
@Stephen. Exactly. It's hard to argue with 'this research is forensically-relevant and can be used to catch criminals and in the legal process'.
Tom W said,
June 16, 2011 @ 2:09 pm
In reading these postings, I reminded that every sector loves its subsidies. But the NSF is charged with wasting money. Claiming hyposcrisy does not refute the charge. Saying waste is greater elsewhere does not refute the charge. Pointing out the amounts spent are small relative to some other spending does not refute the charge. Saying this is an attack on academic liberals does not refute the charge. Why not take it seriously that the government has spent some money badly? This is money to which no one has a right. It is only ethical that recipients have to justify to others their use of money taxed from others. Especially now when the US Federal government is in debt to an extent unknown in its history. I may not need to mention this, as you may have heard something about it.
[(myl) All true. But what does refute the charges is examining them in detail, as the cited article about the robotics project does, and as the cited note about Proxmire's libel-case loss does, and as the examination of the shrimp research that fev points to does, and so on. And the rest of the discussion is relevant in asking the question, why the focus on reckless cheap shots about science research, at a time when such large amounts of money are being wasted on other things?]
Urso said,
June 16, 2011 @ 2:45 pm
If Sen. Coburn is thinks the jello wrestling is bad, he'd probably have a fainting fit at the average weekend recreational activities engaged in by students at universities (who in aggregate receive orders of magnitude more federal funding than the Antartic research station ever will).
peterm said,
June 16, 2011 @ 3:38 pm
What about all those computer science research projects that don't actually solve problems, but merely tell you hard the problems would be to solve were someone to try to solve them! Surely that is a waste of NSF funds! If I had my way, those computer scientists would be re-employed in real jobs, eg, as traveling salesmen or bin packers!
John Burgess said,
June 16, 2011 @ 4:45 pm
As a US diplomat serving abroad, I was–as we all were–frequent recipients of congressional disdain, shock, and horror. "Why would a diplomat live in a house bigger than that of a US Representative? Must be waste1" It would be bad if this were an opinion coming from Washington, but instead it came from the Representative–and his wife–who were actually visiting the US Mission in Saudi Arabia. The country where women and children are confined to their homes by climate and social/customary pressures. Moms can't drive the kids even to a neighbor's house. As a result, with most families spending most of their assignments within the confines of their homes (Little League Baseball at 130F is not popular for some reason), they actually do need larger than average homes.
Proxmire, a Democrat from that most-Democratic of states, Wisconsin, took a turn to populism because outrage was easy to convert into votes, or at least popularity. It's a well-rung bell and can be rung again by any and all who see benefit in it.
Sure, there is wasteful research as there is wasteful everything. But most of the Golden Fleece recipients I looked into were legit. The more you actually know about a subject, the more asinine the media cheap shots and political critiques seem.
Liberty Lidz said,
June 16, 2011 @ 8:48 pm
Note the recommendation to cut the Social, Behavioral, and Economics (SBE) Directorate (p. 53) in the section entitled 'Recommendations' (which begins on p. 51). "But do any of these social sciences represent obvious national priorities that deserve a cut of the same pie as astronomy, biology, chemistry, earth science, physics, and oceanography?"
I can only hope that the legislators who read this report have more common sense than Coburn does.
bloix said,
June 16, 2011 @ 8:51 pm
Tom W says, "Why not take it seriously that the government has spent some money badly?"
The answer is, to use your style of writiing, which may persuade you since you seem to find it persuasive, is that Tom Coburn does not care about government waste. Tom Coburn does not care about pallets of hundred dollar bills that went missing in Iraq. Tom Coburn does not care about ridiculous Army Corps of Engineer projects that eat up hundreds of millions of dollars. Tom Coburn does not care about subsidies to corn and soybean farmers.
But Tom Coburn does care about science. In fact, Tom Coburn hates science. Tom Coburn thinks that global warming is a load of crap. Tom Coburn thinks that evolution is just a theory. Tom Coburn thinks that Silent Spring was junk science.
So don't be fooled. This report isn't about trying to save one-tenth of one percent of what the wars in Iraq and Afghanistan cost per month. It's about bashing scientists. Scientists are egg heads and frauds and grifters and fools, that's what this is about. Don't believe them. Believe the preachers and the oil companies and the agribusiness companies, instead.
Dan Lufkin said,
June 16, 2011 @ 9:06 pm
What else is new? Remember a former governess of a northern state who had a campaign zinger that the gummint was funding research on the sex life of FRUIT FLIES FRUIT FLIES. I kid you not! And the present governor of a southern state used his counter-state-of-the-union address to decry operating a Federal volcano observatory. That was right before whatchamacallit blew and stopped air travel. Senators from Oklahoma have an innate blind spot for science of any kind. Something in the water. Needs more research.
Bill Walderman said,
June 16, 2011 @ 9:17 pm
I guess it's also a huge waste of the taxpayer's money to spend a dime for any sort of entertainment for members of the US armed forces serving in, say, Afghanistan.
Keith M Ellis said,
June 16, 2011 @ 10:01 pm
I really wish that it were the case that at 46 years of age, I'd finally—finally—find that this particular brand of perennial stupidity didn't make me want to rant and throw things…but it appears that not much has changed in my psychology (in this regard) since Proxmire's era.
!!
ShadowFox said,
June 17, 2011 @ 1:00 am
I am somewhat disappointed that Mark stooped to Coburn's level in the following passage:
Remember that you are looking at cherry-picked and often doctored information. It's easy to produce one-line zingers about sex habits of black bears, exercising shrimp, mating fruit flies etc. But these are not meant to be serious attacks on the science–most of Republican Congressional aides who pen these screeds are functionally illiterate in science. They are meant to play well on Fox News, in Washington Times, Human Events, and other equally reputable media establishment, where they will be seen, heard and read by those who already not only possess an anti-intellectual bias, but are conditioned to subdue any credulity they might have otherwise had.
Coburn is not the biggest idiot in Congress–not even in the Senate (Johnson, Paul, Inhofe and DeMint make sure that he does not sink to the bottom)–but he's no intellectual shining star. Despite considerable competition and many years of education, not all members of the medical profession–and, especially, not all lawyers–are particularly intelligent.
My reaction to Mark's comment would be simply to suggest to refrain accepting any such claims at face value, at least until they have been examined and confirmed by people who 1) know better and 2) don't have a political ax to grind. I have been following similar "waste" reports and other attacks on the NSF for over a decade and have yet to have seen a confirmation of any claim that any singled out research project was "wasteful" or "non-transformative" in the sense that the Coburn report uses.
As for jello wrestling, the traditional perspective–particularly in the backwater of Oklahoma–is that scientists are socially inept squares who are incapable of enjoying or participating in any kind of traditional entertainment or even of having a non-science-related sense of humor. So the idea of an NSF-sponsored project staging some kind of oddball entertainment is particularly grating as it goes against this perception, and, therefore, it must have had some sinister ulterior motive behind it. So it's wasteful not because it required $50 worth of jello, but because it proves that the entire project is unserious and does not deserve taxpayer support. The fact that more money has been wasted on this report and on firing the guy who organized polar jello wrestling does not appear to enter the equation.
Barbara Phillips Long said,
June 17, 2011 @ 1:19 am
Recently, a friend has provided me with a subscription to a science magazine. I hadn't been reading any particular science publication since I let my last subscription lapse. The one benefit of becoming poorly informed about the state of science is that when I started reading about it again, I was struck by how little the average U.S. citizen learns about science from "the media."
If I had access to grant money and effective persuasion, I would try to get science magazines that write for general audiences to form some kind of consortium and supply articles about science so students in English-speaking countries and students who want to read in English could read an article a day, Monday through Friday, about science.
I'd like to see a website that was open access and provided articles at various reading levels. Students exposed to the wide range of subjects that various sciences consider might then be able to make more informed career choices.
If I could change education in the U.S., I'd like teachers to use the texts daily as a basis for vocabulary, reading and discussion skills, not as a basis for testing. Emphasis on literature and the bias toward literature by "English" teachers has had unexpected consequences — students don't read much science or history, students don't learn much science and history, and students have weaker vocabularies. Schools are producing voters who take Proxmire-style criticism at face value.
If I could change teacher education, I would require any teacher to become comfortable reading and teaching about many different types of writing, including science writing.
If I'm unaware that a similar service already exists, tell me. If teachers re being trained to emphasize science in reading curricula, I'd also like to know. At least I could then ask a few teachers why they are using it or not using it.
Christian DiCanio said,
June 17, 2011 @ 3:40 am
Many NSF-funded projects require a researcher to state how their work would have some sort of real-world application. Thinking of applications for pure research is sometimes difficult for scientists. So, you can imagine the researcher investigating social networking sites coming up with something in his/her proposal saying: "For instance, does playing FarmVille on Facebook help people to make friends and keep them?"
Coburn then proceeds to attack proposals based on these silly-sounding real-world applications. This is quite hypocritical. The NSF demands some mention of how a project relates to the average person's life, but then these little snippets end up being taken out of context and used against the researcher. The conclusion that we must reach from this is one of the following:
A. Researchers should stop trying to make their projects sound applicable to the real-world. Keep the language very technical and you have less chance that the average person will misconstrue (or understand) what you propose.
B. Researchers should have better training at making their projects sound applicable to the real-world. The amount of funding one receives should be proportionate to how much your work can be construed as fighting "terrorism." Perhaps all DEL project proposals should begin with stories of the Navajo code talkers. Work on syntax could be supported because it can be used to create a universal translator for languages from "terrorist states", much in the way that some of Chomsky work was supported in the 1960's.
maidhc said,
June 17, 2011 @ 4:21 am
I guess this
http://latimesblogs.latimes.com/technology/2011/06/government-plans-to-delete-half-its-web-sites.html
is not exactly the same thing, but it is equally stupid.
6 billion dollars in cash is stolen from Iraq and flown out to the Cayman Islands, never to be seen again, and that is no problem, but $125 to run a website about endangered species is unforgivable government waste?
Trimegistus said,
June 17, 2011 @ 6:38 am
Nobody has refuted Tom W.'s point: waste is waste, and it's praiseworthy to try to find and eliminate it. You all immediately reverted to the same to quoque and non sequitur arguments he pointed out.
Something tells me that if Sen. Coburn had a (D) after his name instead of an (R), many of you would be nodding sagely about the need to trim waste.
Of course, if Sen. Coburn had a (D) after his name, he wouldn't be concerned about waste.
[(myl) It's not praiseworthy to promote an ideological agenda by taking factually careless cheap shots at entirely virtuous research projects, such as the robot study or the shrimp study, in which there appears to have been no waste whatever, but rather, in each case, a serious and useful approach to a scientific problem with obvious and immediate practical applications.
And speaking of being factually careless, were you not aware that Sen. Proxmire had a (D) after his name? This is typical of your comments, which are generally either ignorant or malicious, or perhaps both.]
Ian Preston said,
June 17, 2011 @ 7:23 am
If you go through the report and add up on the back of an envelope the values of the "dumb projects" you find, I think, that it is difficult to get close to the figure of "at least $65 million in wasteful spending" without including the whole of the most recent five year funding of the Panel Study of Income Dynamics mentioned on p.33. (Admittedly figures are missing for a couple of projects but something of the order of $15 million would have had to have been spent researching the impact of terror threats on John McCain's vote or not only current but past grants to the American National Election Studies and more would have had to have been included for it to add up otherwise). That is to say, funding for the continuation of a 40 year project which has led to thousands of refereed publications and is among the main sources of information, for example, on the changing US income distribution appears to be written off in its entirety as "wasteful" because someone used the data to do (what in fact looks to have been rigorous and economically informative) secondary analysis of household time use.
Noah Smith said,
June 17, 2011 @ 8:04 am
I'm one of the co-authors on the paper Coburn's report cited in discussing the Twitter research. CMU's response is a news release — http://news.cs.cmu.edu/article.php?a=2524 — noting, among other things, that the NSF project was much broader, with the Twitter study as one of more than 30 published papers to date on social and biological systems. My point: policymakers like Coburn and his aides (and anyone reading the report) ought to know more about science and the scientific process than to draw conclusions about a large, multi-year research effort from a single publication.
chris said,
June 17, 2011 @ 8:38 am
waste is waste, and it's praiseworthy to try to find and eliminate it.
But the whole point of this post is to point out that Coburn epically failed at the "find" stage. What's praiseworthy about that? The need to trim waste has to be subordinated to the need to *accurately* identify what is waste and what is not. And that can't be done in a sound bite.
[links] Link salad is muzzy | jlake.com said,
June 17, 2011 @ 8:58 am
[…] The Golden Fleece redivivus — Language Log on Senator Coburn and the NSF. […]
Mary Kuhner said,
June 17, 2011 @ 10:19 am
What Coburn and Proxmire do achieve, sadly, is to make scientists afraid for their jobs and careers, and this tends to make them more timid and easily bullied–which is, perhaps, one of the intentions.
I am looking at a wonderful short video of top geneticists doing inflatable-suit sumo wrestling in my other window, and wondering if I am brave enough to post the link, or if it's going to be used to attack the department responsible.
In my experience, hidden among all the "attack this because its name is silly" are always some "attack this because it might reach conclusions I don't agree with". If the rest of us are cowed into letting this happen, we all lose.
Genomic sumo.
Jenny said,
June 18, 2011 @ 6:14 pm
One of the results of reports like Sen. Coburn's is to inflate the amount of money apparently spent on science. Voters in the US have no idea how much programs cost, and when you say "$3 billion" in the same paragraph as "tweet slang," it is easy to read that as "the taxpayers spent $3 billion studying tweet slang." Golly! How much must be spent on real science, whatever that is? It must be $30 billion, or even more! It is all about perception and belief, and Sen. Coburn is essentially lying to change voters' beliefs. (Source: http://politicalticker.blogs.cnn.com/2011/04/01/cnn-poll-americans-flunk-budget-iq-test/)