An editorial by Miranda Robertson in the Journal of Biology ("Biologists who count", 8(34), 2009), starts this way:
The importance of mathematics in biology is a matter of perennial debate. The squabbles of early 20th century geneticists on the value of mathematics to the study of evolution have recently been revisited in Journal of Biology [Crow, "Mayr, mathematics and the study of evolution", JBiol 8(13) 2009], and the 21st century has seen an explosion of information from various -omics and imaging techniques that has provided fresh impetus to the arguments urging the need for mathematical competence in the life sciences [Bialek and Botstein, "Introductory science and mathematics education for 21st-century biologists", Science 2004, 303:788-790]. While there can be no question about the contribution of mathematics to many fields in biology, there is a curious tendency on the part of numerate biologists (often immigrants from the physical sciences) to insist that it is an essential part of the equipment of a biologist and none should be without it. This seems, on the evidence, extreme.
Robertson concludes that
There seems no need for the snobbery (it is said) of the highly quantitative founding biologists at the Cold Spring Harbor Laboratories, in whose early history ex-physicists played a crucial part, and who are alleged to have referred to their nearby colleagues at Woods Hole as biologists 'who don't count'.
There's an analogous set of arguments in the areas of rational investigation related to language, of which the discipline known as "linguistics" is a regrettably small part. (I mean by this that the balkanization of the field is a Bad Thing, not that the work of psycholinguists, linguistic anthropologists, speech pathologists, speech technology researchers, etc., is inadequate or inferior. But that's a different discussion.)
The role of mathematics in the language sciences is made more complex by the variety of different sorts of mathematics that are relevant. In particular, some areas of language-related mathematics are traditionally approached in ways that may make counting (and other sorts of quantification) seem at least superficially irrelevant — these include especially proof theory, model theory, and formal language theory.
On the other hand, there are topics where models of measurements of physical quantities, or of sample proportions of qualitative alternatives, are essential. This is certainly true in my own area of phonetics, in sociolinguistics and psycholinguistics, and so on.
It's more controversial what sorts of mathematics, if any, ought to be involved in areas like historical linguistics, phonology, and syntax.
I don't have time this morning for a longer discussion, so I'll just baldly state my conclusions, and look forward to hearing the opinions of others.
First, attempts to devise and test formal models of language-related phenomena are often helpful, and sometimes essential, in framing theories clearly enough to be able to see what's right and what's wrong with them. And occasionally, such models lead to genuine insight.
Second, mastery of statistical techniques is useful in all forms of rational inquiry; and networked digital computers are increasing this usefulness in a major way, due to increased availability of data and increased ease of statistical investigation.
Third, mastery of language-relevant mathematics crucially includes knowing when a model is inappropriate, misleading or unnecessary. Yeats said this in a more pointed way, in his 1930 "Letter to Michael's Schoolmaster":
Teach him mathematics as thoroughly as his capacity permits. I know that Bertrand Russell must, seeing that he is such a featherhead, be wrong about everything, but as I have no mathematics I cannot prove it. I do not want my son to be as helpless.
I happen to think that Russell was, on the whole, righter than Yeats was; but it would be wrong for the argument to be won by default due to one side's technical incompetence. If you've ever worked in an interdisciplinary area where mathematical backgrounds are variable, you've probably seen attempts to win arguments by this sort of mathematical default — often promoted by people who don't really understand a technique or program that they've learned to use in a cookbook fashion.
(I certainly don't subscribe to the part of Yeats' letter that says "Don't teach him one word of science, he can get all he wants from the newspapers and in any case it is no job for a gentleman". Alas, many of those who write about science in the newspapers seem to have been educated in accordance with this prescription.)
Anyhow, my conclusion is that anyone interested in the rational investigation of language ought to learn at least a certain minimum amount of mathematics.
Unfortunately, the current mathematical curriculum (at least in American colleges and universities) is not very helpful in accomplishing this — and in this respect everyone else is just as badly served as linguists are — because it mostly teaches thing that people don't really need to know, like calculus, while leaving out almost all of the things that they will really be able to use. (In this respect, the role of college calculus seems to me rather like the role of Latin and Greek in 19th-century education: it's almost entirely useless to most of the students who are forced to learn it, and its main function is as a social and intellectual gatekeeper, passing through just those students who are willing and able to learn to perform a prescribed set of complex and meaningless rituals.)