Algorithms: Threat or Menace?
« previous post | next post »
Muhammad ibn Musa al-Khwarizmi (see also here)
… was a Persian mathematician, astronomer, and geographer during the Abbasid Caliphate, a scholar in the House of Wisdom in Baghdad.
In the 12th century, Latin translations of his work on the Indian numerals introduced the decimal positional number system to the Western world. Al-Khwārizmī's The Compendious Book on Calculation by Completion and Balancing presented the first systematic solution of linear and quadratic equations in Arabic. He is often considered one of the fathers of algebra. He revised Ptolemy's Geography and wrote on astronomy and astrology.
Some words reflect the importance of al-Khwārizmī's contributions to mathematics. "Algebra" is derived from al-jabr, one of the two operations he used to solve quadratic equations. Algorism and algorithm stem from Algoritmi, the Latin form of his name.
Al-Khwarizmi flourished in the early 9th century A.D., but algorithms — step-by-step procedures for solving problems by well-defined rules — have been around for a lot longer, e.g. Euclid's algorithm for computing the greatest common divisor of two numbers.
I bring this up because of a lexical change in progress, whereby algorithm is apparently being redefined to mean something like "one of the mysterious and scary AI programs that are invading our lives". At least that's the meaning implicit in the BBC World Service program "Algorithms: Should We Worry?" —
Artificial intelligence is taking over everything from medical diagnosis to legal due process. Rob Young asks if the lack of transparency and risk of error are causes for concern.
Plus, Iran! Also, Arabic!
The start of the program:
Of course there are algorithms in everything a computer does, including what we think of as mundane activities like sensing a key press, or rendering an image, or playing a sound. In fact the algorithms that perform such familiar and everyday tasks are complex and sophisticated. But the word algorithm apparently strikes people as a fancy one, that should therefore be reserved for programs that do edgier and less ordinary things:
The program deals with some serious issues, focused on an exchange with Cathy O'Neil, who wrote Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. You can listen to the rest of the program(at least until the BBC takes it down in accordance with their weird 30-day limit), or better, buy and read O'Neil's book.
But this is Language Log, not Big Data Fairness and Transparency Log, so having noted how the word algorithm is being exoticized, let's return briefly to the word's origins and previous development.
According to the OED's entry for algorithm,
Al-Ḵwārizmī's Arabic surname, which literally means ‘the native of Khorezm’, is < Arabic al the + Ḵwārizm (also Ḵwārazm ), the Arabic name of the town of Khorezm (now Khiva in Uzbekistan) + -ī -i suffix2.
Based on its Wikipedia article, Khiva looks like it might be interesting to visit.
The OED's etymological note includes this catalogue of sources and variants
post-classical Latin algorismus Arabic numerical notation (from 12th cent. in British sources; also in continental sources; also as augorismus (a1274, a1349 in British sources), algarismus (first half of the 15th cent. in British sources), used as common noun of the personal name Algorismus , Algoarismus , Alghoarismus , Alchoarismus (all from 12th cent. in British and continental sources) < Arabic al-Ḵwārizmī (also al-Ḵwārazmī , al-Ḵuwārizmī , al-Ḵuwārazmī ), surname of the mathematician Abū Jaʿfar Muḥammad b. Mūsā (fl. 800–47).
The historical senses of the word include "The Arabic system of numbering, characterized by a zero", as in:
1658 E. Phillips New World Eng. Words Algorithme, the art of reckoning by Cyphers.
And also, I was surprised to learn, "calculus", as in
1721 J. Wilson Let. 21 Jan. in I. Newton Corr. (1977) VII. 126 It does not appear from the Commercium that you were acquainted with the true Characteristics and Algorithm of Fluxions or Differences, before their Master.
a1739 N. Saunderson Elements Algebra (1740) I. p. xv, The War that was lately waged among Mathematicians..concerning the Algorithm or Principle of Fluxions.
The (pre-BBC) modern meaning, "A procedure or set of rules used in calculation and problem-solving; a precisely defined set of mathematical or logical operations for the performance of a particular task", seems to have arisen in the early 19th century:
1811 Eclectic Rev. July 587 It [sc. the calculus of variations] wants a new algorithm, a compendious method by which the theorems may be established without ambiguity and circumlocution.
1819 Philos. Trans. (Royal Soc.) 109 319, I should prefer applying the limiting formulæ in the usual way; passing however from column to column..of the results, at first by means of the neat algorithm..and afterwards by differencing.
1836 O. Gregory Hutton's Course Math. (ed. 11) I. 231 Instead of trying other criteria, apply Horner's algorithm for reducing the equation by 1, 11 …then [etc.].
1902 Trans. Amer. Math. Soc. 3 290 It may be determined by Euclid's algorithm for finding the greatest common divisor.
Rodger C said,
October 28, 2016 @ 6:45 am
I thought an algorithm lived on the banks of the Nile.
[(myl) No, that's an allegory.]
Ed M said,
October 28, 2016 @ 7:04 am
"In fact the algorithms that perform such familiar and everyday tasks are complex and sophisticated."
Algorithms don't perform tasks any more than a cake recipe makes a cake. Machine-level instructions ultimately derived from a human or machine interpretation of an algorithmic process do tasks. Just as a cake recipe "fails" if the human grabs a cup of salt instead of sugar, an algorithm "fails" if the human inserts a subtract function in her code instead of multiply. Thus, bugs.
[(myl) By metonymy, the word "algorithm" is commonly used to refer to the implementation of an algorithm — a quick Google Scholar search turns up hundreds of thousands of phrases like
the proposed algorithm performs especially well on large flow-shop problems
the new algorithm performs better than a version of Samuel's Algorithm
the algorithm performs successfully for many instances
how the new algorithm performs for random graphs
or
the algorithm solves this unambiguous maximisation problem
the proposed EDFSA algorithm solves this problem
the new algorithm solved the problem in 0.11 seconds
many engineering problems have been successfully solved by this kind of algorithm.
]
empty said,
October 28, 2016 @ 7:06 am
I'm see that the word "difference" was being verbed in 1819, apparently with the (infinitesimal calculus) sense "differentiate".
Fionnbharr Ó Duinnín said,
October 28, 2016 @ 7:29 am
Well, we all know how sinister mathematicians are: https://www.washingtonpost.com/news/rampage/wp/2016/05/07/ivy-league-economist-interrogated-for-doing-math-on-american-airlines-flight/
cs said,
October 28, 2016 @ 8:00 am
Based on those clips (I didn't listen to the whole program) it sounds like they were using "algorithms" with a slightly more specific meaning, not just any complex software function. It seemed to mean software that tries to anticipate some human choice or preference e.g. text autocomplete or shopping recommendations.
(Maybe I'm stating the obvious here)
Jeffrey L. Whitledge said,
October 28, 2016 @ 8:24 am
Perhaps the program would have been more accurate if they used the word "heuristic" rather than "algorithm", since there is no mathematically reliable way to anticipate human preferences.
Johan P said,
October 28, 2016 @ 8:56 am
From what I understand, an algorithm in computing is unlike a programme in that it is something that accomplishes a specific, limited set of tasks, using basic instructions. Many of the examples you use would be considered proprietary hardware instructions or programmes in that sense, because they're not solutions to a set goal per se, and they're not comparable to each other or measurable in terms of efficiency. They just do stuff.
"Algorithm" in computing, as I understand it, is stuff like:
Sorting algorithms – what can sort a set of values in order of size as efficiently as possible?
Compression algorithms – what can make a file as small as possible while retaining all (or all relevant, or all perceptible, or reasonably much) information within?
Search algorithms – what can find specific data in a file/database/set of interlinked web pages as efficiently as possible?
And so on. I think this new use is an offshoot of that.
Nicky said,
October 28, 2016 @ 9:24 am
Question: Is it correct to refer to Al-Khwarizmi as Persian? As far as I understand his native language is East Iranian and therefore is not Persian.
[(myl) He apparently came from what is now Uzbekistan, worked in Baghdad, and wrote in Arabic.]
Joe said,
October 28, 2016 @ 9:45 am
So, could a neural net system be rightly called an algorithm? If such a system were to correctly identify a particular pattern of speech (say, after several "learning" iterations), could we identify the "precisely defined set of mathematical or logical operations" that took to get that identification? We can definitely call the underlying mechanism as such, but wouldn't the individual results it produced defy precise definition?
Rodger C said,
October 28, 2016 @ 10:14 am
My favorite fun linguistic fact about Al-Khwarizmi is that in medieval Europe he was known as Algus, a back-formation from "algorithm." (Source: C. S. Lewis, The Discarded Image.)
Bob Ladd said,
October 28, 2016 @ 10:34 am
@Ed M. MYL's comments about metonymic uses of algorithm also apply to recipe. "This recipe serves four." "This recipe doesn't work at high altitudes." Etc.
KWillets said,
October 28, 2016 @ 12:05 pm
@myl, I don't interpret most of those phrases as referring to implementation. The verb "performs" in this case is typically a reference to computational complexity, not the properties of any particular implementation. For instance performance on random graphs is typically a mathematical expression, not a measurement.
That said, there is indeed a popular usage referring to automated systems for decision making, for instance trading stocks or ranking web searches. The implication is that human decision-making has been replaced by explicit rules which are often derived from data analysis.
RP said,
October 28, 2016 @ 2:53 pm
To put it simply, Mark is correct.
I took my first FORTRAN course in 1970, studied math and linguistics in college, worked as a software engineer for a good couple of decades, and for the past dozen years have been a technical writer at a software company, specializing in developer documentation. In other words, I have both written and explained software algorithms for quite a while, and have at least a bit of conceptual background in the issues we are discussing.
Having hopefully established my expert witness cred, I will point out that when a software developer writes an "algorithm," they think of it as being "implemented" in "the code" (that is, a series of programming statements), no matter what is going on at the hardware level (which of course includes "lower-level" algorithms, not to mention that the "machine code" that is executed on the hardware was created by a compiler or interpreter that included algorithms of its own).
And as for the meaning of the term itself, an algorithm is normally thought of in the software world as a series of statements that performs a specific function, which could be quite simple, "like sensing a key press".
Many of the most complex algorithms do in fact need to be developed conceptually–using various techniques, including the development of mathematical formulas–before they are implemented in software. That is, before they are turned into "code." But the day-to-day activities of the average coder include writing literally zillions of algorithms that have only ever seen the light of day in the form of programming statements, whether in C or JavaScript, or anything in between. The usage is so common, that Mark's instantiation of it didn't slow me down at all as I read it just now.
Rubrick said,
October 28, 2016 @ 5:12 pm
Somewhat echoing Joe's point: the recent "sinister" usage (or at least connotation) of algorithm is ironic, since the current "smartest" — and thus scariest — AI systems are increasingly non-algorithmic in the traditional sense. Most people aren't yet aware of this. (Though Mark certainly is!)
Deep-learning-based systems don't arrive at results through a series of discrete steps. A corollary is that it's very difficult, if not impossible, to determine why it arrived at a particular result. A self-driving car that crashes into a wall may not have a reason beyond "I thought there was a driveway there". Those who imagine " we found a bug in line 223 of the 'is this a wall' function' are living in the past.
Mick O said,
October 28, 2016 @ 6:00 pm
Adding to what KWillets said about popular usage: In my experience, the discussed usage of the word algorithm is being perpetuated by programming/dev folks who, when asked WHY the system they built did something silly/counter to stated goals/dangerous/ludicrous, they use the term "algorithm" to deflect responsibility or belittle the person who had asked why.
Put another way, we "normals" didn't leap to exoticizing algorithm on our own, we've just heard it blamed so often by smart folks we figured we should join in.
tangent said,
October 28, 2016 @ 10:32 pm
Mick O, do you have an example? I can't picture a programmer using the word that way. The main excuses I expect are 1) the behavior is according to the specification (even if the specification is dumb), or 2) there was a bug. Yes, some systems have a machine-learning element or similar black box whose emergent behavior the programmer can shrug about, but that's still pretty uncommon, and you can't fake it.
tangent said,
October 28, 2016 @ 10:35 pm
But yes, there seems to be an increasing usage where "algorithm" is used to describe specifically what the practitioner would call machine learning.
Joyce Melton said,
October 29, 2016 @ 1:30 am
In programming, an algorithm is a set of rules that specify the output to a set of varying inputs. The operation of the rules is embodied in the code but the algorithm is conceptual.
The way algorithm is being used as something scary is as an example of the sort of thinking that leads to traffic stops in Arizona for "driving while brown." Sheriff Arpaio's algorithm said stop all brown drivers and ask for papers. Let the citizens go and assume anyone who does not have papers is not a citizen. Turn non-citizens in to La Migra for deportation. The algorithm was flawed.
Did you hear the psychologist's algorithm for diagnosing patients? If they arrive early, they have an anxiety disorder. If they arrive late, they are neurotic and have trouble dealing with authority. If they're on time, they must be compulsive.
Richard Hershberger said,
October 29, 2016 @ 7:01 am
Y'all are missing the big picture. The real question at hand is whether, when someone says "the algorithm," we can smile with pitying condescension and explain that the "al-" part means "the," so they just said "the the gorithm."
James Wimberley said,
October 29, 2016 @ 7:02 am
"I thought an algorithm lived by the banks of the Nile." Like sharks, algorithms have to keep moving to respire, so they are never observed to halt. A large and aggressive animal, it hunts and is hunted by the indigenous Np. The Np hunters use circular pit traps called tu-rings. When an algorithm succeeds in turning the tables and catches and eats an unlucky hunter, it is Np-replete.
Bob Ladd said,
October 29, 2016 @ 9:17 am
A phonological angle on this that no one has mentioned is where the "th" in algorithm came from, given that the Arabic source was a /z/. Presumably there's some sort of confusion or analogy or other type of linguistic cross-pollination with arithmetic?
Andrew said,
October 29, 2016 @ 11:16 am
So is it any wonder people are afraid of technology? https://youtu.be/Fc1P-AEaEp8
KWillets said,
October 29, 2016 @ 12:51 pm
Knuth and the OED claim the spelling and etymology were misattributed to the Greek root arithmos.
Adrian said,
October 29, 2016 @ 3:06 pm
What did the ancient Greeks call algorithms?
[(myl) μέθοδοι ?]
BZ said,
October 31, 2016 @ 12:04 pm
Wow, I always thought the words algorithm and logarithm were related in some way. It turns out this is only tangentially true as "algorism" became "algorithm" due to influence of "arithmos" which is one of the roots "logarithm" comes from.
This may explain something I long found odd in Russian. In the absence of a "th" sound, Russian transliterations turn the offending phoneme into a "t", "f", or, less often, "s" or "z". I always wondered why it was "logarifm", but "algoritm". Now it actually makes sense since the Latin "Algoritmi" never had a "th" in it to begin with (though as I found out while researching all this, "algorfm" is actually a variant in Russian, albeit now mostly an archaic one).
Mick O said,
November 2, 2016 @ 5:30 pm
@tangent To be sure, the true reason was often that the specifications had failed to account for cases that turned out to be common, and the solution was to refine them. You are correct about that. But, we'd heard "the algorithm did it" cited so often (with a straight face) in response to questionable output that it became an in-joke. My point is just we never would have used the term without it being shown to us how commonly and casually it could be bandied about as a catch-all for black-box type issues from the people who actually developed the algorithms.