"In Pāṇini We Trust"

« previous post | next post »

Article in Popular Science:

This ancient language puzzle was impossible to solve—until a PhD student cracked the code

The discovery makes it possible to translate any word written in Sanskrit.

Laura Baisas (12/15/22)

Some universities require Sanskrit for all linguistics students and some universities have two first-year Sanskrit courses, one for linguistics students and one for Indologists and other humanists.  That's a tribute to Pāṇini पाणिनि (ca. 6th-4th c. BC) — no, not the bread roll — rather, the world's first grammarian. His 3,996 verses or rules on linguistics, syntax, and semantics in "eight chapters" (Aṣṭādhyāyī) are as terse and precise as mathematical equations.  You'd think that, after two and a half millennia of intense study by thousands upon thousands of pandits, they'd all have been solved by now.  Apparently not, since one was just solved for the first time a few years ago.

A PhD student studying at the University of Cambridge has solved a puzzle that has stumped scholars since the fifth century BCE. Rishi Rajpopat decoded a rule taught by Pāṇini, an Indian grammarian who is believed to have lived in present-day northwest Pakistan and southeast Afghanistan.

Rajpopat decoded a 2,500-year-old algorithm that can accurately use Pāṇini’s “language machine” for the first time.

“Pāṇini had an extraordinary mind, and he built a machine unrivaled in human history,” said Rajpopat. “He didn’t expect us to add new ideas to his rules. The more we fiddle with Pāṇini’s grammar, the more it eludes us.”

Often, two or more of Pāṇini’s rules can be applied at the same time and step in the process, which has left scholars agonizing over which rule or step to choose.

An algorithm is needed to solve this rules conflict, which affects millions of Sanskrit words, including certain forms of the commonly used “mantra” and “guru.” Pāṇini had a metarule to help the user decide which rule should be applied if a rule conflict occurred, but it has been misinterpreted by scholars for the last 2,500 years.

Traditionally, Pāṇini’s metarule has been interpreted as: in the event of a conflict between two rules of equal strength, the rule that comes later in the grammar’s serial order wins. However, Rajpopat argues that Pāṇini meant that between rules applicable to the left and right sides of a word respectively, Pāṇini wanted us to choose the rule applicable to the right side.

“I had a eureka moment in Cambridge. After nine months trying to crack this problem, I was almost ready to quit, I was getting nowhere. So I closed the books for a month and just enjoyed the summer, swimming, cycling, cooking, praying and meditating,” said Rajpopat. “Then, begrudgingly I went back to work, and, within minutes, as I turned the pages, these patterns starting emerging, and it all started to make sense. There was a lot more work to do but I’d found the biggest part of the puzzle.”

By using this interpretation that Pāṇini expected the rule applicable to the right side to be chosen, Rajpopat found the ancient scholar’s language machine produced grammatically correct words consistently and with almost no exceptions.

Over the next two-and-a-half years, he worked to solve problems in what he had found and presented. In addition to understanding more Sanskrit texts, the algorithm that runs Pāṇini’s grammar can potentially be taught to computers.

“Computer scientists working on Natural Language Processing gave up on rule-based approaches over 50 years ago,” said Rajpopat. “So teaching computers how to combine the speaker’s intention with Pāṇini’s rule-based grammar to produce human speech would be a major milestone in the history of human interaction with machines, as well as in India’s intellectual history.”

Here's the citation for Rajpopat's PhD thesis:

Rajpopat, R. (2021). In Pāṇini We Trust: Discovering the Algorithm for Rule Conflict Resolution in the Aṣṭādhyāyī (Doctoral thesis). https://doi.org/10.17863/CAM.80099
 
And here's the abstract

If two rules are simultaneously applicable at a given step in a Pāṇinian derivation, which of the two should be applied? Put differently, in the event of a ‘conflict’ between the two rules, which rule wins? In the Aṣṭādhyāyī, Pāṇini has taught only one metarule, namely, 1.4.2 vipratiṣedhe paraṁ kāryam, to address this problem. Traditional scholars interpret it as follows: ‘in the event of a conflict between two rules of equal strength, the rule that comes later in the serial order of the Aṣṭādhyāyī, wins.’ Pāṇinīyas claim that if one rule is nitya, and its simultaneously applicable counterpart is anitya, or if one is antaraṅga and the other bahiraṅga, or if one is an apavāda (exception) and the other the utsarga (general rule), then the two rules are not equally strong and consequently, we cannot use 1.4.2 to resolve the conflict between them. The nitya, antaraṅga and apavāda rules are stronger than their respective counterparts and thus win against them. But this system of conflict resolution is far from perfect: the tradition has had to write numerous additional metarules to account for umpteen exceptions. In this thesis, I propose my own solution to the problem of rule conflict which I have developed by relying exclusively on Pāṇini’s Aṣṭādhyāyī. I replace the aforementioned traditional categories of rule conflict with a new classification, based on whether the two rules are applicable to the same operand (Same Operand Interaction, SOI), or to two different operands (Different Operand Interaction, DOI). I argue that, in case of SOI, the more specific i.e., the ‘exception’ rule, wins. Additionally, I develop a systematic method for the identification of the ‘more specific’ rule – based on Pāṇini’s style of rule composition. I also argue that, in order to deal with DOI, Pāṇini has composed 1.4.2, which I interpret as follows: ‘in case of DOI (vipratiṣedha), the right-hand side (para) operation (kārya) prevails.’ I support my conclusions with both textual and derivational evidence. I also discuss my interpretation of certain metarules teaching substitution and augmentation, the concept of aṅga, and the asiddha and asiddhavat rules and expound on not only their interaction with 1.4.2 but also their influence on the overall functioning of the Pāṇinian machine.

I should note that one of the world's greatest authorities on Pāṇini, George Cardona, taught in the linguistics department at Penn.  Supposedly he could recite the Aṣṭādhyāyī from memory, parts of it even backward, syllable by syllable, so it is said.  I do not know that for a fact, but I did hear George flawlessly and effortlessly recite long sections of the text. When I left Harvard (where they had one Sanskritist) to come to Philadelphia in 1979 — taking a pay cut in the process — there were half a dozen other distinguished Sanskritists at Penn.  Together, they produced many of the greatest Sanskrit scholars and teachers in North America for the next three decades.  Being at Penn in those days was like being in Sanskrit heaven.  It's still a great place for Indian studies.

 

Selected readings

[h.t. Don Wyatt and Angela Sheng]



16 Comments

  1. mg said,

    December 16, 2022 @ 12:24 am

    I didn't think anyone ever took a pay cut leaving Harvard! At least not when you factor in relative costs of living between Boston/Cambridge and elsewhere.

  2. martin schwartz said,

    December 16, 2022 @ 1:48 am

    "'… translate any word written in Sanskrit" ???
    The pun on the grammarian's name
    and Italian panini, which means 'sandwiches', was also made by a Sanskrtist,Jeffrey Mousaieff Masson, who co-ownd a an Italian-style sandwich shop by that name in Berkeley.

  3. Scott P. said,

    December 16, 2022 @ 11:39 am

    I'm very puzzled — if people were not applying this rule for the last 2500 years, then this rule may as well have not existed, so what does figuring out mean, other than we learn what one particular grammarian wanted to say, but failed to communicate? It seems to me that the 'wrong' rule is the 'right' rule, at least as far as practice is concerned.

    I mean, if we found out Samuel Johnson put the wrong definition for 'puffery' in his Dictionary, we wouldn't all go out and suddenly use the one he intended…

  4. Rodger C said,

    December 16, 2022 @ 12:56 pm

    Scott P.: If I have it right, Panini said that it was a matter of "earlier" and "later" rules, and everyone's been assuming he meant "in the grammar," but he meant "in the word."

  5. Jonathan Smith said,

    December 16, 2022 @ 1:57 pm

    @Scott P. I think because Panini presented in effect a *descriptive* Sanskrit grammar — cf. e.g. from earlier threads here the "rules" regarding English modifier order which no one knows but everyone follows. This author's claim is that Panini's description is made more accurate by the proposed interpretive adjustment

  6. Scott P. said,

    December 16, 2022 @ 2:10 pm

    Jonathan —

    I am just trying to figure out why this is so significant. I'm imagining a Latinist finding a better manuscript of Marcus Valerius Probus that clears up a point about noun declensions, but would that really revolutionize translations of, say, the Aeneid?

  7. DJL said,

    December 16, 2022 @ 3:17 pm

    An Italian-style sandwich shop? A paninoteca, then.

  8. Fred Smith said,

    December 16, 2022 @ 4:42 pm

    Victor, I totally appreciate what you said about Sanskrit at Penn. Those were among the best years of my life. As for this issue, Rajpopat's diss can be downloaded here:

    https://www.cam.ac.uk/system/files/rajpopat_phd_thesis_15_dec_2022.pdf

    Many legitimate questions have been asked about this on the Indology listserve in the last couple of days, but the consensus has been that he's on to something. Pāṇini's system is thorough and thoroughly arcane, and not for the faint of heart. Rajpopat is clear in his intro chapters, which are all I've looked at so far. Very few Sanskrit grammarians (vyākaraṇa scholars) are equipped to discuss these maters, but of course, as you point out, Victor, the inimitable and ageless George Cardona is one of them.

  9. J.W. Brewer said,

    December 16, 2022 @ 4:50 pm

    I would be interested in hearing more about these universities that refuse to award linguistics degrees to anyone who has not dipped a toe into Sanskrit. I can sort of imagine that as of a century or so ago, but can't say I've heard of it more recently. OTOH I do regret not taking Sanskrit as an undergrad, so maybe the linguistics dep't should have nudged me, rather than leaving me to my own devices as to dead IE languages of interest (three different varieties of ancient Greek, Old Norse, and a completely failed attempt at Old Irish).

  10. Victor Mair said,

    December 16, 2022 @ 6:12 pm

    Offline, Fred Smith added:

    Rajpopat's diss has created a bit of a stir, which is unusual for dissertations anywhere. I like the name Rajpopat. In Marathi popaṭ means parrot. I knew an old paṇḍit in Nasik long back known as Popaṭ Śāstrī, because of his flawless memory and his constant babbling of Sanskrit. It's possible that our young Cambridge scholar is either a royal parrot or descends from a lineage of vaidikas. Hmmm.

  11. Miriam Dexter said,

    December 16, 2022 @ 7:46 pm

    Thank you so much for this, Victor. The opportunity for me to study Sanskrit while still a Classics undergrad led me to Indo-European Studies and a wonderful life path. I can only imagine what it would have been like to study Sanskrit at Penn.

  12. Kristian said,

    December 17, 2022 @ 5:01 am

    I am with Scott P., puzzled. Reading the description in the post, it makes it sound as though Rajpopat had deciphered some ancient cryptogram. But Sanskrit is a language that people know, right? So apparently he has discovered what some ancient grammarian meant? But why couldn't scholars discover that rule for themselves?

  13. Jason B. said,

    December 17, 2022 @ 10:23 am

    Please excuse a question from a non-Sanskritist. The way Rajpopat interprets the rule as "right" and "left" seems to assume writing. Does he mean that, or is that just careless expression? It is my understanding that, while Panini knows about writing (lipi) and scribes (lipikara), his grammar is not describing a written system of Sanskrit, but an oral one, in which "right" and "left" would make no sense.

  14. Jerry Packard said,

    December 17, 2022 @ 1:41 pm

    George Cardona – A scholar and a gentleman.

  15. Fred Smith said,

    December 17, 2022 @ 3:56 pm

    In answer to Jason's question, this issue of right and left has indeed come up. Rajpopat's diss advisor at Cambridge, Vincenzo Vergiani, has written about the possibility (in his view the likelihood) that Pāṇini was literate and knew a written language. Not everyone agrees with this, but it's a legitimate possibility. Rajpopat himself cleared up this question on the Indology website yesterday, saying that in his future work, incl revisions to his diss, he would be more explicit. What he meant was earlier or later, temporally, than articulation of the word base, earlier being left and later being right.

  16. crturang said,

    December 18, 2022 @ 12:16 am

    >> What he meant was earlier or later, temporally, than articulation of the word base, earlier being left and later being right.

    Yes. paraM can mean temporally further – gIta 4.4 has aparaM bhavato janma paraM janma vivasvataH.

RSS feed for comments on this post