Duolingo Mandarin: a critique

November 21, 2017 @ 6:06 pm · Filed by under Language and computers, Language teaching and learning, Pedagogy

« previous post | next post »

A friend sent this lifehacker article to me:

"Mandarin Chinese Is Now Available on the Language Learning App Duolingo", by Patrick Allan (11/16/17)

Duolingo claims that it "is the world's most popular way to learn a language. It's 100% free, fun and science-based. Practice online on duolingo.com or on the apps!"

After reading Allan's article, I sent the following note to my students and colleagues:

Judging from the description in this article, I'm dubious about the efficacy of their method.  Never mind about misleading statements emanating from the author of the article (e.g., there are 1.2 billion native speakers of "Chinese"), they seem to overemphasize individual characters, downplay words, don't talk about sentence structure, grammar, and syntax, and don't give any indication of how or whether pinyin is used.

Has anyone checked this app out?

I received the following response from an undergrad who has been taking Mandarin for one and a half years:

I have been using Duolingo to supplement my French for a while, and I have often been quite annoyed at the fact that Duolingo did not have Mandarin (though it had Esperanto, Klingon, and other such languages). So, I was pretty excited when I saw that they were "hatching" Mandarin and, as soon as it was in beta, I started using it. I took a placement test, then started a lesson, but was disappointed with the format because it seemed oddly structured, as it introduced characters without saying their meaning. Also, some of the translations into English are a bit awkward and particular.

After seeing your email, I remembered that the online version of Duolingo is more comprehensive (than the phone app version) — it generally has grammar and pronunciation notes for each lesson set. I went through some of the grammar notes for the lesson sets, and I think they're somewhat helpful. Also, when looking around, I saw that they did not have a comprehensive explanation of pinyin. I was pretty disappointed about that because pinyin was incredibly helpful when I started learning Chinese and continues to be. After seeing how Mandarin works on Duolingo, I'll probably only (sparingly) use it for review and extra vocab.

To be more positive, Duolingo's Mandarin is only in beta right now, so in a while it may be better.

Here are some more comprehensive articles about Duolingo's Mandarin:

"Duolingo Has Mandarin Now. Can It Really Teach You the Language?", Aaron Mak, Slate (11/18/17)

"Duolingo now supports Chinese, but it probably won’t help you become fluent:  To be fair, learning Chinese is hard", Shannon Liao, The Verge (11/16/17)

Silas Brown then sent me this evaluation:

I tried saying I was a beginner.  It put me into a multiple-choice test, asking if 你好 is nǐhǎo, hǎo or zài.  Then it did similarly for just 好. Then played recordings of nǐhǎo and hǎo, and had me select the pinyin from a multiple-choice list.  Then had me select the correct character for hǎo out of 再, 好, 你 and 见.  Then it had me type the English word for 好. And pair up 你好 with nǐhǎo and 好 with hǎo.

Not exactly what I'd call "learning Chinese" .. but it might give some people a 'nice feeling' that they're making progress.

Then I tried the "placement test".  It asked me to translate 你叫什么？ and 我也很高兴认识你, then asked me what means "incorrect" giving me buttons to insert 哪, 对, 国, 都, 不 or 香港.  Then similarly asked me to translate "is tomorrow Sunday?" from a set of hanzi, and "He drinks coffee at 6:15 in the morning."  etc.  (When asking to translate sentences like 你认识到医院的路吗 into English, the source is presented as hanzi with no pinyin, but with audio available.)

I hit the first snag when I translated 我的朋友都喜欢跳舞 into "My friends all like dancing" and was told this answer is incorrect because the correct answer is "All my friends like dancing".

(I didn't think my style of English was the thing under test here. If this is supposed to be a self-study aid, then there really ought to be a button saying "I was close enough, mark that one correct please" for this kind of question.)

All answers have a "report" button, which suggests it's been crowd-sourced. I hope the test doesn't just keep getting bigger as people add more questions / answers to the database. Because there's no way to finish the "placement test" early.  That means if you're starting to find it too difficult, the only way to say "OK, you've found my level, don't ask me any more questions I can't do" is to press the "skip question" button AND acknowledge that your 'blank' answer is incorrect, One By One For Every Single Question Until The End Of The Test.  Nor is there any way to take the questions out of order (i.e., stop asking me easy ones, I want to see how hard it is at the very end).  With all that "5 minutes a day" advertising, they ought to warn people that the 'placement test' has to be sat in sequence, in its entirety, and could take an hour if you're not very fast.

Need to get on with things, so I haven't sat it through to the very end to see what happens.

Taking these two reports and my initial reaction into consideration, I have little doubt that Duolingua Mandarin is better than ChinEasy, but it still leaves much to be desired as an effective means for becoming fluent in Mandarin.

On the difficulty of learning to read and write Chinese and how to cope with it:

Tips for how (and how not) to learn Chinese:

Learning Literary Sinitic / Classical Chinese is a rather different matter from learning spoken and / or written Mandarin, but with some of the same problems:

Make no mistake about it, learning how to read and write Chinese is extremely hard, though I personally think that learning to speak and understand Mandarin is relatively easy (I haven't forgotten that I need to write a follow-up to this post [and I will!]):

So, back to Duolingo.  What's the verdict?  Let me put it this way:

Bad apps are worse than no apps because they waste time, while frustrating and misleading the user.

Bad pedagogy is worse than no pedagogy because it wastes time, while frustrating the learner and destroying their confidence that they can realize noticeable progress within a reasonable period of time if they make an honest effort.

Best:  substitution and pattern drills, build-up exercises, and anything else that encourages the student to compose and understand phrases, clauses, and sentences beyond the mere lexical level.

Worst:  dictation and focus on words and characters in isolation, which, alas, are emphasized all too often by old-fashioned pedagogues.

Two basic rules:

  1. Emphasize spoken language (pronunciation, vocabulary, grammar, etc.) during the initial stages.
  2. Utilize the best and most advanced electronic aids for reading and writing after you have acquired a solid foundation in the spoken language.

[h.t. Michael Carr; thanks to Rhosean Asmah]

November 21, 2017 @ 6:06 pm · Filed by under Language and computers, Language teaching and learning, Pedagogy


32 Comments

  1. J said,

    November 21, 2017 @ 6:09 pm

    Duolingo is not a good app for Esperanto. That's the good news. It's not TERRIBLE for Esperanto. But you won't really learn Esperanto from it. It'll just help reinforce vocabulary.
    Now, regarding Chinese…

  2. Ryan said,

    November 21, 2017 @ 6:49 pm

    I've been peripherally associated with Duolingo for a few years now, and I'd have to say that Silas Brown's criticism of the placement test isn't exactly valid. As I understand it, the placement test is meant to get easier if you miss a lot of questions, so if you skip questions, you should eventually start getting questions that you should be able to answer. Second, the report button is meant to alert whoever maintains each language of equally valid answers. From my experience, languages that are in beta are quite fast at accepting new answers, so it wouldn't surprise me if "My friends all like dancing" is now considered acceptable.

    Oh, and the reason why Duolingo had Esperanto and Klingon before Mandarin is most likely because Duolingo gravitates toward languages that use the Latin alphabet, which would explain why Vietnamese was one of the first Asian languages to be implemented.

    From my experience, Duolingo is good for those who want to learn "tourist questions", i.e. "Where is the hotel?" or "How much is this?" In other words, you can't use it part and parcel to learn a new language. It really only serves as a stepping stone, nothing more and nothing less.

  3. CyberiaGirl said,

    November 21, 2017 @ 9:12 pm

    I was using Duolingo to learn Korean from scratch, and it wasn’t going well. When they added Mandarin I tried that instead, thinking that it would be a good way to reawaken the couple of years of formal lessons I’d taken. It’s doing a great job of that so far. So maybe it’s an app to help you bring back the memory of the language they taught you in high school that you never really used.

  4. maidhc said,

    November 21, 2017 @ 11:48 pm

    I don't know how Duolingo would be for Mandarin. It's not bad for European languages, although I think some languages can be better than others. The German bombards you with new vocabulary which doesn't get reinforced enough, and the Irish has some errors in it.

    It's basically a drill generator. It works its way up to some fairly complex sentences. Maybe something like "I don't need one of those because I already have one at home". Mixed in with the drill there is a slow drip of grammar lessons.

    The one thing it doesn't really do is help you with your pronunciation, although possibly there is a way to use it with a microphone that I don't know about.

    I think it would be best to supplement it with something else, preferably a real teacher, but failing that a good introductory textbook and some of those oral language lessons that are more based on following a continuing story.

    For languages with accents you have to enter accented characters using the mouse, which is mildly irritating. I guess it would be fine for pinyin. I don't know how it works for Cyrillic, Korean or other such writing systems.

  5. cliff arroyo said,

    November 22, 2017 @ 1:21 am

    "For languages with accents you have to enter accented characters using the mouse, which is mildly irritating"

    Actually for languages using the latin alphabet most of the time you can just stay with ASCII. Even the Vietnamese course you can enter toi co mot cai mu and it will accept it while pointing out the 'typos' in your answer.

    The weirdest course for me was Hungarian which for some reason generates very weird and long sentences very early on. I think long and reasonably normal or short and weird both have their places but the course goes too far too soon.

    for languages that don't use the latin alphabet it mostly seems (looking some at the Hebrew and Modern Greek and Mandarin) to expect you input answers in the original script – maybe not always that easy early on. The Greek lets you do some input but only in english based phonetics which is maybe not the optimatl romanization for such a course…. ime instead of eimai (which can help reinforce the spelling in Greek)

    As someone who knows only a few words of Mandarain (and has a general idea of the outlines of the grammar) I only did the first two units was very surprised that you can't use pinyin to enter for the Mandarin course. I was also surprised at the bizarre emphasis on characters in isolation without any meaning given and quickly tired of the many matching exercises.

    Overall I think duolingo in general works better as a way of refreshing knowledge than active learning and even then it shouldn't be the only thing a person uses.

  6. reader_not_academe said,

    November 22, 2017 @ 5:08 am

    there is a very reasonable writeup of duolingo chinese by nikhil sonnad: https://qz.com/1128185/how-to-learn-chinese-duolingo-has-finally-released-a-chinese-course-heres-a-guide-to-getting-the-most-out-of-it/

    i liked that text because it's fairly honest about the app's limits, makes suggestions about using it as one learning tool among many, and helpfully includes links to a number of these.

    unfortunately, even this text falls short of challenging the misguided focus on characters instead of the language per se.

  7. Hans Adler said,

    November 22, 2017 @ 5:41 am

    I hope it's OK if you tell you about my experience with Duolingo. I think it's quite relevant to judging the new Chinese course.

    I used Duolingo to learn Spanish with reasonable success. Then I used the Duolingo English course for Dutch speakers to learn Dutch in about a year. (This was before the Dutch course for English speakers came out; being a native speaker of German obviously helped.) And then I learned Esperanto on Duolingo in just a few months. I now enjoy reading books in Dutch and Esperanto and newspaper articles in Spanish.

    The efficacy of a teaching method depends on various factors on the side of the student. For me, Duolingo has consistently been the best method available – probably because it feels like a game and keeps drawing you in. Thus the most important factor – how much time you actually spend learning – is optimised, provided you are susceptible to gamification. I have tried various other methods before, but with all of them (other than English and French in school and a three-week full-time crash course in Chinese I once attended) I lost interest rather quickly.

    The methodology is also fine. You spend most of your time translating sentences between source and target language. This is basically done through a graded list of sentences to translate, and for each of them a list of accepted answers that can be extended through user feedback (only available on some platforms, including at least web and Android). Interestingly, like in real-life immersion, Duolingo mostly doesn't explain anything but teaches new words and concepts by asking you to translate new sentences for which you still lack the knowledge. You then learn by looking at the solution. In my experience, this works orders of magnitude better than Rosetta Stone (which feels similar overall but tends to bore me to death), and still significantly better than more traditional computer courses.

    When you get stuck, there is a discussion forum for every single sentence. If you ask a serious question there, you usually get a helpful response within a few days. Apart from some static pages and some posts in central forums, this is the *only* place where grammar is explained in Duolingo.

    The reason Duolingo is still weak on languages with non-Latin-based writing systems is that these require developer work. New language pairs in Duolingo are mostly (and in the case of exotic languages such as Esperanto and Klingo exclusively) developed by volunteers (with only very little developer activity on thinks such as workarounds for missing accents on keyboards), then improved through user feedback. That's very cheap for Duolingo. But the first course each in languages with right-to-left and ideograph-based scripts cost Duolingo significant amounts of money.

    I am sure that a Pinyin-based Mandarin course on Duolingo would be very effective for most users, and they must have known that. The main problem Duolingo had to focus on to provide a full course was thus the necessary support for learning the characters as well. (So far only simplified characters are supported.)

    The reason they don't talk about words, sentence structure etc. in the case of Chinese is therefore (1) that from the point of view of Duolingo these are all solved problems anyway, and (2) their solution consists in mostly not making them explicit to learners.

  8. Victor Mair said,

    November 22, 2017 @ 7:49 am

    From April Zhang (Zhang Xuehong):

    I have been teaching Mandarin Chinese in HK for years now. I can relate to what you said. Also, I tried Duolingo on two other languages, and have realised that it does not work.

    I took a year Spanish lessons long time ago. Last year, after discovering Duolingo, I thought it would be cool to continue my lessons using latest technology. It was at the beginning pretty good as a revision tool. But the format is so repetitive, and focusing so much on singe words and single sentences. I have been way down to the lessons (the app gave me a 40% fluency rating), everything stayed the same! And there is no way this 40% is a actual reflection of my Spanish ability. Eventually I lost interest and motive.

    However, I tried again with German a few months ago before my trip to the Frankfurt Book Fair. (I know, the definition of insanity) From this two month learning experience, I realised that learning a new language from scratch with this app is impossible. The words and topics are so random and disconnected. And the brain-damaging repetition drove me crazy. I did learn a couple of phrases and used them in Frankfurt. That I found out that I could easily pick up in Germany.

    After reading your post, I tried Chinese, starting as a beginner. Judging from my own teaching experience, there is no way any beginners will be able to keep up.

  9. DWalker07 said,

    November 22, 2017 @ 11:02 am

    @Ryan: Yes, but if the initial test takes an hour, then a claim of "5 minutes a day" is not valid!

  10. Yan said,

    November 22, 2017 @ 1:03 pm

    I've spoken Esperanto for several decades and I'm active in my local club. We've been seeing people coming to the club for the first time after using Duolingo and speaking remarkably well, even speaking fluently even though they've never spoken Esperanto with another person. The difference compared to beginners who use a book or a correspondence course is astounding. Whatever Duolingo is doing, it works very well for Esperanto.

    Regarding Silas Brown's complaint that Duolingo didn't accept his correct answer, the problem is common with all computer instruction and probably unavoidable. There can be many correct ways to say the same thing: a human instructor can recognize and accept unanticipated answers, but a computer can't.

  11. david said,

    November 22, 2017 @ 3:23 pm

    Following Hans – Duolinguo makes learning languages a game. Playing a game is a better way to learn about it than being told how to play. If you really want to know the "rules" of a living language they are available elsewhere on the internet at varying levels of sophistication and including competing versions. Indeed some of these are referred to in the forums Hans describes.

    For me the nicest Duolingo experience is when I discover I can answer the questions without knowing the rules but rather just because it seems like a better response.

  12. cliff arroyo said,

    November 22, 2017 @ 3:49 pm

    "Silas Brown's complaint that Duolingo didn't accept his correct answer"

    Duolingo does give users the option of reporting that new accepted answers are added (not tremendously regularly but it does happen).

    But some mistakes remain unfixed (a few I reported have been generating complaints for a couple of years with no changes).

    Also, sometimes the accepted translations (into English) depend on what seem to be rules learned by non-native speakers who contribute a lot of the material. I can't think of an example off the top of my head but it's as (the following is an imaginary example but similar to some hiccups I've found in acceptable English translations of target langauge sentences) if speakers of languageX were taught that there's an important difference between someone and somebody and so in languageX's course translations aren't accepted if you use someone where they were taught to use somebody….

  13. Anne Henochowicz said,

    November 22, 2017 @ 3:55 pm

    I downloaded Duolingo before a trip to France this summer. I used to speak French fairly well, but haven't used it much since I devoted myself to Mandarin. In the middle of Duolingo's diagnostic text, it asked me to translate the sentence "Nous sommes samedi ou dimanche?" (Are we Saturday or Sunday?) That was the end of my experiment with the app.

  14. Yan said,

    November 22, 2017 @ 4:09 pm

    Anne Henochowicz: "Sommes-nous samedi ou dimanche?" would be better, but I suspect you think, erroneously, that "nous sommes samedi" is incorrect.

  15. Alyssa said,

    November 22, 2017 @ 4:46 pm

    At least we've moved on from the days when Rosetta Stone would mark my Japanese incorrect because I'd put the spaces between words in the "wrong" places…

  16. Marcos said,

    November 22, 2017 @ 6:39 pm

    Duolingo was hugely helpful for me as a beginner in Portuguese. Some people may not appreciate the format, but I was able to go from no knowledge to being able to work my way through newspaper articles in just a few months. After that, just through lots of reading in Portuguese and conversations with Brazilians, I was able to reach rudimentary conversational fluency in about 6 months total.

    Subsequently, I used it for French and Catalan with similar results. I can't understate how convenient it was for me, but of course it isn't a stand-alone ticket to full conversational fluency.

    More recently, I tried to use Duolingo for Vietnamese (which I haven't studied previously) and Mandarin (which I studied previously for a couple semesters).

    Duolingo was great to learn Vietnamese vocabulary and start developing an intuitive feel for sentence structure, but it really falls down when it comes to pronunciation. I had to make my own separate study materials to drill myself on Vietnamese pronunciation and particularly on tones, since this is absent from the course. I got frustrated with how Duolingo teaches Vietnamese exactly the same as Western European languages and gave up (although I hope to go back to Vietnamese at some point when I have time for a more rigorous, non-Duolingo study of the language). I really felt like I was learning Vietnamese badly because I couldn't remember which words had which tones. (My grandfather learned Vietnamese to fluency in the 1950s or 1960s, but he passed away before I could ask him for any insight)

    With Mandarin, I didn't get very deep into Duolingo lessons but I was immediately struck by the focus on hanzi. There are also no tone drills or, for that matter, any pronunciation material at all. Essentially, you're expected to learn pronunciation from listening to the voice recordings and imitating, and from the dictation exercises. This works fine for English speakers with discerning ears who are learning Romance languages, but it's simply not enough input to teach a speaker of a non-tonal language to learn to produce and recognize tones.

    I think Duolingo's greatest downfall is a one-size-fits-all approach to language learning. The only modifications they seem to have implemented to their system for Mandarin are related to teaching hanzi. On top of that, they only drill on recognition of hanzi. They don't teach how to actually write hanzi, or any relevant information about hanzi beyond being able to pair individual hanzi with pinyin or a sound recording. (Radicals? Stroke order?)

  17. boiko said,

    November 22, 2017 @ 6:44 pm

    I've finished the tree for German and about half the tree for Norwegian, plus some of the High Valyrian. Here's my assessment of Duolingo.

    The good:

    – Basic interaction mode is text-based, but all sentences are read aloud (sadly, not for HV). This helps reinforcing the sound-writing mapping in the target language. At the very least, doing Duolingo exercises will get one used to that (you don't have to do a whole tree to get the hang of it though).

    – It's free and require little personal investment. If you're having problems with time or motivation, or you have social anxiety, or you can't find language schools or afford them etc., it's very convenient to just fire up the website (don't use the mobileapp—it skips grammar explanations) and drill something. Duolingo isn't good, but it's better than absolutely nothing.

    – Can be a way for complete beginners to get started with the very, very basics of grammar and a tiny smattering of vocabulary.

    The bad:

    – Mind-numbling BORING. The boring-est boring thing to bore bored bores. It manages to be boring-er than traditional language classes, which is quite the feat. Finishing an entire tree basically counts as an extended exercise in psychological self-harm. Rather than make learning interesting, it tries to cajole you into pulling through your daily excursions to Tediumland via abuse of gamification tricks—badges and running streaks and cute animals begging you to practice and so on. The result is a good demonstration of the limitations of gamification tricks.

    – No explanation of phonology (what sounds are particular to target language, how to articulate them, and how are they mapped to orthography). Teaching adults a foreign language without instruction in phonology creates "phoneme blindness" and fossilization. No explanation of orthography either. No course design to practice those thing.

    – Horrible course design in general, for about everything.

    – Translation-based. Originally a translation app, Duolingo never moved away from its roots, and translationese pervade the entire approach to its great detriment. For example, whenever there are multiple ways of translating something from L2 to L1, Duolingo will use a fluent, adapted translation close to natural L1, which is of course exactly the opposite of what it should be doing. It generally drags you back to the L1 frame of mind all the time. Idioms and set phrases, for example, are translated idiomatically: German bis bald is rendered "see you later". That's how it should be translated, but that's not what it is; what it is is, "until soon". It doesn't matter that "until soon" isn't grammatical in English; that's what the German is, and unless you can parse it that way you don't know German. Examples like that are endless, and very often I found myself knowing how to translate a piece of German mechanically into a full English sentence but not grokking it. In fact, quite often it actively punishes you for trying to stay close to the L2.

    – Another consequence of translationese is too much focus on output. Duolingo had the potential of being a good way of breaking through the very first stage of language learning, and getting to the point where you can read some simple text. This potential is ruined by the presence of output exercises from the get-go. Spaced repetition algorithms ensure that you waste the most time doing what you get wrong most often, and of course that will be output; so my time with Norwegian was wasted trying to write Norwegian with some tricky preposition (why?), rather than soaking my mind in Norwegian input.

    – Contextless exercises. Just floating, disembodied sentences drilled for hours without end in a vacuum. Repetition is controlled by faulty, abstract, blind algorithms, rather than flowing naturally from real language in context (if you practice a language by reading texts that make sense, or having meaningful conversations, then the words and grammar forms you'll see more often will magically be precisely those that occur more often in natural language). "Duolingo Stories" seems to be a tiny step in the right direction here, but it's pre-pre-alpha at the moment.

    – Dishonesty. The "fluency" meter is a travesty. Treating a full tree as anything more than a single step in a journey of a thousand miles is dishonest, but nowhere is it clear how little you will have learned upon completion. The single study they cite as "scientific proof" that "Duolingo is better than college classes" is ridiculous, and claiming so is preposterous.

    My final evaluation of Duolingo is, I learned more German in a one-week intensive conversation course than with six months of Duolingo.

    Suppose it takes X hours to finish a tree. Suppose you take the same X hours and do literally anything else with your language—Grabbing a dictionary and plowing through a book you love, partial understanding and all? Watching dramas with closed captions? Chatting up with the locals in a bar? Playing Pokémon in L2?: anything you do, you will have acquired significantly more L2 than with Duo. And with intrinsic motivation, rather than chasing after gold stars in a sea of ennui.

  18. Joyce Melton said,

    November 22, 2017 @ 7:03 pm

    I use Duolingo as one tool in my program to learn German. It seems to work pretty well within its limitations and it is free. Other, better computer learning apps with access to conversations with native speakers cost money. I'm using a couple of those, too, and Duolino for drill.

    One good thing about Duolingo? It is a gentle nag to remind you to practice.

  19. Matt said,

    November 22, 2017 @ 8:39 pm

    Leonardo, not to argue with your assessment, but just curious: why did you spend an entire six months finishing the German tree if it was so awful? I can understand giving it a try, but surely after a few weeks or a couple of months at most you realized that (as you put it) "literally anything else" would be a better use of your time — why not switch to graded readers or whatever at that point?

  20. Carol said,

    November 23, 2017 @ 12:31 am

    No one so far has mentioned that there are two alternative Mandarin apps that are Duolingo-like: HelloChinese and ChineseSkills. HelloChinese is the best — it includes voice recognition so that it can give feedback on your pronunciation (very helpful), and also character writing help (not perfect but sometimes helpful). You can set it to show pinyin, hanzi, or both simultaneously. I prefer to practice the vocabulary and grammar with pinyin so as to be able to focus on the sounds.

    I was surprised when I downloaded Duolingo Mandarin in how much more limited it was. I searched for a pinyin display option, but couldn't find it. The lessons start with memorizing the hanzi-pinyin pairs, then you have to read the hanzi while listening to the pinyin during translation practice. No voice recognition or character writing practice.

    The order of topics is spoken Chinese oriented — starting with ta xing Liu, etc — rather than starting with the easiest characters to learn (da, shan, numbers, etc).

    At this point I plan to keep using it because I need hanzi practice and it is very convenient to practice with the app during random points in the day. I also don't find it to be boring — not as engaging as a game or a novel, but not bad.

  21. MTC said,

    November 23, 2017 @ 2:48 am

    I’ve been using Duolingo for 4 years. I find it works very well for European languages at getting me from no knowledge of a language at all to a level at which I’m reasonably able to read books in that language as long as I don’t mind stopping to look up a word every now and then, but I certainly agree that its approach needs to be modified quite a bit for Chinese (and, for that matter, Japanese; and to a lesser extent Korean).

    Duolingo’s courses in Japanese, Chinese, and Korean seem to have been somewhat rushed. The Duolingo community have been asking for each of these languages for years, and Duolingo had been reluctant to add them precisely because they were worried that their existing approach wasn’t well‐suited to these languages. It seems they eventually decided to add some very basic and flawed character‐learning exercises and rush out courses for all three. I have no doubt that they will eventually update the courses to include better ways of teaching the characters (or at least ways that they think are better).

    Reading this article and the comments made me wonder whether the Language Log community have positive or negative opinions on the other free Chinese learning apps that seem better tailored for Chinese than Duolingo currently is. Ones I’m aware of are HelloChinese, ChineseSkill, and LingoDeer. I’ve been using some combination of these apps to learn Chinese for a while now. They seem to be doing a decent job of it, and they each seem to avoid most of the problems reported here with Duolingo’s approach, though they all have their own problems too. I’d be interested to know whether they are seen as negatively as Duolingo; and indeed whether there are any better free ways to learn Chinese.

  22. poftim said,

    November 23, 2017 @ 3:41 am

    What I'd like to know is this: Does Duolingo "calibrate" the frequency of the repetitions (to keep your tree golden, if that's the right terminology) based on your initial session? The first time I used Duolingo to hopefully improve my Romanian, I did a three-hour blitz, but I had no intention of making that a daily occurrence. However my strength bars dropped off extremely quickly and there was no way I could learn new lessons while also keeping my previous skills fully charged, without Duolingo almost taking over my life.

    The gamification worked: after a month I was losing the game badly despite spending at least an hour a day on it (which was starting to feel like a massive chore), so I pulled the plug and haven't been back since.

    In principle I don't think Duolingo is too bad if you're aware of its limitations before you start, and from my experience you should pay no attention to whether your tree is golden, and instead repeat the tasks when you feel like it, otherwise you'll be bored to tears. Some of the Romanian sentences and translations were ridiculous, much like the French example Anne gives above. The worst topic was the one on arcane units of measurement (hectolitres?!) that as far as I'm aware nobody in Romania actually uses. And there were several errors, which were to be expected perhaps for a relatively unpopular language to learn.

  23. Hans Adler said,

    November 23, 2017 @ 5:16 am

    @poftim: My experience with Duolingo has been that it is best to ignore the strength bars altogether and rush through the new lessons. Just use "Strengthen skills" for at least half an hour or so every day, and it should keep the sentences that Duolingo 'thinks' you need to redo sufficiently balanced so as to be useful.

    I also tested what happens when I test out of basically the entire course, such as German for English speakers. I think the problem was significantly worse in that case. Based on my overall experience. I tried to fill all the strength bars, but that would have required doing the entire course anyway. So, basically, testing out and the strength bars are mutually exclusive features. But I think in this case I had the additional problem of being pestered with very early sentences under "Strengthen skills". Apparently, Duolingo 'thinks' that you must have answered each a certain number of times, even those that are way beyond your initial skill level.

    I think the Duolingo team is quite strong on testing features and acting according to the results, even if they are counter-intuitive. For example, I suspect that the number of times you see a sentence is in no relation to how often you got it right in the past, and that this is due to A-B testing. But they may be a bit less strong on asking the right question and probably do nothing to adapt their interface to individual learning styles.

  24. Hans Adler said,

    November 23, 2017 @ 5:31 am

    @boiko: You clearly had no fun using Duolingo, so it's really quite logical that you didn't have much success with it. I have an eye on research on how various factors influence learning. It is well known that intrinsic motivation is extremely important, and if a method is no fun then that's obviously going to suffer.

    One study also found that in schools, total time spent undistracted with a topic is the best predictor for learning success regardless of teaching quality. Thus focusing on user activation and retention rather than improving the method seems to be a reasonable decision by Duolingo.

    Also, a regular change of locations is important for learning success. The Duolingo apps are much more popular than the web interface for obvious practical reasons. I got my first smartphone several years after starting with Duoling on the web. For some years I used the Android app extensively in public transport. Compared to the web interface it is limited, but it's just as effective. I think that's because the weaker teaching method isn't that important, but people tend to spend more time with Duolingo when using a smartphone, and to shift locations more often.

  25. Anne Henochowicz said,

    November 23, 2017 @ 8:03 am

    Yan, I did say my French was rusty :-) But Duolingo definitely presented me with "nous sommes," not "sommes-nous." Even if it had, is that something people would actually say? It seems along the lines of "colorless green dreams sleep furiously"–grammatically correct, but meaningless.

  26. poftim said,

    November 23, 2017 @ 10:06 am

    @Anne
    I'm pretty sure French speakers do commonly say that sentence (either with "nous sommes" or "sommes-nous") to mean "Is it Saturday or Sunday?"
    They also do it with dates: "Nous sommes le 23" means "It's the 23rd".

  27. Yan said,

    November 23, 2017 @ 12:05 pm

    Anne Henochowicz – Now I see that you weren't wondering about "nous sommes samedi", but about the omission of "est-ce que" or the non-inversion of "nous sommes".

    According to the standard French grammar, Le bon usage (13th ed., par. 391), asking a question this way is correct, and extremely common when speaking ("Tu viens?").

  28. James Wimberley said,

    November 23, 2017 @ 5:59 pm

    "Where is the hotel?" does not strike me as a priority phrase for aspiring students of Klingon. My ideal course would be more like the intensive French language course offered to new recruits to the French Foreign Legion, whose officers are all French. The pass rate is reputedly high, as otherwise you get thrown out. The legionnaires do learn how to rocket, machine-gun, or blow up the hotel.

  29. Anne Henochowicz said,

    November 23, 2017 @ 6:48 pm

    Thanks, @poftim and @Yan. I stand corrected.

  30. K said,

    November 26, 2017 @ 9:36 pm

    I'm curious what research there is regarding boiko's comment about "bis bald".

    By this I mean, boiko criticizes duolingo's approach as overly translation-based with the example that duolingo wants "bis bald" translated to "see you later" rather than more literally to "until soon" with the reasoning that the more literal translation is staying more in the L2 mindset rather than mentally switching back to L1 with the more fluent translation. However my intuition is completely reversed: the literal translation seems to break out of L2 because it cuts it up analytically in a way that you would not as a speaker of that language unless your attention was drawn to it. Later in the duolingo tree you do get bis and bald as separate words, so you can then realize what the pieces mean, comparably to how you sometimes suddenly realize (mostly in late childhood but from time to time in adulthood too) what a word or phrase you have used all your life in your native language means literally.

    If it came down to it, I would trust boiko's intuition over mine for his greater overall linguistic expertise, but even better would be some evidence: does literal translation break one out of a language more or less; is it better or worse for learning.

    For what it's worth I found learning German in duolingo very helpful. I strongly agree with boiko's later comment that working through a book you like with a dictionary and partial understanding is distinctly better, but I found that I needed something (and duolingo served well) before I could work through a book that way. Maybe I'm not just not as good at that kind of reading as boiko is or my patience for puzzling through sentences with a dictionary is smaller than my patience for gamified duolingo exercises.

    It also took me 6 months to get the German tree done, but I didn't find it so dreadfully boring. It sounds, though, like I should give duolingo Mandarin a pass at least for another couple of years.

  31. Victor Mair said,

    November 29, 2017 @ 12:23 am

    From Matt Anderson:

    I took a look at Duolingo Mandarin (I used the iphone app), and I was not at all impressed. It is, as your post suggested, impossibly character heavy. This approach might be useful if its goal was to teach people who don’t know any Sinitic language how to read signs or menus or something, but in that case, the vocabulary is all wrong. I can’t imagine it would be at all useful for someone trying to learn Mandarin from scratch.

    That said, I’d previously played around with Duolingo’s Spanish lessons, and I got quite a bit out of them. I’ve never formally studied Spanish, but I know a decent amount of French and I’ve spent time living in Spanish-speaking neighborhoods, so it’s not unfamiliar to me. I went to Mexico for a few days a couple of years ago after playing around with the app off & on for a month or so, and I was able to have simple conversations, which I’d never been able to do before. So I’d recommend it for that. And my wife seems to have learned a certain amount of Mandarin from a Duolingo clone called HelloChinese. It works in almost the same way, but it gives you the option of using pinyin instead of characters, which she sensibly chose, and it even tries to correct pronunciation, including tones, though I have my doubts on the quality of its speech processing. Still, I’d at least not be against beginners using HelloChinese or Duolingo for Spanish — I can’t say the same for Duolingo for Chinese.

    Also, I took the placement test for Duolingo Chinese, and found it really frustrating! Of course I wanted to get every question right, just to see where it’d place me, but it took me many attempts to do so, even though I literally never made a single real mistake, other than some English typos (and they should have given me a break there, as their English is not infrequently unidiomatic). For example, it asked me to translate the sentence “你喝过台湾的珍珠奶茶吗？” I translated this as “Have you ever drank Taiwan bubble tea”, but this was apparently wrong on several counts — they wanted (I’m quoting from memory, but this is at least basically correct) “Have you tried Taiwan’s bubble milk tea?” I’ll grant that “Have you tried…” is more elegant than “Have you ever drank…”, though that shouldn’t be an issue here, but “bubble milk tea” sounds ridiculous (to me) in English. And “Taiwan” sounds better to me here than “Taiwan’s”, though that’s neither here nor there. And many of my other attempts were foiled by similar reactions. For one thing, it unpredictably sometimes called for “the” or “a” and sometimes required their absence in structurally identical sentences. I must have clicked the Report button a dozen times to let them know that my English translation should be considered correct (I never had this problem from English —> Chinese, but the options there were limited by the input system, while Chinese —> English is often open ended).

    Once I finally did pass it with no errors, I still had a half dozen or so lessons to test out of — several on business, one on what to do in an emergency, one on the weather (maybe emergencies and the weather should come earlier in the curriculum?), one on internet slang, and, finally, one on their mascot, Duo the owl. I did perfectly on all of these except for a couple missed questions on internet slang (still good enough to test out, though), but I have to admit that I only coasted through the business questions because of characters. If they had used pinyin, as they should have, I wouldn’t have known how to respond on many of them — I’m not at all fluent in Chinese business terminology, and passing that section because I could read characters felt like cheating. (Additionally, one of my attempts at acing the placement test was foiled because I translated “Duo”, the name of their mascot, in “Duo likes eating oranges” or something as 多 when they wanted 多儿 (you had to select characters from a list to write the Chinese translation); that was frustrating, as I’d never heard of this “Duō’ér” — I thought it was just “Duō”, though that’s kind of a weird name — & I’d almost completed the test.)

    Anyway, that’s my experience with Duolingo Chinese. My advice: stay away. But maybe they’ll adapt it to copy pre-existing Duolingo clones, in which case my advice might be: well, it’s better than nothing. But not yet.

  32. Hans Adler said,

    November 30, 2017 @ 10:47 am

    @Matt Anderson:

    I think it is normal that the placement test isn't very useful yet. It relies on the database of correct translations being essentially complete. But this is never the case for a language pair that has just entered the public beta phase, as is the case for the Mandarin course.

    For Duolingo, a course being in beta doesn't mean: "We think it's perfect, but users may still find some bugs". It means: "We need enthusiastic users to help us complete the database."

    As a related matter, some Duolingo weirdness becomes clearer once you understand how it decides what to present you as a correct answer when it doesn't accept yours. Basically it goes through the database of correct answers and compares each to your answer. How many characters does one have to change/add/remove to get from one to the other ("editing distance")? It then presents you the most similar answer according to this simple heuristic.

    For mature courses, this algorithm often results in surprisingly good suggestions. But when the database is still very incomplete, or when it contains weird answers, it can also result in very poor suggestions. One source of weird answers is when course maintainers erroneously accept mistyped or incorrect user suggestions. Another is answers that are technically correct in weird contexts. And there is another source of weirdness just for English: Whenever a sentence is added to the database of English translations, Duolingo automatically adds variants with all the major contractions, apparently with no safeguards regarding acceptability in context. I wouldn't be surprised to see a suggestion such as "I've'd it" or "I have'd it" (both automatically generated from "I have had it").

RSS feed for comments on this post