Word lens
« previous post | next post »
Competing with Culturomics for meme room today is Word Lens, which has a great YouTube ad:
According to Sam Grobart at the NYT Gadgetwise blog ("Una App loca", 12/17/2010), the reality is a bit rougher around the edges. He tried
El Gobierno avisa: “A la comunidad que no cumpla con el déficit, le va a hacer cumplir”
and got
PI GOVERNMENT ALERTS: “TO THE COMMUNITY THAT NOT CHERISH WITH el DEFECIT HIM WILL TO DO COMPLY”
I haven't yet found a reviewer who tried it on a realistic set of signs, menus, etc.
[Update — this person gave it "Dead End" in English and got "muerto fin" back in Spanish. Not so impressive. (The author of the post was impressed, but this seems to be because they didn't know any Spanish.) So far, it looks like a cool idea hobbled by poor-quality translation software — this isn't surprising given that the translation engine runs on the phone. Coupled with competitive MT running in the cloud, this might start to live up to the ad — though other reviews suggest that there are still some OCR issues as well.]
Cialan said,
December 17, 2010 @ 7:33 pm
The Spanish on some of the signs in that ad was obviously translated from English by someone who didn't know what they were doing, and probably by Word Lens itself. For instance, they used *"ropas" instead of "ropa" for "clothes," failed to make the adjective "opcional" agree in number with their newly-invented plural noun *"ropas," and used the object pronoun "lo" as a subject pronoun a couple of times (instead of just omitting the subject "it") in order to get the English translation to say "it." I wonder how it would do with authentic Spanish. Probably not very well, judging by its complete inability to comprehend gender agreement, as evidenced by *"el otro dirección" instead of "la otra dirección."
That said, this spells doom for academic integrity on language tests. A quick swipe of the phone over a paper or computer screen ("I was just checking the time!"), and the meaning becomes clear.
Chris said,
December 17, 2010 @ 8:12 pm
Yes, upon further investigation, the MT part fails to impress. And the fact that the initial roll-out only includes the Spanish-English pair is disappointing (though understandable). Also, the web site states that the tool "does NOT recognize handwriting or stylized fonts"; nevertheless, if this technology improves over the next 5 years or so to a point where it can do what it currently claims it can do, it's a spectacular advance. However, I won't truly be impressed until my Droid X can download it.
Keith Trnka said,
December 17, 2010 @ 8:31 pm
A friend and I played around with it earlier in the lab (just the "reverse" demo) and the OCR was reasonable on a textbook page and the white on dark text on a coffee can. But the OCR was poor on a textbook cover (the parts with an image background) and the low-contrast text on the coffee can. At least you can try out the English OCR part for free.
Somewhere I saw said it was just one-to-one dictionary lookup for the translation (though there wasn't much evidence given). Part of me thinks that the simple translation is related to their effort to re-render the text; something too long or too short might look unnatural.
As far as cloud-based MT like using Google Translate as a backend, it probably really depends where you are. If you're talking about Spanish-speaking restaurants in Philly it's probably fine, but I imagine data rates in Spain might be bad. Actually even if it's local, a second of latency might be bad enough to break the illusion of augmented reality.
@Cialan
I'm not sure if I agree about cheating. It's another way to do it, but you can already use Swype in other languages as input to a web translator, right? I think it even adds diacritics for you.
Plus the simple fix is to hand-write your exams and photocopy them, cause Word Lens can't handle handwriting. (I'd guess that it can't handle anything very different than it's font library)
Paul W said,
December 17, 2010 @ 9:36 pm
I think the technology here is incredible. So the translation isn't perfect. The visual processing is incredible. The fact that this idea is implemented this well on a mobile phone should stun you. I'm really surprised you're being so critical. Improving the translation quality will be the easy part.
J. Goard said,
December 17, 2010 @ 9:43 pm
Hehe, good luck with that one, dudes.
I'm currently looking at a hilarious "Fire Escape Device" box in original Korean with translations in English, Japanese and French. That is, except for the final bullet point, which replaces the French with German. Also, whereas none of the German nouns are capitalized, two English nouns are.
Funniest sentence:
벽면을 안전하게 타고 내려간다.
Descend your body facing the wall.
Apres avoir examin'e les murs descendez.
The English is not too bad, but the French (with additional errors) suggests that you're supposed to look at multiple walls before going down. The walls of your room?
These things are baffling. Here's a gloss of the Korean:
벽면을 안전하게 타고 내려간다.
wall-face-ACC safely ride-CONJ descend-go
'(you) ride safely down the face of the wall'
So where the hell does "apres" 'after' come from in the French? And where does he awkward "descend your body" come from in the English? It sounds like a literal translation from a language that would have to express "body", but Korean need not, and does not here. Best of all, why is "safely" removed in both translations?
I should try again to get a good quality cell phone photo…
Twitter Trackbacks for Language Log » Word lens [upenn.edu] on Topsy.com said,
December 17, 2010 @ 9:57 pm
[…] Language Log » Word lens languagelog.ldc.upenn.edu/nll/?p=2849 – view page – cached December 17, 2010 @ 7:21 pm · Filed by Mark Liberman under Computational […]
Chris said,
December 17, 2010 @ 11:12 pm
I can't help but be reminded of Cuil, the "Google-killer" search engine that had a gorgeous interface, with beautiful flipping pages as a search result. Too bad they failed completely and entirely. I love the Word Lens interface, but there needs to be serious content underlying it.
Mark Mandel said,
December 18, 2010 @ 12:48 am
From what I could see, the translations in the ad were word-for-word. The clearest violation of a word order constraints was at about 0:42ff, "Tongue Bolivian with a sauce spicy of anchovies": twice, Spanish standard N+Adj rendered as english *(N+Adj); otherwise, they avoided the problem.
Teresa G said,
December 18, 2010 @ 1:10 am
@Paul W
"Improving the translation quality" is never the easy part. MT that passes the Turing Test is still very far off in the future. People often assume that what's easy for humans must be easy for computers, but it's almost always the opposite: the easier it is for humans to do, the harder it is for computers. Never more so than when it comes to human language.
Obviously making the graphics snazzy was the easy part, since that's what they succeeded at first!
C Thornett said,
December 18, 2010 @ 2:39 am
In the UK, Cialan, mobile phones are banned from formal examinations and have been for some years, to prevent people photographing the exam paper or transmitting the content of an oral examination.
There are some on-line reading exams for adult literacy, but so far as I know, no language exams allow the use of computers unless special needs mean a candidate is unable to write. We asked, and were told that it would be too expensive to provide enough secure computers for written language exams.
The examples would reveal that technology had been used–but then, teaching people to translate sense or at least phrases rather than individual words takes time and effort.
Lance said,
December 18, 2010 @ 3:43 am
There's no question that the visual side of the technology is really very impressive. But, honestly, Paul W, this is Language Log, not Image Processing Log; of course the focus of criticism will be on the language part of the app as opposed to the image processing.
Incidentally, relative to Mark's observation about word order, I think the signs were chosen very savvily in that respect. Including the one with the wrong word order–a minor mistake, easy to comprehend, actually makes the app look more impressive, because a viewer will think "Ah, look, these are real translations, not just something preloaded for the sake of the ad, or they wouldn't have included mistakes!"
Cialan said,
December 18, 2010 @ 5:08 am
Thanks for the feedback, Keith and C. On a more positive note, this app could be a good tool to use in a language class for the purpose of demonstrating visually why it isn't a good idea for students to translate word-for-word when learning a second language (that going the other way results in the awkward English *(N+Adj), and so forth).
Pflaumbaum said,
December 18, 2010 @ 6:55 am
To be fair, at least they're not trying to hide the inadequacies of the translation. Credit to them for marketing it for what it (currently) is, a quick, rough guide to the gist of a text.
MattF said,
December 18, 2010 @ 7:53 am
Here's a link to the developer's website with some more info:
http://questvisual.com/
Note that the developers are apparently image processing/gaming types, and note the self-imposed constraints– in particular, that the app should work with no network connection. I'd bet they're thinking of hiring a language specialist.
john riemann soong said,
December 18, 2010 @ 8:17 am
There should be a web-based option though, to improve the translation.
KCinDC said,
December 18, 2010 @ 2:08 pm
Pflaumbaum, doesn't using ungrammatical Spanish for the signs so the English translation comes out looking better count as trying to hide the inadequacies?
Cialan said,
December 18, 2010 @ 3:08 pm
It's possible that they're using ungrammatical Spanish so the English comes out better, but it's also possible that they didn't realize their Spanish was ungrammatical, if Word Lens produced it in the first place…
Nelida said,
December 18, 2010 @ 6:25 pm
The crux of the matter lies in what is the purpose of the APP. Is it image-capturing or is it translating? If the first, then, OK, but many apps do that already; so I rather suspect it's the second, and then it is not impressive at all. Awful translations. I think that they rushed the launching, should have waited to refine the translation tech. Truly doubtful to compete with us human translators. Therefore, I am siding in this with @Teresa G's comment above.
richard howland-bolton said,
December 20, 2010 @ 7:25 am
@Cialan
Ah!
The opposite of the famous 'invisible imbecile'??
Nìall Beag said,
December 20, 2010 @ 4:40 pm
Looking at the blogosphere on the whole, it seems to me that there's two very different camps in this debate.
In the red corner, Team "It's rubbish at language, connect it to Google Translate" and in the blue corner "It's OK that it's rubbish at language, cos it works offline".
Personally, I don't subscribe to either viewpoint.
Google Translate is overkill for this — even a very basic transformational grammar program would do, and the iPhone has the processing power for it.
The usual complexities of grammar can be cast aside, as the material this is designed to deal with is pretty much just signposts and menus, and as such you can make a lot of assumptions.
For example, the current software assumes "fries" is what I'd call "chips", and translates it as such (the Spanish for fries/chips being literally "fried potatoes"), because the designers assume you're unlikely to find the third-person-present verb (he/she/it) "fries" in any of the target text types, and very likely to meet French fries.
It's a very good assumption, but it's still far from perfect, because it forgets that "fries" often comes after "french", so you'll often end up with something equivalent to "french fried potatoes" in Spanish, and most Spanish speakers will probably assume that it's some kind of fancy French dish.
But the leeway for assumptions means that this is perfect, as I said, for transformational grammar based translation, and they really should have talked to the big names about a partnership rather than release a half-finished product onto the market.
leo lyons said,
December 21, 2010 @ 7:11 am
Us English speaking nations are already notorioulsy crap at foreign languages this app means we don't have to even learn what the signs mean. I can imagine brits driving down spanish autoroutes holding up their i-phones at every traffic sign…and I do think they have been selective with their choice of signs.
Thomas Matthews said,
December 21, 2010 @ 12:47 pm
First step on the road to Star Trek's universal translator.
Will Steed said,
December 21, 2010 @ 9:51 pm
It's a nice beta for future stuff. It is the first step, combining OCR, translation and refitting text in one go, of a new style of translator, so I wouldn't have expected perfection.
There's a very LONG way to go, but points have to be given to the creators for putting a new set of features together in one lot.
Matt McIrvin said,
December 22, 2010 @ 10:48 am
It's less interesting as a translator than as an impressive shot at "augmented reality" technology.
There should be one that just translates everything to BAD WOLF BAD WOLF.
Kimi said,
December 22, 2010 @ 1:27 pm
@Niall Beag:
You bring up the very good point of regional differences causing problems, in both the English and the Spanish; for example, in Mexican Spanish, french fries are actually often called papas a la francesa, to distinguish them from papitas, "chips" in American English (crisps, I think, in BrE), which would probably be translated as "little potatoes," and either way not be immediately understandable.
Otavio Good said,
December 23, 2010 @ 2:22 am
Hi, I just had to chime in because I'm one of the developers of Word Lens. It was an idea I had 2 years ago. I was never trying to make this app do perfect translations. I was trying to make something that would be useful for tourists. Sometimes the translations can be pretty terrible, but most of the time they are useful if you are a tourist. An example is a restaurant menu. If you can just translate a few of the nouns, it will be a big win. Chicken, tomato, olive. Win. Think of this as a dictionary with a good interface, because that's what it is. It does word-for-word translations with a few exceptions. You might even have to think, like you would if you look up a word in a dictionary. :) And yes, I am looking to make the translation component better.