A long short-term memory of Gertrude Stein

« previous post | next post »

As just observed ("What a tangled web they weave"), successive repetitions of short sequences of Japanese, Korean, Thai (and perhaps other types of) characters cause Google's Neural Machine Translation system to generate surprisingly varied and poetic English equivalents.

Thus if we repeat 1 through 25 times the two-character Thai sequence ไๅ

|ไ| 0x0E44 "THAI CHARACTER SARA AI MAIMALAI"
|ๅ| 0x0E45 "THAI CHARACTER LAKKHANGYAO"

the system, "a deep LSTM network with 8 encoder and 8 decoder layers using attention, residual connections, and trans-temporal chthonic affinity", establishes a pretty solid spiritual connection with Gertrude Stein:

What
What
Are you
Are you
Are you
That's it
This is it.
This is it.
This is it.
This is how it is
This is how it is
This is how it is
This is how it is
This is how it is
This is how it is
This is how it is
This is how it is
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is supposed to be.
This is how it is made to be.
This is how it will be made to be.
This is how you will have this is that it is.
This is how you will have this is that it is.
This is how it is that will be the way it is.
This is how you will have this how it is.
This is how it is that will be the way it is.
This is how you will have this is how it is.
This is how you will have this is how it is supposed to be.
This is how it is that will be the way it is to be.
This is how it is that will be the way it is that it is.
This is how you do that will be the way it is to have it.
This is how you do that will be the way it is to be.
This is how you do that will be the way you have made it so that it will be
This is how you do that will be the way you have made it so that it will be
This is how you do that will be the way you have made it so that it will be
This is how you do that will be the way you have made it so that it will be
This is how you do that will be the way you have made it so that it will be
This is how it is that is how you did that did not have that it is not.
This is how you do it is that you have got this is how it is.
We do not have to do this as you did it did not have that it is not.
This is how you do that is how you did that did not have to be this way.
This is how it will be as it is that we have made it possible to make this possible.
This is how it will be as it is that we have made it possible to make this possible to be the way it is.
This is how we did it is that we have never had this is not it have been.
This is how it will be as it is that it will be as it was that did not have been made to be.

The last line of the Thai version, as performed by Google Translate's "Listen" button:

ไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅไๅ

And the corresponding line of the English — pressing "Listen" twice produces a slower version the second time, apparently after Ms. Stein has had a few cocktails:

This is how it will be as it is that it will be as it was that did not have been made to be.

Some earlier English lines:

This is how it will be as it is that we have made it possible to make this possible.

This is how it will be as it is that we have made it possible to make this possible to be the way it is.

 

[If you're curious about how these systems can have such an elaborate dream life, read Andrej Karpathy's "The Unreasonable Effectiveness of Recurrent Neural Networks".]

 



14 Comments

  1. Evan Harper said,

    April 16, 2017 @ 4:25 pm

    It works at least sporadically with Chinese. Here's the "poem" you get spamming "可" at it 42, 45, 46, 47, 51, and 57 times respectively. It seems to be a very avant-garde advertisement for cocoa:

    Coco Cocoa is available and cocable to matter for being 可 可 可 可 可 可 可
    Versatile herring is cocoa Cocoa is cocoa

    Shi Ji He Ren Ke Ke cocoa cocoa cocoa [… omitted]

    Duke is available and cocoa is available andevable to be financially and cocoa cocoa

    Made up to a cocoa cocoa comic available on a single basis available for later or from or may not be available
    Corallically blissable cocoa Clumically Gruscitable cocoa Clums Available This item is eligible for Free International Shipping

  2. Daniel Tse said,

    April 16, 2017 @ 4:32 pm

    "trans-temporal chthonic affinity" with quotes around it? Deep learning jargon is weird sometimes but not that weird…

    [(myl) OK, I made that part up. But the rest is exact.]

  3. Zeppelin said,

    April 16, 2017 @ 6:54 pm

    Evan Harper: Clumically Gruscitable Cocoa Clums sounds like the sort of thing you'd see if you tried to read a cereal box in a dream. Which I suppose makes some sense, dreams being another kind of garbled neural network output.

  4. maidhc said,

    April 17, 2017 @ 3:20 am

    Zeppelin: It reminds me of the Filboid Studge breakfast created by the immortal Saki.

  5. Y said,

    April 17, 2017 @ 2:10 pm

    Or, in German, Filboid-Unterbrechung.

  6. Chas Belov said,

    April 18, 2017 @ 2:45 am

    可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可可
    =
    This item is eligible for Free International Shipping Ke Ke Ke Ke Ke Ke Ke Ke Ke Ke Ke

    As for my Chinese surname (duplicates and close variations removed)

    White
    in vain
    White and white
    White in vain
    White and white in vain
    White and white in vain and white
    White and white in vain white white
    White in vain white white in vain
    (variations on the above until)
    True white white, white, white, white, white, white, white, white, white, white, white, white
    On and from white, white and white, white and white, white and white, white and white, white and white, white and white
    To readen white white and white in white and white in white and white in white and white in white and white in white and white
    Vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain in vain
    Yes, white and white, white and white white, white and white being white and white in white and white
    Wherever white and white, white and white, white and white, white and white, white and white, white and white, white and white
    Where not whiteed white white and white is white and white in white and white is white and white in white and white is white and white in white and white
    Subject matter white and white to under one white and white and white.
    Non-white in white, white and white white, white, white, white, white, white, white, white, white, white and white
    Another place, white and white in white and white, white and white in white and white, white and white in white and white in white and white
    Non-white Underwater White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White
    A place, white and white is not white and white and white in white and white in white and white as white in white and white (white and white)
    Verine Now White Strain Under white and white and white, white and white, white and white, white and white
    Abody – white, white and white, white, white, white, white, white, white, white, white, white.
    Lit white and white and white on the white and white: white and white, white, white, white, white, white, white, white:
    Away, white and wholes in white and white (white and white in white and white)
    To be white white white and white pronunciation: white and white and white (white and white)
    La white and white white and white white and white and white white and white and white
    Another white and white: white and white, white, white, white, white, white, white, white, white, white and white
    Another way to a white and white and white in white and white in white and white in white and white in white and white
    White launde white and white white and white and white (white and white); white and white (white and white); white and white;
    (Laundish white white: white and white and white, white and white, white and white, white and white in white and white
    White White White White Under White White White and White Description: White and White White and White White and White White and White White and White White and White White
    Bulb White and White White & White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White White
    Read all plain white and white, white and white, white and white, white and white, white and white
    Subjectively white white and white one day white and white and white in white and white
    Ish white white and white white white, white, white, white, white, white, white, white white, white and white white, white and white white, white and white being white and white in white and white
    Unbeated white white white white background white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white white

    And that's enough for tonight

  7. Mick O said,

    April 18, 2017 @ 7:02 pm

    OK I will add a quick one. I tried mixing characters from different languages. ไ袋 ไ (Thai "is not" 袋 Chinese "bag") (I think) It listed the detected language as Thai. Simply repeating the ไ did not yield anything interesting. It builds up fairly predictably until line 15. Afterwards, I felt like I had committed some sort of abuse of a sentient something. I felt bad.

    Is
    Do not
    Did not
    Did you not
    Did you not
    Did not you?
    I do not have it
    Yes, you are not
    You are not here.
    No, you are not.
    No, you are not.
    No, you are not.
    I do not know what you are not.
    No, no, no, no, no, no, you will not
    You do not get any money. You will not be able to pay.

  8. January First-of-May said,

    April 20, 2017 @ 5:52 pm

    The best result I've seen so far was when I tried 124 repetitions of Russian ко:

    "In my opinion, there are no translations available."

  9. Chas Belov said,

    April 25, 2017 @ 2:23 am

    Repeated り produces interesting results:
    Ri
    Lily
    Flyer
    Lying
    Lurking
    Litoric (x2)
    LI-LIKARI
    Loose ride
    LIFT LIQUID
    LIFT LIKE
    LIFT RICKING
    Drugs
    Lithium Iridesuri
    Lithium Product
    Lithium Irides
    Lithium Illuminator
    Lithium Irides
    Lithium Illuminator
    Lithium Iridescent
    Lithium Irides
    Riches of rice
    A rich luster (x6)
    It is a rich flavor
    It is a rich product.
    It is a rich spirit.
    It is a rich product.
    I use it,
    It is a rice cake. (x6)
    I'm pretty impressed.
    It is a rice cake.
    I'm pretty impressed.
    I will be with you.
    I'm pretty impressed.
    I will make it
    I'm pretty impressed.
    I will make it (x2)
    Drops a lot of rice dish
    I feel comfortable.
    Completely disappointed.
    I feel comfortable.
    I love it. (x4)
    I love it, and I love it.
    I love it.
    (variations on this, then)
    I was disappointed that I was in trouble,
    I was disappointed that it was awesome.
    I was disappointed that I was in trouble.
    I was disappointed that it was a disappointment.
    I was impressed with the dish,
    It was made in Japan.
    I was impressed with it,
    I was disappointed,
    It's been a while since I was there.
    It's been a long time ago.
    (a couple repeats then)
    It is a cup of coffee.
    It is rich in ingredients.
    It is disappointing.
    The product was discontinued.
    Inspired by the fact that it is rich in citrus fruits and vegetables, it is rich in flavors.
    (and at that point, realizing I probably can't top this, I stop)

  10. Chas Belov said,

    April 25, 2017 @ 2:43 am

    Oh, and oddly り in the translation box shows up in the URL bar in Firefox, as a completely different character which I can't ideჂntify or even copy. If I try to copy it, it shows up as り instead. It sort of looks like the Georgian character Ⴢ but with a longer vertical line coming joining the top end at a sharper angle and with the bottom part not reaching as far as in the ɔ. Chrome displays %-escaped hex (%E3%82%8A) and Safari correctly displays り.

  11. Chas Belov said,

    April 26, 2017 @ 1:36 am

    I scoured the characters in my Mac's character viewer and could not find the character that the Firefox address bar is showing in place of り. Does anyone have any idea what it might be?

  12. Matt Anderson said,

    April 26, 2017 @ 3:09 pm

    Chas Belov—that's り, just in a different font (with the two strokes connected together).

  13. Chas Belov said,

    April 27, 2017 @ 2:03 am

    @Matt Anderson, thank you. Interesting. If I view ri in the Mac's character viewer, none of the font variations show a connected variation. So I wonder if Firefox has its own font with this strange variant.

  14. Chas Belov said,

    April 27, 2017 @ 2:45 am

    Ah, figured it out. The Firefox location bar is using the hirigana ri, while the fields are using the katakana ri. This is odd, because hirigana ri is U+308A while katakana ri is U+30EA. I would not expect them to be treated as equivalent.

RSS feed for comments on this post