My brain hurts

« previous post | next post »

The Typalyzer website gives an instant and fun psychological profile of any blog based on the language used. Asked about Language Log, it says we're "scientists". It's true! It's true!

Although we are "intellectually curious and daring", we "might be pshysically hesitant to try new things." I admit it. I'm so pshysically challenged that I can't even pronounce it.

Further, we "tend to be so abstract and theoretical in [our] communication [we] often have a problem communcating [our] visions to other people and need to learn patience and use conrete examples." Yes, also true. Communcating has never been our strong point. Note to self: more conrete.

But ok, spelling aside, this stuff isn't bad at all, until you get to the next part of the analysis. Which is the most misleading picture of a brain since Dan Hodgins showed us where to find the Crockus:

That's why my brain hurts.

[Below the fold, a note for the curious on how the Typalyzer website works.

The Typalyzer seems to use a standard machine classification technique. Presumably the creators have created a little database consisting of example websites with classifications that they think are appropriate, then trained the classifier with these examples. Now the classifier gives instant classifications of sites it's never seen before based, broadly, on how similar the new sites are to the old ones. It's likely that the classifier is primarily using what's called a "bag of words" technique. That is, similarity of one site to another is based on the relative frequencies of words used on the sites. Then based on the classification, the machine chooses which misleading picture to take from a vast database of misleading pictures. All well and good. If you've any faith at all in the psychological categories they use, then this is neat way to do cheap psychology.]



22 Comments

  1. mae said,

    November 20, 2008 @ 5:45 pm

    Speaking of brain un-science: gregmankiw.blogspot.com, blog of economist Greg Mankiw, steered readers to a site that claims to analyze your blog and tell you "what parts of the brain … were dominant during writing" — I think it's a crockus. There's even a little graph superimposed on a brain that is supposed to show this.

    One example: the analysis found that "You Don't Say" from John McIntire was not in English or Swedish, so it couldn't analyze it.

  2. mae said,

    November 20, 2008 @ 5:47 pm

    Sorry I thought I was posting on an older post. You are way ahead of me. But the thing about John McIntire is still funny.

  3. D Jagannathan said,

    November 20, 2008 @ 6:15 pm

    This does look like a(n amusing) load of crockus, but to be fair, Professor Beaver, I don't think they could possibly intend their brain picture to be taken literally. It just looks like a background image for a spiffy graphic indicating which of the eight "types" the blog's authors fall into. The strongest claim the site seems to make is that these eight types correspond to different parts of the brain (which are not necessarily the ones indicated by the sectors of the image).

    On a side note, I wonder why Myers-Briggs typology is so widespread — not just in popular contexts, but in the business world, for example, where one might expect its users to have had some training in organizational psychology and thus be dubious of its usefulness. The original Jungian parameters were unscientific and MB inventories are essentially unfalsifiable. It's a psychometrician's nightmare!

  4. Mark Liberman said,

    November 20, 2008 @ 6:33 pm

    For any of youwho don't already know this, Myers-Briggs Type Indicator is a method for locating individuals in a four-dimensional pseudo-scientific theory of personality. You can read about its psychometric properties here. If you think of it as astrology in psychological disguise, you won't go too far wrong.

    As for the brain image behind the MBTI chart in the cited image, that's clearly an attempt to engage "the seductive allure of neuroscience explanation" — it's not just someone's theory of personality, it's In The Brain!

  5. Theodore said,

    November 20, 2008 @ 6:46 pm

    This is almost as much fun as the Blog Readability Test: http://www.criticsrant.com/bb/reading_level.aspx (It puts LL in the "Genius" category.)

  6. Rubrick said,

    November 20, 2008 @ 7:09 pm

    I'm so pshysically challenged that I can't even pronounce it.

    Pshaw!

  7. fred said,

    November 20, 2008 @ 7:20 pm

    Another (briefly) amusing analysis: literature expert o'Faust.

    Testing it, I pasted the two paragraphs of Mark Liberman's above comment into the text box: the Literature Expert told me that his writing bore a 40% similarity to Friedrich Wilhelm Nietzsche.

    Noticing that a space was missing between two small words in the first sentence ("youwho"), I made the correction and re-submitted. Now Dr. Liberman comment was only 39% like a comment Nietzsche might have made.

  8. blahedo said,

    November 20, 2008 @ 7:26 pm

    What I found interesting is how quickly it returned a result. It must be only looking at the front page, not always the best way to judge the long-term oeuvre of a blogger.

  9. Mark Liberman said,

    November 20, 2008 @ 7:32 pm

    A reference to "Barnum statements" and the rhetoric of "cold reading" is also in order.

  10. Orange said,

    November 20, 2008 @ 7:38 pm

    My crossword blog pegs me as ESTP: Clearly, I engage in outdoor physical activities and have a hard time sitting still. (You gotta be hardcore to be able to solve crosswords while rappelling down a cliff.)

    My feminist/nerd/mom blog pegs me as ISTP and inclined to be a race car driver, cop, or firefighter. (You gotta be hardcore to be able to write and edit manuscripts at 200 mph.)

  11. Ryan Denzer-King said,

    November 20, 2008 @ 7:41 pm

    According to Typealyzer, I'm less practical than LL and more arrogant.

  12. Nicholas Tam said,

    November 20, 2008 @ 8:44 pm

    I have to admit, I didn't even see the little graphic of the brain behind the axes of the graph until this post pointed it out to me. I'd like to see the people behind this disclose some specifics behind their methods… specifically, I want to know the extent to which they hand-classified the corpus they used as their training data a priori, or if the sorting evaluation was rule-bound somehow. My hunch is that there has to be some pre-classification of the corpus going on, or some other form of human-driven reinforcement.

    All the same, I was initially enthusiastic about the tool and still am – not because it tells us a lick about personality or the brain (it doesn't, duh), but because it could tell us about the preconceptions we may have about style/voice. Although it's founded on a fudgy Myers-Briggs typology, a lot of people do intuitively represent things in terms of Jungian archetypes or thinking/feeling polarities, and if a machine can replicate that representation and produce familiar results, it's still of some value.

    I also somehow doubt it's based on lexicon and word counts alone… sooner or later, a project like this has to account for syntax.

    What it needs is a "rate this assessment" button so human feedback can train the system with each new input text sample. Again, it's fudgy, but it would be neat.

  13. Tim Silverman said,

    November 20, 2008 @ 8:54 pm

    One paragraph, typed at random off the top of my head, gave me a 15% resemblance to Edgar Rice Burroughs. A second paragraph increased the resemblance to 17%. Was that it? Was I doomed to a slight, but ever-increasing, resemblance to a classic pulp adventure writer? No. A third paragraph in the same vein gave me an 11% resemblance to Goethe. What a difference a paragraph makes!

    On the other hand, perhaps these slight resemblances reveal an unusual originality of style …

    <struggles to repress choke of disbelief>

  14. Nathan Myers said,

    November 20, 2008 @ 10:17 pm

    I'm not about to defend the Typalyzer, but both Myers-Briggs and astrology have valid uses that are not obvious to people trying to shoehorn them into the world of scientific disciplines. Yes, it's true.

    When you take the little Myers-Briggs quiz and get assigned your four-letter MB score, you are offered a little blurb that describes you more or less correctly, partly through vagueness, partly because we contain multitudes, partly because the quiz does measure something. It will say that you are like 2% or 4% of the population, and that the rest of humanity sort into other slots. The point isn't to make you feel smug about your slot, it's to quantify that most other people aren't like you, but not because there's something wrong with them. They use this in business to encourage people to communicate better with people who are different. Does it matter if the methods aren't rigorous? No, because everybody ends up in some slot, and then finds most other people in the group in other slots.

    Astrology is very similar, but even more random. Does it actually matter if you're an Aquarius or Libra? Everybody gets some sign. Some days you're told to get out of the house, or spend money, or meet new people, and other days to do stay in, or scrimp, or visit old friends. Everybody gets told something different every day, but not everybody gets told to do the same thing at the same time. It's a tool to keep people out of ruts. Lots of people really need this service, and need to believe a load of mumbo-jumbo about it before they can use it.

    The function of the I Ching is left as an exercise for the reader.

  15. David Beaver said,

    November 20, 2008 @ 11:03 pm

    @D Jagannathan:

    Dhananjay, maybe you're right that they don't *intend* their brain picture to be taken literally. But here's my question. Whose brain is less active,

    (a) someone who writes "Brain Activity" in big letters above a picture of a brain with a big shaded polygon on it, a list of stereotypically left brain categories on the left of the brain picture, and a list of stereotypically right brain categories are on the right, and who intends the brain picture to be taken literally, or
    (b) someone who writes "Brain Activity" in big letters above a picture of a brain with a big shaded polygon on it, a list of stereotypically left brain categories on the left of the brain picture, and a list of stereotypically right brain categories are on the right, and who doesn't intend the brain picture to be taken literally?

  16. Matt said,

    November 21, 2008 @ 12:09 am

    Theodore said,

    November 20, 2008 @ 6:46 pm

    This is almost as much fun as the Blog Readability Test: http://www.criticsrant.com/bb/reading_level.aspx (It puts LL in the "Genius" category.)

    That is quite an honour for Language Log. Puts it at the same readability level as http://www.sesamestreet.org/home

    Maybe today's pre-schoolers are being fed a lot of fish oil?

  17. D Jagannathan said,

    November 21, 2008 @ 1:08 am

    @dib: Yes, I noted the fatuous left/right distinction, and I am quite inclined to assign as little brain activity to the makers of the site as possible!

  18. Christopher Stone said,

    November 21, 2008 @ 3:45 am

    @Matt,

    I have to wonder if the 'genius' classification for Sesame Street is mostly because of the fact that most of the 'text' is contained in images and thus couldn't really be read by whatever bot crawls over the page. The only actual text on the Sesame Street website is on the bottom, which provides links to other parts of the site, mostly legal-y stuff: "©2008 Sesame Workshop | Terms of Use | Privacy Policy: Your Privacy Rights | Help | Site Map ".

  19. John said,

    November 21, 2008 @ 8:08 am

    The Blog Readability Test could be taken the other way: to say one needs to be a genius to read it would imply (to some) that it is lacking in "readability." Not to criticize Language Log though, its "score" is likely due to its technical nature–stuff us amateurs just don't quite get (Blog Readability Test included, apparently).

  20. Chris said,

    November 21, 2008 @ 10:18 am

    Nathan Myers: The main difference between the two, AFAIK, is that if you read a random horoscope it describes you about as well as your own, but if you read a random MBTI type description you think "I know people like that, but I'm not one of them". IIRC people asked "which of these describes you best" pick their own MBTI description (with the name of the "type" removed) out of a list with a success rate better than chance, which you certainly can't do with astrology.

    MBTI also has test-retest correlation proving that it is indeed measuring something.

    So just because media discussions of MBTI contain large balls of feathers does not necessarily mean that it's all pseudoscience.

    (The linked PDF, I notice immediately, is testing a grossly overambitious hypothesis about using MBTI to predict career paths. I wouldn't expect that to work outside of maybe a few highly specialized jobs like computer programming, and even then the effect would probably be weak. Then, within a few pages, they distort the system further by imposing discontinuities on the four-dimensional manifold measured by MBTI to chop it into neatly discrete categories – Keirsey, for one, explicitly warns about the limitations of this approach, especially for people who are near a boundary. Treating the four dimensions as the continua they are and running regressions might have given better results. Overall, it proves at most that there are pseudoscientific claims made *about* MBTI, not that there is no there there.)

  21. Mark Liberman said,

    November 21, 2008 @ 11:45 am

    Chris: MBTI also has test-retest correlation proving that it is indeed measuring something.

    A review of MBTI psychometrics is here. The conclusion:

    …there are a number of psychometric limitations pertaining to the reliability and validity of the MBTI, which raise concerns about its use by practitioners. In view of these serious limitations, routine use of the MBTI is not recommended, and psychologists should be cautious as to its likely misuse in various organisational and occupational settings.

    Chris: ..they distort the system further by imposing discontinuities on the four-dimensional manifold measured by MBTI to chop it into neatly discrete categories…

    But this seems to be exactly what most practitioners who use the test (e.g. "in organisational and occupational settings") do. I've been given various forms of MBTI three times — at corporate retreats and the like — and was always given a four-letter code (a different one each time), not a vector of numbers. And this dichotomous classification is often justified by proponents of MBTI on theoretical (e.g. Jungian) grounds, as (at least implicitly) in the case of the Typalyzer.

  22. Nathan Myers said,

    November 21, 2008 @ 6:57 pm

    Mark: I have taken various MBTI quizzes, and I always come up as ENTP or INTP. I have seen assertions that the more "balanced" you are, the less repeatable your results will be. On that risible diagram in the original posting, that would place you near the center, which they interpret as a desirable condition, indicating versatility. Most of the rest of us languish in the hinterlands, doomed to specialize. If you don't feel so well balanced, it may be that, like my wife, you just get caught up in analyzing the wording of the questions, washing out their meaning and answering them more or less randomly, washing out the signal with noise.

    For those of us who find the results very repeatable, the question is not whether it measures anything, but what the measurements can be construed to mean. For typical business uses, it doesn't matter. However, my being a "P" and my major professor coming out a "J" (INTJ) reflected that I was much more comfortable with uncertainty, and less comfortable about likely being wrong than he. Recognizing this made it much easier to communicate with him. One may recognize this without MBTI, but MBTI was how I came to it in that instance.

RSS feed for comments on this post