{"id":45696,"date":"2020-01-07T12:40:51","date_gmt":"2020-01-07T17:40:51","guid":{"rendered":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=45696"},"modified":"2025-07-15T18:25:02","modified_gmt":"2025-07-15T23:25:02","slug":"alignment-charts","status":"publish","type":"post","link":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=45696","title":{"rendered":"Alignment charts and other low-dimensional visualizations"},"content":{"rendered":"

The current xkcd<\/a>:<\/p>\n

<\/a><\/p>\n

<\/p>\n

There's a long history of similar diagrams in language-related areas.<\/p>\n

A century and a half ago, C.S. Peirce was fond of triangles, like this one<\/a>:<\/p>\n

<\/a><\/p>\n

And this one<\/a>:<\/p>\n

<\/a><\/p>\n

About a century ago, we get the \"semantic triangle\", from Charles Ogden and Ivor A. Richards, The Meaning of Meaning<\/a><\/em>, 1923:<\/p>\n

<\/a><\/p>\n

But those diagrams are not really in direct line to modern \"alignment charts\", since they represent a graph of pair-wise relations among concepts or processes, rather than points in a low-dimensional vector space whose dimensions have intuitive meaning.<\/p>\n

For a more appropriate proximate history, we can start with the idea of \"semantic differential<\/a>\" spaces, based on the ideas in Charles Osgood<\/a>, \"The Nature and Measurement of Meaning<\/a>\", 1952. Here's a three-dimensional example:<\/p>\n

<\/a><\/p>\n

And another one:<\/p>\n

<\/a><\/p>\n

A similar set of ideas lie behind Valence-Arousal theories of emotion:<\/p>\n

<\/p>\n

Joe Kruskal<\/a>'s non-metric Multi-Dimensional Scaling<\/a> (and similar dimensionality-reduction techniques) were in large part an attempt to extend Osgood's ideas about using factor analysis to learn semantic-differential dimensions from observational data of various kinds.<\/p>\n

Doug Biber<\/a> used factor analysis<\/a> to induce a set of dimensions for register\/style\/genre:<\/p>\n

<\/p>\n

\"Latent Semantic Analysis<\/a>\", originally developed by Tom Landauer, Sue Dumais and others in the late 1980s, has a similar historical relationship to Osgood's Semantic Differential ideas, and similarly learns to place words in a multi-dimensional space based on Singular Value Decomposition of a term-by-document matrix. For some pictures, see Landauer et al., \"From paragraph to graph: Latent semantic analysis for information visualization<\/a>\", PNAS 2004. A relevant quote:<\/p>\n

[W]e conjecture that verbal meaning is irreducibly high dimensional. Thus, the value of automatic reductions to two or three best dimensions may be inherently limited; although they may be valuable for some purposes, they must often provide only an impoverished and possibly misleading impression of the relations in a dataset. Different researchers and scholars are often interested in different aspects of articles, only some of which may have been indexed, key-worded, the object of citation, or shown in a particular view. The alternative we have explored here is a combination of measuring similarity of the entire content of articles with high dimensional visualizations that support search for projections that are of special interest to the user. […]<\/span><\/p>\n

Despite decades of highly creative and sophisticated innovation, and a plethora of claims for obvious superiority of the visualization approach, we do not see visual maps of verbal information in popular and effective use. It is, of course, possible that visualizing verbal information is in large part just an appealing bad idea. A more optimistic view is that the application of more user testing to understand what does and doesn't help people do what, will steer innovations in more effective directions.<\/span><\/p>\n

Of course, \"alignment charts\" (and \"vowel charts\" and \"soil charts\" and so on) represent theories about the low-dimensional representations that explain variation in some domain of interest, rather than methods for visualizing the relations among very large sets of arbitrary entities like documents or words.<\/p>\n

More recent \"word embedding\" methods<\/a> are variations on the same theme as Latent Semantic Analysis — sharing its commitment to residual dimensionality on the order of several hundred rather than two or three. Like LSA, they also descend from Osgood and other mid-20th-century psychologists, a relationship that deserves to be better known than it is.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The current xkcd:<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9],"tags":[],"class_list":["post-45696","post","type-post","status-publish","format-standard","hentry","category-linguistics-in-the-funny-papers"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/45696","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=45696"}],"version-history":[{"count":26,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/45696\/revisions"}],"predecessor-version":[{"id":70043,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/45696\/revisions\/70043"}],"wp:attachment":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=45696"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=45696"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=45696"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}