the big picture: automatic metaphor identification

The recently popularized IARPA Metaphor Program piqued my curiosity, so I've been reviewing a variety of articles on contemporary approaches to automatic metaphor identification. I've read three articles so far and one thing is somewhat dissapointing: they all severely restrict the notion of metaphor to mean local metaphors within single sentences.

They all pay considerable lip service to Lakoff & Johnson's seminal 1980 work Metaphors We Live By, taking as gospel the notion that metaphor is defined as a mapping from one conceptual domain to another. But their examples are all of a limited type. Here are three representative examples from the papers I've been reading:

• Achilles was a lion. (Babarczy et al.)
• The sky is sad. (Tang et al.)
• I attacked his arguments (Baumer)

What struck me is the methods used to identify metaphor are remarkably lexalist. The dominant strategy is Selectional Preferences whereby a list of source and target conceptual domains is created. Then from each, a list of words typically associated with that domain is culled from corpora or intuition or dictionaries. Then, each word is given a set of selectional preferences which constrain what kinds of subjects or predicates it typically occurs with.

Here is my Ling 101 version of this methodology: If I understand correctly (and I may not), for Tang et al.'s example "The sky is sad", we would have a concept like THE ENVIRONMENT IS HUMAN. We would have a list of words typically associated with the environment (e.g., "sky") and a list of words typically associated with being human (for example "sad"). A computer could then recognize the following:

1. The subject (the sky) is associated with the environment.
2. The predicate (sad) is associated with humans.
3. This subject (the sky) is not typical for this predicate (sad).
4. This sentence is incoherent on first analysis.
5. The concept THE ENVIRONMENT IS HUMAN links these non-typical phrases coherently.
6. This sentence is only coherent using conceptual mapping, therefore it is probably metaphorical.

This is a gross oversimplification, but I think it gets the big picture about right.

At first blush, I'm impressed with the simplicity and elegance of this solution. However, it seems to me that much metaphorical language is not local like this (local here = within a single sentence). For example, imagine a situation in a biology class where two students, Alger and Miriam, were originally going to be partners for a lab assignment. Then they got into an argument. A third student, Annette, asks Miriam:

• Annette: Are you still going to be lab partners with Alger?
• Miriam: No. That ship has sailed.

In this scenario, the sentence "That ship has sailed" is entirely coherent from a selectional preferences perspective (i.e., ships really do sail). Yet it is clearly being used metaphorically (there is literally no ship). Here, the metaphor is only detectable if we link two sentences together via co-reference. The phrase "the ship" does not co-refer to a real ship in the discourse. Rather, it refers to the possible event of be-lab-partners-with-Alger. Unless we can link phrases between sentences and between types (i.e., allowing an NP to co-refer to an event), then we are not going to get a computer to recognize these types of metaphors (which I suspect are quite common).



Xuri Tang, Weiguang Qu, Xiaohe Chen, & Shiwen Yu (2010). Automatic Metaphor Recognition Based on Semantic Relation Patterns International Conference on Asian Language Processing


Other citations:
The Automatic Identification of Conceptual Metaphors in Hungarian Texts: A
Corpus-Based Analysis. Anna Babarczy, Ildikó Bencze M.1, István Fekete1, Eszter Simon1

Computational Metaphor Identification to Foster Critical Thinking and Creativity. ERIC BAUMER (dissertation). 2009.

the largest whorfian study EVER! (and why it matters)

Let me take the ball Mark Liberman threw on Monday and run with it a bit. Liberman posted a thorough discussion of Fausey and Broditsky's neo-Whorfian English and Spanish speakers remember causal agents differently. Specifically, he invited readers to carefully examine the methodology of the experiments themselves, and not just focus on the conclusions. It turns out that a few years ago another set of neo-Whorfians, Jürgen Bohnemeyer and company, published a paper that addressed similar methodological concerns:

Ways to go: Methodological considerations in Whorfian studies on motion events. (With S. Eisenbeiss and B. Narasimhan) Colchester: University of Essex, Department of Language and Linguistics (Essex Research Reports in Linguistics 50: 1-19). 2006.

This paper addressed experiments involving motion events like rolling and falling whereas Fausey and Broditsky's work addressed agentivity like breaking and popping, but there's enough overlap to warrant some comparison, particularly since the Bohnemeyer et al. paper specifically addresses methodology wrt Whorfian experiments.

But before I get into the details, let me state clearly why I think this is important. In other posts, I have dismissed popular lingo-topics like language evolution as outside the mainstream of linguistics because they don't bear directly on what I consider to be the center of the linguistics universe: How the brain does language. But linguistic relativity (aka, The Whorfian hypothesis) is one of the great questions of linguistics and cognitive science precisely because it bears directly on the question of how the brain does language. And we're only just now developing the proper tools and methodologies to study the question with scientific rigor. It may turn out that language does not affect other cognitive processes or the effect is minor. I don't care. I just want to know one way or the other. And it's work like Bohnemeyer's and Broditsky's that will lead us to knowing, eventually.

Now the fun stuff.

Through the Language Glass (Part 2) [reposted]

This is part 2 of my review of Guy Deutscher's new book Through the Language Glass: Why the World Looks Different in Other Languages. This covers The Language Lens (129-249). Part 1 is here. This review will cover the scientific evidence that Deutscher reviews suggesting that language affects thought, and will end with a shocking proposal.

To sum up my review of part one: meh. Okay, we've established that culture can influence language. This is a lot less controversial than Deutscher makes it seem and he spent a large amount of text defending that position. Okay, whatever, time to move on. In part 2 he again begins with historical review explaining why he thinks Whorf was a con man, but also why he thinks the core insights of early linguist relativity deserve closer, honest investigation. He complains that based his Hopi claims on just one lonely informant (p142). We'll see later that Deutscher himself falls for the same trap. He replaces Whorf with the Boas-Jakobson principle that languages differ in what they must convey, not what they may convey” (151). I respect Deutscher for making this a central theme in his book because I think he's right. To parrot his own recitation of Humbolt: any thought can be expressed in any language. It is what our native language forces us to foreground that makes linguistic relativity an interesting topic.

Deutscher spends most of the second part of the book reviewing three areas of language that have provided evidence that language affects thought: spatial coordinates, grammatical gender, and color terms (familiar from part 1). The general point I want to make about his evidence is that it is far weaker than he maintains. But is is interesting. A brief set of reactions:

Spatial Coordinates -- everything is embodied
Most of his argumentation about the affect of spacial coordinate terms on thought stems from Levinson's evidence from speakers of the Australian language Guugu Yimithirr which is famous for giving us the word “kangaroo.” Speakers of GY do not generally use ego-centric terms like "right" and "left" but rather use cardinal direction terms like "east" and "west." As a result, Deutscher claims, they remember information about situations differently than speakers of English. They have, so the argument goes, a perfect pitch for direction and they are always attuned to where north is. Deutscher's claim is that only the linguistic repetition of such terms can possibly account for this. Hence, their language affects what they pay attention to and what they remember, hence language affects thought.

I've never found this line of research all that convincing regarding linguistic relativity and Deutscher does not really add much to the debate. Like Deutscher's complaint above regarding Whorf's one lonely Hopi speaker, it turns out there are not many native speakers of Guugu Yimithirr left and haven't been for a while. These experiments on directional language involve very few speakers, and most of them have both cardinal direction and ego-centric direction in their dialect. If we're going to complain about Whorf's restricted subject pool, we must complain about Levinson's too.

But more to the point, I believe all direction terms are ultimately ego-centric insofar as they are embodied. The terms "north" and "south" are not magically universal. They are based on a human being's body and orientation (i.e., ego-centric). Don't believe me, ask yourself, what does "north" mean in space? What does "north" mean to an amoeba? Mostly what Deutscher does in his discussions of direction terms is reiterate the point he belabored in Part 1: culture affects language. Yeah, we got that already.

The rise of similarity judgments
That is until he discusses the table experiments. These experiments show subjects tables with objects on them and ask them to arrange them in accordance with a target. Basically, they ask for similarity judgement. How can you make this table arrangement similar to the previous table. This methodological paradigm has become prominent in psycholinguistics and cognitive linguistics, especially studies testing linguistic relativity. In fact, all of the studies Deutscher discusses are similarity judgment studies of one sort or another. The point is that I show you one target thing, then two test things and ask, which test is MORE SIMILAR to the target than the other? Ultimately Deutscher himself problematizes spatial coordinate terms so much, they fall flat and remain unconvincing as a base of evidence for linguistic relativity.

Grammatical Gender
Most languages have terms for classifying things. Some languages have more elaborate classifier systems than others. In German, the term for the fork is die Gabel, marked by feminine die. Ultimately, most languages with elaborate classifiers have systems that can be described as incoherent in so far as most things given one classification have no inherent properties that signify that classification (there is nothing inherently feminine about a fork). However, Deutscher provides evidence that speakers of languages with grammatical gender will evoke properties of things in keeping with their gender classifier, suggesting that the classifier is causing them to imagine a fork would speak with a female voice, for example. But these experiments mainly test vague associations of imagination, not linguistic causality, as Deutscher admits.

Color Terms
It is not until chapter 9 Russian Blues that Deutscher really delivers the goods. It is this chapter which provides the most interesting evidence for the effect of language on thought. Pity it is only about 15 pages of the book. The whole book should have been more like this. The facts he discusses involve the basic point that the brain sees what it wants to see. It turns out our perception of color has little to do with any objective feature of the thing we're looking at (he explains this fact brilliantly in the Appendix which I highly recommend, and frankly, should have been the first chapter, not relegated to the attic of an appendix). The point is that our brains change the input. As our eyes take in objective photons, our brain photoshops the input (a great analogy from Deutscher which really brings the point home).

The experimental results Deutscher discusses involve more similarity judgements, albeit with a twist. Instead of relying solely on the similarity judgments, researchers studied the more objective reaction time. They showed people different color patches and asked them to judge the sameness. Despite the various and clever variations on this theme, they all relied on subjective judgements of similarity. And this is where they fail to extricate themselves from the problem of strategizing.

Unfortunately they all share the critical flaw that making a similarity judgment is a logical reason act and may be mitigated by strategizing. Deutscher discusses this fact, but doesn't realize that none of the fixes work. A similarity judgment is always a logical process susceptible to the effects of strategizing. This will be a major issue in my Shocking Proposal at the end. You see, regardless of how clever the test, as long as you are basically asking a subject to make a similarity judgment, you are asking them to reason about the task. So your results will be tinged by the strategizing of human subjects as they logically try to game the system. This is well known in psycholinguistics and difficult to avoid. So how do you objectively test what colors a person considers blue?

A Shocking Proposal
The paradigm already exists. How can you objectively prove that English speakers really do consider aspirated /kh/ and unaspirated /k/ both the same phoneme? You condition them to fear aspirated /kh/ by shocking them every time they hear it (measuring their galvanic skin response). Once they are conditioned, you then play them unaspirated /k/ (with no shock) and check to see if you get the same GSR spike (in anticipation).

Okay, now apply this to color terms. Condition subjects to fear center of the category blue, then show them gradations. What causes the GSR spike? That's what they consider blue. now do that with speakers of 40 different languages.

If the hippies on the human subjects review board let you do it, there's your dissertation.


Guy Deutscher (2010). Through the Language Glass: Why the World Looks Different in Other Languages. Metropolitan Books

Super 8: more homework than movie

A rare non-linguistics post: Saw Super 8. Critics are spot on: it's a movie that shows how to construct a coherent narrative like they used to make. It is simple, straight forward. It makes contemporary movies like Transformers, Thor, and Pirates seem amateurish and sad. It's like a tutorial in narrative film-making.

Unfortunately, it's like a tutorial in narrative film-making ... more like homework than enjoyment. I felt like I should be taking notes. Like there'd be a quiz at the end.

One reason this is so is simple modern history. 30 years ago, good narrative films were strictly the domain of Hollywood. TV was the domain of kitsch and schmaltz. But since the 1990s, TV (largely due to HBO, and more recently Showtime, TNT, AMC...) has taken over the mantel of prime story teller. We know from good narratives. We know from good structure. We know from good plot. We know from good formula (we loved Law and Order for 20 years, after all).

So now, in 2011, the ET/Goonies routine rings a bit hollow. Like movie-plot-by-the-book; paint-by-numbers film making. Sure, it's more coherent than Thor (a truly gawdawful film), but Breaking Bad and Mad Men are still way better, and they don't cost me $11 and a special trip. Don't even get me started on how much better the revived Doctor Who is than this Hollywood trash.

simple english wikipedia

I didn't know this existed until it was referenced briefly in an unrelated article. Simple English Wikipedia:

This is the front page of the Simple English Wikipedia. Wikipedias are places where people work together to write encyclopedias in different languages. We use Simple English words and grammar here. The Simple English Wikipedia is for everyone! That includes children and adults who are learning English.

Great idea! I perused some of the articles relating to linguistics, language, and grammar. Not bad. Under Grammar, they write:

When we speak, we use the native-person's grammar, or as near as we can. When we write, we try to write with correct usage grammar. So, speaking and writing a language each have their own style.

While one may want to quibble a little with this (I don't quite understand the "as near as we can" hedge), it's actually a nice way to explain the difference between speaking and writing to, say, a 10 year old.

Shakespeare and the brain? maybe not...

Jezebel, The Guardian, and other sources have been promoting recent neuroscience involving reading Shakespeare. This research has popped up in the blogosphere before and was widely misunderstood. I fear this time is no better, so I offer this re-posting of my original response from late 2007:

Even though he blogs at a mere undergrad level I basically respect Andrew Sullivan as a blogger. He blogs about a diverse set of topics and has thoughtful and intelligent (even if controversial) comments and analysis. And he’s prolific, to say the least (surely the advantage of being a professional blogger, rather than stealing the spare moment at work while your test suite runs its course). That said, he can sometimes really come across as a snobbish little twit. Like yesterday when he linked to an article about Shakespearean language which talks about a psycholinguistics study initiated by an English professor, Philip Davis; as is so often the case, the professor has wildly exaggerated the meaning of the study. Please see Language Log’s post Distracted By The Brain for related discussion. Here’s crucial quote from that post:

The neuroscience information had a particularly striking effect on non-experts’ judgments of bad explanations, masking otherwise salient problems in these explanations.

My claim: the neuroscience study discussed in the Davis article distracts the reader from Davis’s essentially absurd interpretations, and Andrew Sullivan takes the bait, hook, line and sinker (and looks like a twit in the end).

The article does not go into the crucial details of the study, but it says that it involves EEG (electroencephalogram) and MEG (magnetoencephalograhy) and fMRI (Functional Magnetic Resonance Imaging) noting that only the EEG portion has been completed. A pretty impressive array of tools for a single psycholinguistics study, I must say. Most published articles in the field would involve one or maybe two of these, but all three for a single study? Wow, impressive.

It’s not clear to me if this was a well designed study or not (my hunch is, no, it is a poorly designed study, but without the crucial details, I really don’t know). However, it is undeniable that professor Davis has gone off the deep end of interpretation. The study does not even involve Shakespearean English!!! It involves Modern English! Then Davis makes the following claims (false, all of them, regardless of the study):

["word class conversion"] is an economically compressed form of speech, as from an age when the language was at its most dynamically fluid and formatively mobile; an age in which a word could move quickly from one sense to another… (underlines added)

This is the classic English professor bullshit. I don’t even know what “economically compressed” means (Davis gives no definition); it has no meaning to linguistics that I know of. The quote also suggests Shakespeare’s English had some sort of magical linguistic qualities that today’s English does not possess. FALSE! Modern English allows tremendous productivity of constructions, neologisms, and ambiguity. A nice introduction to ambiguity can be found here: Ambiguous Words by George A. Miller.

Davis ends with a flourish of artistic bullshit hypothesizing:

For my guess, more broadly, remains this: that Shakespeare's syntax, its shifts and movements, can lock into the existing pathways of the brain and actually move and change them—away from old and aging mental habits and easy long-established sequences.

Neuroplasticity is only just now being studied in depth and it’s far from well understood, but the study in question says NOTHING about plasticity!!! There’s also no reason to believe that Shakespeare’s language does anything that other smart, well crafted language does not do. And we’re a generation at least away from having the tools to study any of this.

I’m accustomed to simply letting these all too common chunks of silliness go without comment, but then Andrew had to slip in his unfortunate bit of snooty arrogance. After pasting a chunk of the obvious linguistics bullshit on his site (then follow-up comments), he has to finish with "I knew all that already". Exactly what did you know, Andrew? Since all of the major claims Davis makes are obvious bullshit, what exactly do you claim to have had prior knowledge of? What did Andrew know, and when did he know it?

Really, Andrew, did you never take so much as a single linguistics course during all your years at Harvard and Oxford? The University at Maryland has excellent psycholinguists as does Georgetown. Please, consider sitting in on a course, won’t you?

Google linguist interview

Purpose: This post reviews my experience interviewing for a Linguist position at Google in Santa Monica, CA on February 29, 2008. I've long meant to post this but only now got around to it. There are lots of Google interview stories on the web. It appears to be its own genre. This is my contribution to the genre.

I originally wrote it as an email to a friend who wanted to know how my big day at Google went. It’s rather long, but then again, you don’t have to read it, you clearly have better things to do…

I found a job posting on the Google jobs board for a full time Linguist. I applied and was given a phone interview with a recruiter around late January, 2008:

Thank you for your interest in Google. I'd like to set up a time for us to discuss Google Linguist opportunities and your qualifications. Please let me know a day/time when you would be available to speak with me as well as the best phone number for me to contact you. I'll email you back to confirm.

I hope to hear from you soon!

Cheers,
JF
Google Staffing

During that phone interview the recruiter shared a Google doc which I was instructed to complete in about 45 minutes…