Blue Meat and Clever Research

Cognitive Daily reviews some really clever research on synesthesia, the phenomenon of associating words with colors, as well as other multimodal associations (not to be confused with its poor cousin sound symbolism). For example, there are people who will experience seeing the color blue when they hear the word meat (the actual word-color associations are not fixed or predictable, as far as I know). There is neuroscience research suggesting that people who experience this have some sort of overlap in processing areas for the word-color pairs (read an excellent roundup of the research here at NeuroLogica Blog). But this is a difficult area to study because there are so few true synesthetes and their experiences are inconsistent.

Bargary et al. 2009 wanted to discover when the color association was triggered in the time course of lexical recognition. Exactly how were they going to track that?
Clever people that they are, the fell back on an old standard in psycholinguistics, the classic McGurk effect which shows how people integrate both auditory cues and visual cues (i.e., lips) to determine what word they're hearing. More to the point, the McGurk effect shows how people will mis-recognize a word when the word they hear is slightly different from the word they see lips pronounce. For example, if subjects hear an audio file of the word been and see a soundless video clip of lips pronouncing beep, then they report having heard the word beam (ignore the orthographic difference). Cognitive Daily has a good YouTube demo here.

Bargary et al. used lexical stimuli that had reliable McGurk effects, but they used several conditions to test synesthesetic associations. In (1), participants got the full McGurk effect and were asked to choose which color they saw. In (2), participants got the visual cue, but white noise on audio, then were asked which color they saw. In (3), the lips were pixellated and only the audio cue was present.

The researchers had to do some clever teasing apart of color terms too, but in the end they concluded that synesthesetic associations occurred late in lexical processing, after both auditory and visual cues were integrated. I'd have to take a much closer look at the research to see if I felt their conclusion was valid, and this head cold I'm nursing rather precludes such close reading of empirical research right now; nonetheless, I'm impressed with the cleverness of the methodology.