Friday, July 4, 2008

Online Psycholinguistics Experiments

Experimental psycholinguists requires experimental subjects like any other empirical cognitive science. Unfortunately, researches are often constrained by limited resources. Typically, psycholinguists use college students bribed with money or extra credit as subjects. It's not unheard of for a published psycholinguistics study to have involved as few as 12 subjects. This has been a necessary evil because there has never been a good way to collect large numbers of subjects together and provide them with a coherent experimental design.

Lately, however, researchers are turning to the web as a place to conduct experiments with large groups of subjects. Yes, there are issues regarding control (e.g., if you need native speakers of English, how can you ensure that a subject really is a native speaker?), but these issues come up in all types of experimental paradigms. I believe that good standards and practices to ensure quality online psycholinguistic experiments will emerge over time. So, I'm all for moving ahead.

With that in mind, here are a set of sites offering online psycholinguistic experiments:

  • University of Edinburgh School of Informatics, Blizzard Challenge 2008 (this is actually an evaluation of speech synthesis, but good enough for starters).


Jason M. Adams said...

Even if there is noise in the data (like a few non-native English speakers slip through), shouldn't that be less important in the presence of many data points? This seems like a really, really good thing.

Vernon Lynn Stephens, MSSW said...

I am 'out of the loop' of the linguist-circle: my interests are truly in language/communication science as a hyle-bio-psycho-social set (of disciplines.) My training is utterly and all in social sciences OTHER than linguistics: I was a bachelor's sociologist, Masters-level social worker in psychiatrics, then turned interested hugely in my own discourse as a mental-patient (DSM IV-TR 296.44-- 'manic').

What most of psycholinguistics under the Chomskyan Era has seemed to me from this outsider's peep is a bunch of arcana about highly contrived sentences + mathematico-logic stuck atop. While this claims to be social science, it does not appear to do much empirical work, and shuns the I can determine.

I did recently obtain a work-- I read much in language and semiotic-- with the title THE PSYCHOLOGY OF LANGUAGE: FROM DATA TO THEORY by Trevor A. Harley, and this is refreshingly the other-way: up to the snuff of research in real systems of communication, with real people-- not so much armchair stuff. I use this book frequently, as it seems to instruct me about my own penchant for aberrant-- and sometimes innovative speech patterns.

Now on disability for these 'head-problems,' I am not any longer persuasive to IRBs for doing research on these lively topics with real subjects (focal interests: mental health 'consumers' like me.) The use of the Internet for this purpose has progressively and for some time seemed to be an option for this though. To a limited extent, I have done some corpus-linguistics with mental health/illness documents, using concordancing freeware for this purpose. This has convinced me that content-analysis + concordancing has immense value for examining the psycholingo of mental patients, and in an uncontrolled uncontrived unbiasing way can detect quite subtle speech behavior. In the case of my cohort (mental health users) -- whereas with the utmost extremity must one go through the haze-maze assocated with any form of data gathering with these (sensitive! protected!) huuman subjects by such guides as THE BELMONT RECOMMENDATIONS, perhaps even better, almost-best information from these very subjects is in real-time, real-real-ways VOLUNTEERED by numbers of 'consumers' on the Web.

It would be my fondest hope that this form of work per Internet be maxxed. What I would like to see in addition to what I read herein, and into my own budget-limited access to cheapo-- but great-- tools-- would be the full systems application of lingo analysis equippage to A-V on the Web, like Communications Anal meters applied to U-Tube... with my bunch, those patient-patients stuck in psych places with psych straightjackets for everything... for the beginnings of a true cognitive neurolinguistic discipline.

Concordances and Content-Anal is not EVERYTHING in communications, of course. But perhaps as anyone doing Bible-language and mental-health-consumer-communique analysis can vouche-- these are antepenults which can provide the NECESSARIES prior to the SUFFICIENCIES of understanding 'rap.'

I am by comparison to you-guys on the margins, too-old, too-broke, and too-unlearned to do more than cheer-you-on. But: keep up this trend: Evidence-Based-Practice (EBP-- the application of real science method) requires for starters that you have EVIDENCE, which does start with facts derived from the realest-world-one-can-get. Otherwise, linguistics-- or any other science-- 'ain't really what it's trumped to be!'

--Vernon Lynn Stephens, MSSW
D.S.M. IV-TR # 296.44
Louisville, Kentucky USA

Chris said...

I can attest to the fact that Trevor Harley book is standard in psycholinguistics because it was, in fact, the textbook I was assigned in my graduate level psycholinguistics course.

Coming from a functionalist school, I find it easy to criticize Chomskyean linguistics too. But those days are fading. Chomsky simply isn't a force in 21st century linguistics. Empirical, data driven approaches are the norm in all but a handfull of circles now.

Christian said...

Vernon, I studied linguistics at university and I cannot but agree with your description of the subject. Having had a fascination with language all of my life, I was quite disappointed about the narrow scope. Linguistics does after all more or less claim to be THE study of language. I think part of this narrow-mindedness is connected with the way research is conducted - having a more empirical approach would force scholars to take a much broader view of what language is and how to explain it.

Putting the Linguistics into Kaggle Competitions

In the spirit of Dr. Emily Bender’s NAACL blog post Putting the Linguistics in Computational Linguistics , I want to apply some of her thou...