Sunday, February 6, 2011

why we need good tools...

Because we're not all interested in being R experts. By far, the single most frustrating part of my own graduate linguistics experience was the fact that in order to study the kinds of linguistic phenomena I wanted to, I had to spend most of my time learning tools that I didn't actually care about, like Tgrep2, Perl, Python*, R, etc. As a linguist, I don't really give a damn about any of those things. They were all obstacles in my way. The more time I spent learning tools, the less interested in linguistics I became. I respect the hell out of engineers who build great tools that are valuable to linguists, but if those tools are not user friendly, I might as well scream into the darkness.

Which is why I am impressed with The Stanford Visualization Group's recent Visualization Tool for Cleaning Up Data:

Another thing I often hear is that a large fraction of the time spent by analysts -- some say the majority of time -- involves data preparation and cleaning: transforming formats, rearranging nesting structures, removing outliers, and so on. (If you think this is easy, you've never had a stack of ad hoc Excel spreadsheets to load into a stat package or database!).

Yes, more help please.

HT LingFan1

*Mad props to the NLTK!

6 comments:

Brandon C. Loudermilk said...

I know what you mean. I wouldn't be able to do the linguistic research I do, had I not been a software engineer in my previous life. My last experiment required writing a handful of Java programs for analyzing stimuli, a presentation script for conducting the experiment, a handful of Matlab programs for filtering data, artifact rejection, etc., and finally an SPSS syntax script for statistical analysis.

Chris said...

One day, will all of that be as easy as posting a photo on Flikr or finding frequencies on COCA? A boy can dream...

Brandon C. Loudermilk said...

As a former sw engineer specializing in human computer interaction and GUI applications, my primary beef as a psycholinguist is with the way software in our discipline is created... i.e. by linguists or psychologists that happen to know a bit of programming. Undoubtedly, these applications "do" what they are supposed to do, but interacting with them (as an end user) is difficult at best. For example, the interface to Praat, is pretty much the least user-friendly application I have encountered. Software should be intuitive (and naturally grow with the experience of the end user)... that's why/how a two-year old can operate an iphone.

Chris said...

Totally agree. And Apple is probably the world's greatest champion of user friendly software ... if only they would create a Praat competitor, or an E-Prime competitor.

bulbul said...

As a linguist, I don't really give a damn about any of those things.
Sounds really funny from a guy who insists linguists should study math :)
Hey, I wanna do linguistics, I don't give a shit about math. And the more time I have to spend learning math, the less fun linguistics becomes.
Besides, if I had the brains for math, I'd be something useful, like a programmer.

Now excuse me, I have a Perl script to write and that damn chicken keeps running away...

Chris said...

bulbul, fair paint, hehe. But then again, I DID quit linguistics as an academic pursuit, so I'm not entirely hypocritical. If I were to take up academic linguistics again, I would consider myself obliged to become competent at R and Python in the very least.