Thursday, August 12, 2010

marc hauser reactions

The social media world is abuzz with reactions to the Marc Hauser story.  From the blogs--

Mark Liberman:
Like many other linguists, Geoff and I have felt from the beginning that the results of Hauser's monkey experiments were of dubious relevance to the evolution of speech and language. Now we're forced to question whether there were any reliable results at all.

Drug Monkey:
What I am worried about in this type of coverage is the conflation of a failure to replicate a study with the absence of evidence (per the retraction blaming a trainee) with scientific debate over the interpretation of data. The mere failure of an investigation to be able to replicate a prior one is not in and of itself evidence of scientific misconduct. Scientific findings, legitimate ones, can be difficult or impossible to replicate for many reasons and even if we criticize the credulity, scientific rigor or methods of the original finding, it is not misconduct.

John hawks:
The problem of subjective data is not unique to Hauser's work but is systemic in the field of primate cognition. It reminds me of some discussion in Jeremy Taylor's recent book Not a Chimp: The Hunt to Find the Genes that Make Us Human. There's the issue of whether experiments are designed clearly enough to yield conclusions. Then there's the second issue of whether observations are replicable, or whether they result only from somewhat "wishful" researchers. Such experiments often get heightened scrutiny, but rarely is there clear misconduct. That makes this a really shocking case.

Razib Khan:
Hauser is a prominent public intellectual...Obviously problems in some aspects of his work doesn’t necessarily invalidate all his findings, but it doesn’t look good for his credibility. This sort of incident points to the importance of trust within the culture of science. Collaborators and researchers who cited his results are scrambling to make sense of it all.

Open Parachute:
...we should recognise that we are seeing one of the methods science has for self correction. The science community treats deliberate distortion of evidence, poor record keeping and biased interpretation of results very seriously.

There are going to be people who use this news to attack science. But we should ask them if they are prepared to submit their beliefs, ideology or claims to such scrutiny? And are they willing to be disciplined if an investigation finds that they have made distorted or false claims?

Art Markman:
I find cases like this both frustrating and reassuring at the same time.

The frustrating part of cases of misconduct is fairly obvious. As a scientist, all I really have is the integrity of my data. Theories are nice, of course. We create theories to help us to explain patterns of data. But, really, theories are most useful because they help use to develop new questions that we can ask that will help use to collect new data. [...] At the same time, cases of misconduct are reassuring. Science is remarkably self-correcting. When we publish papers in scientific journals, we organize our papers in a way that reflects the ideals laid out by Francis Bacon. We give enough of the details about our methods that someone else could repeat the study we are presenting. We present details about the analysis of our data. After a paper is published, authors often make their data available to others who want to do additional analyses of the work.

David Dobbs:
To me the allegations, vague as they are, don’t quite rise to shocking yet, though I may be missing something. (Please point it out if so.) But that confusion underlines how important it is, methinks, for Harvard to spell out just what is being looked at here. Not just Hauser but a lot of people who have drawn on, contributed to, or worked parallel to his work are hanging in the wind here.

FYI: The author of the original story follows up today with this: Harvard is urged to detail inquiry.

No comments:

A linguist asks some questions about word vectors

I have at best a passing familiarity with word vectors, strictly from a 30,000 foot view. I've never directly used them outside a handfu...