(images from Neuroskeptic)
Neuroskeptic reports on some disturbing news that the results of fMRI studies can be seriously impacted by the software package used to analyze the results. There are several packages available and while most do much the same thing, at least one uses a unique statistical approach which produces different results. Not "better" or "worse" mind you, just different. The image above contrasts results using the same data but different analysis software. Money quote:
Analysis using both programs revealed that during the processing of emotional faces, as compared to the baseline stimulus, there was an increased activation in the visual areas (occipital, fusiform and lingual gyri), in the cerebellum, in the parietal cortex [etc] ... Conversely, the temporal regions, insula and putamen were found to be activated using the XBAM analysis software only (emphasis added).
The comments on Neuroskeptic's post are detailed and instructive.
3 comments:
"Disturbing news"?
I wouldn't consider this disturbing or news. Every statistics course emphasizes that you can get different "results" based on the statistical test you use. You have to know your data and statistics well enough to choose the appropriate test for the assumptions you can make about your data. This is true for ALL science, not just fMRI research. It seems obvious that software packages that use different statistical tests (and therefore different assumptions about the data) will yield different results. As will all science, the key is for the experimenter to justify those assumptions. This is just a case study in that principle.
neuromusic, fair point. You are right, anyone trading in stats should 'know their data.' I guess I found it disturbing in the sense that psycholinguists (the academics I'm most interested in) tend to under-explain their fMRI methodology, making it seem as though the mere fact that they collected fMRI data means they have magic data that is beyond reproach.
This makes me worry that many of them under-understand the data interpretation issues. Tools are great (as I've discussed here) but a little knowledge can be a dangerous thing.
I'm in agreement with neuromusic: the fact that doing different stats tests on the same data gives somewhat different results is not surprising.
What does worry me - as Chris says - is that many fMRI researchers (& it's not just psycholinguists) don't fully understand the statistics they're doing and hence don't know the assumptions they're making, what the possible biases in the data are, etc.
Post a Comment