关于fmri统计推断的一些讨论
Maillist里面关于FWE和FDR的讨论很有趣，其中一封信给了我很多启示。
Dear All,
Many mails concerning the FWE and FDR have been posted last days. More generally, I think that the problem with any statistical test of hypothesis is that we give a result which is dependent of the chosen threshold ( Dan Kimberg says " I don't really know much about the historical basis for the gold standard of FWER=0.05, but it's certainly in part cultural (how the standard is imposed varies across sub-fields even within a discipline).") I remember (in field of molecular biology) the example of a well-reputed scientist which definitively rejected a result associated with a p=0.052 and accepted without any doubt a result associated with a p=0.047!
I think the point is that the statistical test is only ONE indication (among many others) which has to be taken in the general context of the result. The result has to be related to the physiological and data context. In the physiological context, is the result in accordance with the previous literature? Is-it a surprising result? Are we able to give a satisfactory physiological/psychological explanation for this result? ....
In the data context, whatever the statistical test, we have to carefully examine the data (in fMRI studies, it is difficult because the data are very big). More precisely, I think that we have to perform at least 3 verifications:
- firstly for a group map, after chosen a threshold, examine all the clusters surviving the thresholding. For giving a sense to a particular cluster, I have the feeling that we have to find a logical (according to the experimental paradigm) explanation for all (or the large majority) the clusters exhibiting a greater significance. Is-it not some analogy with FDR, except that this method add a knowledge about the activated voxels?
- secondly, we have to observe the fitted, adjusted and residual time series and compared these time series with the characteristics of the data: realignment parameters, mean activities of the scans, observation of the autocorrelation, ...
- finally, the observation of each subject separately is a key verification. If the effect is observed for each subject (even at a lower threshold and a location slightly different of the group location), I think that the statistical test (either FWE, FDR or any other test) on the group make sense even if the p value is relatively high (p=0.05 and a moderate effect in 10/10 subjects seems to me more credible that a p=0.001 and the effect observed in 5/10 subject).
In brief, if it is no objective arguments to choose a test or an other test, a threshold or an other threshold, we have to consider not only the p value but other arguments (independent of the test) to evaluate the credibility of an effect.
Pierre
Dear All,
Many mails concerning the FWE and FDR have been posted last days. More generally, I think that the problem with any statistical test of hypothesis is that we give a result which is dependent of the chosen threshold ( Dan Kimberg says " I don't really know much about the historical basis for the gold standard of FWER=0.05, but it's certainly in part cultural (how the standard is imposed varies across sub-fields even within a discipline).") I remember (in field of molecular biology) the example of a well-reputed scientist which definitively rejected a result associated with a p=0.052 and accepted without any doubt a result associated with a p=0.047!
I think the point is that the statistical test is only ONE indication (among many others) which has to be taken in the general context of the result. The result has to be related to the physiological and data context. In the physiological context, is the result in accordance with the previous literature? Is-it a surprising result? Are we able to give a satisfactory physiological/psychological explanation for this result? ....
In the data context, whatever the statistical test, we have to carefully examine the data (in fMRI studies, it is difficult because the data are very big). More precisely, I think that we have to perform at least 3 verifications:
- firstly for a group map, after chosen a threshold, examine all the clusters surviving the thresholding. For giving a sense to a particular cluster, I have the feeling that we have to find a logical (according to the experimental paradigm) explanation for all (or the large majority) the clusters exhibiting a greater significance. Is-it not some analogy with FDR, except that this method add a knowledge about the activated voxels?
- secondly, we have to observe the fitted, adjusted and residual time series and compared these time series with the characteristics of the data: realignment parameters, mean activities of the scans, observation of the autocorrelation, ...
- finally, the observation of each subject separately is a key verification. If the effect is observed for each subject (even at a lower threshold and a location slightly different of the group location), I think that the statistical test (either FWE, FDR or any other test) on the group make sense even if the p value is relatively high (p=0.05 and a moderate effect in 10/10 subjects seems to me more credible that a p=0.001 and the effect observed in 5/10 subject).
In brief, if it is no objective arguments to choose a test or an other test, a threshold or an other threshold, we have to consider not only the p value but other arguments (independent of the test) to evaluate the credibility of an effect.
Pierre
评论
发表评论