Statistical significance testing with mahalanobis distance for thresholds estimated from constant stimuli method

Takehiro Nagai, Takahiro Hoshino, Keiji Uchikawa

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

The t-test and the analysis of variance are commonly used as statistical significance testing methods. However, they cannot assess the significance of differences between thresholds within individual observers estimated from the constant stimuli method; these thresholds are not defined as averages of samples, but they are rather defined as functions of parameters of psychometric functions fitted to participants' responses. Moreover, the statistics necessary for these statistical testing methods cannot be derived. In this paper, we propose a new statistical testing method to assess the statistical significance of differences between thresholds estimated from the constant stimuli method. The new method can assess not only threshold differences but also main effects and interactions in multifactor experiments, exploiting the asymptotic normality of maximum likelihood estimators and the characteristics of multivariate normal distributions. This proposed method could be used in similar cases to the analysis of variance for thresholds estimated from the adjustment method and the staircase method. Finally, we present some data on simulations in which we tested assumptions, power and type I error of the proposed method.

Original languageEnglish
Pages (from-to)91-124
Number of pages34
JournalSeeing and Perceiving
Volume24
Issue number2
DOIs
Publication statusPublished - 2011
Externally publishedYes

Fingerprint

Analysis of variance (ANOVA)
Testing
Normal distribution
Maximum likelihood
Statistics
Analysis of Variance
Social Adjustment
Experiments
Normal Distribution
Psychometrics

Keywords

  • Constant stimuli method
  • Monte Carlo simulation
  • Psychophysics
  • Statistical hypothesis testing

ASJC Scopus subject areas

  • Sensory Systems
  • Cognitive Neuroscience
  • Ophthalmology
  • Experimental and Cognitive Psychology
  • Computer Vision and Pattern Recognition

Cite this

Statistical significance testing with mahalanobis distance for thresholds estimated from constant stimuli method. / Nagai, Takehiro; Hoshino, Takahiro; Uchikawa, Keiji.

In: Seeing and Perceiving, Vol. 24, No. 2, 2011, p. 91-124.

Research output: Contribution to journalArticle

@article{2533ac1573b644f6a2e8a5d75e359fe4,
title = "Statistical significance testing with mahalanobis distance for thresholds estimated from constant stimuli method",
abstract = "The t-test and the analysis of variance are commonly used as statistical significance testing methods. However, they cannot assess the significance of differences between thresholds within individual observers estimated from the constant stimuli method; these thresholds are not defined as averages of samples, but they are rather defined as functions of parameters of psychometric functions fitted to participants' responses. Moreover, the statistics necessary for these statistical testing methods cannot be derived. In this paper, we propose a new statistical testing method to assess the statistical significance of differences between thresholds estimated from the constant stimuli method. The new method can assess not only threshold differences but also main effects and interactions in multifactor experiments, exploiting the asymptotic normality of maximum likelihood estimators and the characteristics of multivariate normal distributions. This proposed method could be used in similar cases to the analysis of variance for thresholds estimated from the adjustment method and the staircase method. Finally, we present some data on simulations in which we tested assumptions, power and type I error of the proposed method.",
keywords = "Constant stimuli method, Monte Carlo simulation, Psychophysics, Statistical hypothesis testing",
author = "Takehiro Nagai and Takahiro Hoshino and Keiji Uchikawa",
year = "2011",
doi = "10.1163/187847511X568180",
language = "English",
volume = "24",
pages = "91--124",
journal = "Multisensory research",
issn = "2213-4794",
publisher = "Brill",
number = "2",

}

TY - JOUR

T1 - Statistical significance testing with mahalanobis distance for thresholds estimated from constant stimuli method

AU - Nagai, Takehiro

AU - Hoshino, Takahiro

AU - Uchikawa, Keiji

PY - 2011

Y1 - 2011

N2 - The t-test and the analysis of variance are commonly used as statistical significance testing methods. However, they cannot assess the significance of differences between thresholds within individual observers estimated from the constant stimuli method; these thresholds are not defined as averages of samples, but they are rather defined as functions of parameters of psychometric functions fitted to participants' responses. Moreover, the statistics necessary for these statistical testing methods cannot be derived. In this paper, we propose a new statistical testing method to assess the statistical significance of differences between thresholds estimated from the constant stimuli method. The new method can assess not only threshold differences but also main effects and interactions in multifactor experiments, exploiting the asymptotic normality of maximum likelihood estimators and the characteristics of multivariate normal distributions. This proposed method could be used in similar cases to the analysis of variance for thresholds estimated from the adjustment method and the staircase method. Finally, we present some data on simulations in which we tested assumptions, power and type I error of the proposed method.

AB - The t-test and the analysis of variance are commonly used as statistical significance testing methods. However, they cannot assess the significance of differences between thresholds within individual observers estimated from the constant stimuli method; these thresholds are not defined as averages of samples, but they are rather defined as functions of parameters of psychometric functions fitted to participants' responses. Moreover, the statistics necessary for these statistical testing methods cannot be derived. In this paper, we propose a new statistical testing method to assess the statistical significance of differences between thresholds estimated from the constant stimuli method. The new method can assess not only threshold differences but also main effects and interactions in multifactor experiments, exploiting the asymptotic normality of maximum likelihood estimators and the characteristics of multivariate normal distributions. This proposed method could be used in similar cases to the analysis of variance for thresholds estimated from the adjustment method and the staircase method. Finally, we present some data on simulations in which we tested assumptions, power and type I error of the proposed method.

KW - Constant stimuli method

KW - Monte Carlo simulation

KW - Psychophysics

KW - Statistical hypothesis testing

UR - http://www.scopus.com/inward/record.url?scp=80053387306&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053387306&partnerID=8YFLogxK

U2 - 10.1163/187847511X568180

DO - 10.1163/187847511X568180

M3 - Article

VL - 24

SP - 91

EP - 124

JO - Multisensory research

JF - Multisensory research

SN - 2213-4794

IS - 2

ER -