Publication: Sampling plans for fitting the psychometric function.
Loading...
Files
Official URL
Full text at PDC
Publication Date
2005-11
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Citations
Exportar
Abstract
Research on estimation of a psychometric function psi has usually focused on comparing alternative algorithms to apply to the data, rarely addressing how best to gather the data themselves (i.e., what sampling plan best deploys the affordable number of trials). Simulation methods were used here to assess the performance of several sampling plans in yes-no and forced-choice tasks, including the QUEST method and several variants of up-down staircases and of the method of constant stimuli (MOCS). We also assessed the efficacy of four parameter estimation methods. Performance comparisons were based on analyses of usability (i.e., the percentage of times that a plan yields usable data for the estimation of all the parameters of psi) and of the resultant distributions of parameter estimates. Maximum likelihood turned out to be the best parameter estimation method. As for sampling plans, QUEST never exceeded 80% usability even when 1000 trials were administered and rendered accurate estimates of threshold but misestimated the remaining parameters. MOCS and up-down staircases yielded similar and acceptable usability (above 95% with 400-500 trials) and, although neither type of plan allowed estimating all parameters with optimal precision, each type appeared well suited to estimating a distinct subset of parameters. An analysis of the causes of this differential suitability allowed designing alternative sampling plans (all based on up-down staircases) for yes-no and forced-choice tasks. These alternative plans rendered near optimal distributions of estimates for all parameters. The results just described apply when the fitted psi has the same mathematical form as the actual psi generating the data; in case of form mismatch, all parameters except threshold were generally misestimated but the relative performance of all the sampling plans remained identical. Detailed practical recommendations are given.
Description
UCM subjects
Unesco subjects
Keywords
Citation
Alcalá-Quintana, R., & García-Pérez, M.A. (2002, August). Bias and standard errors in forced-choice Bayesian staircases. Paper presented at the 33rd European Mathematical Psychology Group Meeting, Bremen, Germany.
Alcalá-Quintana, R., & García-Pérez, M.A. (2004a). The role of parametric assumptions in adaptive Bayesian estimation. Psychological Methods, 9, 250-271.
Alcalá-Quintana, R., & García-Pérez, M.A. (2004b). Empirical performance of optimal Bayesian adaptive psychophysical methods. Perception (Suppl.), 33, 178. Balakrishnan, N. (1992).
Handbook of the logistic distribution. New York: Marcel Dekker.
Berkson, J. (1955). Maximum likelihood and minimum χ2 estimates of the logistic function. Journal of the American Statistical Association, 50, 130-162.
Brand, T., & Kollmeier, B. (2002). Efficient adaptive procedures for threshold and concurrent slope estimates for psychophysics and speech intelligibility tests. Journal of the Acoustical Society of America, 111, 2801-2810.
Brown, L.G. (1996). Additional rules for the transformed up-down method in psychophysics. Perception & Psychophysics, 58, 959-962.
Dixon, W.J., & Mood, A.M. (1948). A method for obtaining and analyzing sensitivity data. Journal of the American Statistical Association, 43, 109-126.
Evans, M., Hastings, N., & Peacock, B. (1993). Statistical distributions (2nd ed.). New York: Wiley.
Foster, D.H., & Bischof, W.F. (1991). Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators. Psychological Bulletin, 109, 152-159.
Freeman, P.R. (1970). Optimal Bayesian sequential estimation of the median effective dose. Biometrika, 57, 79-89.
García-Pérez, M.A. (1998). Forced-choice staircases with fixed step sizes: Asymptotic and small-sample properties. Vision Research, 38, 1861-1881.
García-Pérez, M.A. (2001). Yes-no staircases with fixed step sizes: Psychometric properties and optimal setup. Optometry & Vision Science, 78, 56-64.
García-Pérez, M.A., Giorgi, R., Woods, R.L., & Peli, E. (2005). Thresholds vary between spatial and temporal forced-choice paradigms: The case of lateral interactions in peripheral vision. Spatial Vision, 18, 99-127.
Hall, J.L. (1981). Hybrid adaptive procedure for estimation of psychometric functions. Journal of the Acoustical Society of America, 69, 1763-1769.
Harvey, L.O., Jr. (1997). Efficient estimation of sensory thresholds with ML-PEST. Spatial Vision, 11, 121-128.
Kaernbach, C. (1991). Simple adaptive testing with the weighted up–down method. Perception & Psychophysics, 49, 227-229.
Kaernbach, C. (2001). Slope bias of psychometric functions derived from adaptive data. Perception & Psychophysics, 63, 1389-1398.
Kershaw, C.D. (1985). Statistical properties of staircase estimates from two interval forced choice experiments. British Journal of Mathematical & Statistical Psychology, 38, 35-43.
King-Smith, P.E., & Rose, D. (1997). Principles of an adaptive method for measuring the slope of a psychometric function. Vision Research, 37, 1595-1604.
King-Smith, P.E., Grigsby, S.S., Vingrys, A.J., Benes, S.C., & Supowit, A. (1994). Efficient and unbiased modifications of the QUEST threshold method: Theory, simulations, experimental evaluation and practical implementation. Vision Research, 34, 885-912.
Klein, S.A. (2001). Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63, 1421-1455.
Kontsevich, L.L., & Tyler, C.W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Research, 39, 2729-2737.
Lam, C.F., Mills, J.H., & Dubno, J.R. (1996). Placement of observations for the efficient estimation of a psychometric function. Journal of the Acoustical Society of America, 99, 3689-3693.
Leek, M.R., Hanna, T.E., & Marshall, L. (1992). Estimation of psychometric functions from adaptive tracking procedures. Perception & Psychophysics, 51, 247-256. Maloney, L.T. (1990). Confidence intervals for the parameters of psychometric functions. Perception & Psychophysics, 47, 127-134.
Marks, B.L. (1962). Some optimal sequential schemes for estimating the mean of a cumulative normal quantal response curve. Journal of the Royal Statistical Society, Series B, 24, 393-400.
McKee, S.P., Klein, S.A., & Teller, D.Y. (1985). Statistical properties of forced-choice psychometric functions: Implications of probit analysis. Perception & Psychophysics, 37, 286-298.
Miller, J., & Ulrich, R. (2001). On the analysis of psychometric functions: The Spearman–Kärber method. Perception & Psychophysics, 63, 1399-1420.
Müller, H.G., & Schmitt, T. (1990). Choice of number of doses for maximum likelihood estimation of the ED50 for quantal dose-response data. Biometrics, 46, 117-129.
Myers, R.H. (1990). Classical and modern regression with applications. Boston, MA: PWS-KENT.
Numerical Algorithms Group (1999). NAG Fortran library manual, Mark 19. Oxford: Author.
O’Regan, J.K., & Humbert, R. (1989). Estimating psychometric functions in forced-choice situations: Significant biases found in threshold and slope estimations when small samples are used. Perception & Psychophysics, 46, 434-442.
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351-356.
Ramsey, F.L. (1972). A Bayesian approach to bioassay. Biometrics, 28, 841-858.
Robbins, H., & Monro, S. (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22, 400-407.
Santoro, L., Burr, D., & Morrone, M. C. (2002). Saccadic compression can improve detection of Glass patterns. Vision Research, 42, 1361-1366.
Serrano-Pedraza, I., & Sierra-Vázquez, V. (2003, August). Comparison between two methods for estimation of the parameters of a psychometric function: Effect of initial guess. Poster presented at the 34th European Mathematical Psychology Group Meeting, Madrid, Spain.
Simmers, A.J., Bex, P.J., Smith, F.K.H., & Wilkins, A.J. (2001). Spatiotemporal visual function in tinted lens wearers. Investigative Ophthalmology & Visual Science, 42, 879-884.
Snoeren, P.R., & Puts, M.J.H. (1997). Multiple parameter estimation in an adaptive psychometric method: MUEST, an extension of the QUEST method. Journal of Mathematical Psychology, 41, 431-439.
Snowden, R.J., & Hammett, S.T. (1998). The effects of surround contrast on contrast thresholds, perceived contrast and contrast discrimination. Vision Research, 38, 1935-1945.
Solomon, J.A., & Morgan, M.J. (2000). Facilitation of collinear flanks is cancelled by non-collinear flanks. Vision Research, 40, 279-286.
Strasburger, H. (2001). Invariance of the psychometric function for character recognition across the visual field. Perception & Psychophysics, 63, 1356-1376.
Swanson, W.H., & Birch, E.E. (1992). Extracting thresholds from noisy psychophysical data. Perception & Psychophysics, 51, 409-422.
Treutwein, B. (1997). YAAP: Yet another adaptive procedure. Spatial Vision, 11, 129-134.
Treutwein, B., & Strasburger, H. (1999). Fitting the psychometric function. Perception & Psychophysics, 61, 87-106.
Watson, A.B., & Pelli, D.G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception & Psychophysics, 33, 113-120.
Watson, A.B., & Turano, K. (1995). The optimal motion stimulus. Vision Research, 35, 325-336.
Werkhoven, P., & Snippe, H.P. (1996). An efficient adaptive procedure for psychophysical discrimination experiments. Behavior Research Methods, Instruments, & Computers, 28, 556-562.
Wetherill, G.B., & Levitt, H. (1965). Sequential estimation of points on a psychometric function. British Journal of Mathematical & Statistical Psychology, 18, 1-10.
Wichmann, F.A., & Hill, N.J. (2001a). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293-1313.
Wichmann, F.A., & Hill, N.J. (2001b). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 1314-1329.