Andrew Maul of UC Santa Barbara’s Gevirtz School will give the talk “Rethinking Traditional Methods of Survey Validation” on Wednesday, May 23 from 12 pm – 1:30 pm in Education 1201. The free event is sponsored by Quantitative Methods in Social Sciences.
Quantitative Methods in the Social Sciences at UCSB is an interdisciplinary Ph.D. emphasis available to students who wish to develop and use cutting-edge quantitative methods in social science research. Its curriculum is designed to provide students with the rigorous mathematical and statistical background necessary for advanced quantitative work, while also providing a broad interdisciplinary perspective on the use of quantitative methods in the social sciences. A growing number of departments and faculty participate in the emphasis and the weekly colloquia, which features speakers engaged in quantitative research across all social science disciplines.
Maul's talk will discuss that while it is commonly believed that self-reported, survey-based instruments can be used to measure a wide range of human properties of relevance to education and the social sciences, such as self-control, growth mindsets, and grit; the typical strategies employed for the validation of such surveys fall short of providing the kinds of rigorous tests of relevant hypotheses commonly expected in scientific research. His presentation will consist of three parts. In the first part, he will aim to illustrate the deficiency of common validation strategies by presenting a series of studies in which respondents were presented with survey items deliberately constructed to be uninterpretable, but the application of the mainstream validation procedures nonetheless returned favorable-appearing results. In the second part, he will attempt to diagnose some of the meta-theoretical issues that have contributed to the present state of affairs, in particular by examining the legacy of operationalist and behaviorist modes of thinking in the social sciences. In the third part, he will discuss some possible strategies for the improvement of surveys, and of survey-based research more generally.
Andrew Maul is an Assistant Professor of research methodology in the Department of Education. He earned his Ph.D. at UC Berkeley, working with Mark Wilson; he then completed a three-year postdoctoral appointment at the University of Oslo, Norway, and served for two years on the faculty of the University of Colorado, Boulder, prior to coming to UCSB. Maul’s work focuses on the conceptual foundations of research methodology in the social sciences, and in particular on the measurement of psychological attributes. He attempts to rethink and challenge the received wisdom on research methods via an exploration of the history and philosophy of social research. He is also currently working on a National Academy of Education / Spencer Postdoctoral Fellowship focused on improving the measurement of ‘affective’ characteristics including perseverance, self-control, and academic mindsets.