Journal of Applied Measurement
A publication of the Department of Educational Psychology and Counseling
National Taiwan Normal University
Volume 23, Issue 3/4 (2022)
Special Issue: The Pillars of Measurement Wisdom
FOCUS ARTICLE
The Pillars of Measurement Wisdom
George Engelhard, Jr.
The University of Georgia
The purpose of this study is to discuss the foundations of measurement in the human sciences. This discussion is framed by a consideration of the pillars of statistical wisdom proposed by Stigler (2016), and their relationships to key concepts in measurement theory. Stigler (2016) identified seven pillars of statistics: aggregation, likelihood, information, intercomparison, regression, design, and residuals. Each of these pillars has an interesting counterpart for measurement. There are several unique pillars for educational measurement that include a consideration of the power and consequences of using measures. Rasch measurement theory provides the guiding framework for considering the pillars of measurement.
Keywords: Foundations of statistics, foundations of measurement, history and philosophy of measurement, Rasch measurement theory
Citation:
Engelhard, G., Jr. (2022). The pillars of measurement wisdom. Journal of Applied Measurement, 23(3/4), 80–95.
COMMENTARIES
Download Link Here:Are There Pillars of Measurement?
R. J. De Ayala
University of Nebraska – Lincoln
n/a
Citation:
De Ayala, R. J. (2022). Are there pillars of measurement? Journal of Applied Measurement, 23(3/4), 96–102.
Download Link Here:
Discussant Remarks on The Pillars of Measurement Wisdom
Carol M. Myford
Professor Emerita
University of Illinois at Chicago
n/a
Citation:
Myford, C. M. (2022). Discussant remarks on the pillars of measurement wisdom. Journal of Applied Measurement, 23(3/4), 103–104.
Download Link Here:
How to Avoid a Parody of Measurement: Some Models are Wiser Than Others A Commentary on the Pillars of Measurement Wisdom by George Engelhard, Jr.
Thomas Salzberger
WU Wien University of Economics and Business
n/a
Citation:
Salzberger, T. (2022). How to avoid a parody of measurement: Some models are wiser than others. A commentary on the pillars of measurement wisdom by George Engelhard, Jr. Journal of Applied Measurement, 23(3/4), 105–112.
Download Link Here:
Measurement and Data Information
Michelle Liou
Institute of Statistical Science
Academia Sinica
n/a
Citation:
Liou, M. (2022). Measurement and data information. Journal of Applied Measurement, 23(3/4), 113–116.
REJOINDER
Download Link Here:Response to Commentaries on Pillars of Measurement Wisdom
George Engelhard, Jr.
The University of Georgia
n/a
Citation:
Engelhard, G., Jr. (2022). Response to commentaries on pillars of measurement wisdom Journal of Applied Measurement, 23(3/4), 117–120.
REGULAR ARTICLES
Validity and Test-Length Reduction Strategies for Complex Assessments
Lance M. Kruse
invontics, LLC.
Gregory E. Stone
University of Toledo
Toni A. May
Drexel University
Jonathan D. Bostic
Bowling Green State University
Lengthy standardized assessments decrease instructional time while increasing concerns about student cognitive fatigue. This study presents a methodological approach for item reduction within a complex assessment setting using the Problem Solving Measure for Grade 6 (PSM6). Five item-reduction methods were utilized to reduce the number of items on the PSM6, and each shortened instrument was evaluated through validity evidence for test content, internal structure, and relationships to other variables. The two quantitative methods (Rasch model and point-biserial) resulted in the best psychometrically performing shortened assessments but were not representative of all content subdomains, while the three qualitative (content preservation) methods resulted in poor psychometrically performing assessments that retained all subdomains. Specifically, the ten-item Rasch and ten-item point-biserial shortened tests demonstrated the overall strongest validity evidence, but future research is needed to explore the psychometric performance of these versions in a new independent sample and the necessity for subdomain representation. Implications for the study provide a methodological framework for researchers to use and reduce the length of existing instruments while identifying how the various reduction strategies may sacrifice different information from the original instrument. Practitioners are encouraged to carefully examine to what extent their reduced instrument aligns with their pre-determined criteria.
Keywords: item reduction, validity evidence, psychometric, assessments, Rasch
Citation:
Kruse, L. M., Stone, G. E., May, T. A., & Bostic, J. D. (2022). Validity and test-length reduction strategies for complex assessments. Journal of Applied Measurement, 23(3/4), 121–143.
Examining the Psychometric Properties of the Spanish Version Life Orientation Test-Revised (LOT-R): Finding Multi-Dimensionality and Issues With Translation for Reverse-Scored Items
Rosalba Hernandez
Kendon J. Conrad
University of Illinois at Chicago
John Ruiz
Melissa Flores
University of Arizona
Judith T. Moskowitz
Northwestern University, Feinberg School of Medicine
Linda C. Gallo
San Diego State University
Erin L. Merz
California State University
Frank J. Penedo
University of Miami
Ramon A. Durazo-Arvizu
The Saban Research Institute, Children’s Hospital Los Angeles
University of Illinois at Chicago
Angelica P. Gutierrez
San Diego State University
Jinsong Chen
University of Illinois at Chicago
Martha L. Daviglus
Northwestern University, Feinberg School of Medicine
University of Illinois at Chicago
The Life Orientation Test-Revised (LOT-R) is the most used instrument to assess dispositional optimism. We examined the psychometrics of the LOT-R in a diverse sample of U.S. Hispanics/Latinos. Data analysis included 5,140 adults ages 18–74 in the Hispanic Community Health Study/Study of Latinos and Sociocultural Ancillary Study. We employed the Rasch measurement model using Winsteps software. The Rasch person reliability for the 6-item LOT-R and Cronbach’s alpha both had values of 0.54. When testing convergent validity, correlations were statistically significant, but small to medium in magnitude. The ratio of percentage of variance explained by the measures to the variance explained in the first contrast and the correlation of subscales did not meet the expected unidimensionality criterion. The item, “I hardly expect things to go my way,” displayed differential item functioning by language (Spanish vs. English) and reverse-scored items were found to be problematic. Use of the LOT-R in its present form in U.S. Hispanics/Latinos is unsupported by psychometric evidence.
Keywords: Life Orientation Test-Revised, dispositional optimism, psychometric testing, Rasch analysis, Hispanics/Latinos
Citation:
Hernandez, R., Conrad, K. J., Ruiz, J., Flores, M., Moskowitz, J. T., Gallo, L. C., Merz, E. L., Penedo, F. J., Durazo-Arvizu, R. A., & Gutierrez, A. P. (2022). Examining the psychometric properties of the Spanish Version Life Orientation Test-Revised (LOT-R): Finding multi-dimensionality and issues with translation for reverse-scored items. Journal of Applied Measurement, 23(3/4), 144–164.