Title
Author(s)
Date
Title
A Computationally Simple Method for Estimating Decision ConsistencyAuthor
Wolkowitz, Amanda A.
Date: Jun 14, 2021Author(s): Wolkowitz, Amanda A.
Description:As published in the Journal of Educational Measurement (JEM), June 14, 2021
Abstract:
Decision consistency (DC) is the reliability of a classification decision based on a test score. In professional credentialing, the decision is often a high-stakes pass/fail decision. The current methods for estimating DC are computationally complex. The purpose of this research is to provide a computationally and conceptually simple method for estimating DC that produces results comparable to, and at times potentially better than, the widely used Livingston-Lewis method.
Authors and Affiliations:
- Amanda A. Wolkowitz, Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
Title
Worth the Squeeze? An Investigation into the Psychometric Performance of Alternative Item TypesAuthor
Wolkowitz, Amanda A.; Foley, Brett P.; Zurn, Jared
Date: Feb 1, 2021Author(s): Wolkowitz, Amanda A.; Foley, Brett P.; Zurn, Jared
Description:As published in the Journal of Applied Testing Technology (JATT) Vol 22(1), 12- 24, 2021
Abstract:
As assessments move from traditional paper-pencil administration to computer-based administration, many testing programs are incorporating alternative item types (AITs) into assessments with the goals of measuring higher-order thinking, offering insight into problem-solving, and representing authentic real-world tasks. This paper explores multiple applications of AIT items and the psychometric properties of these items, including item response time, difficulty, item-total score correlations, and distractor analyses. The appropriate use of these items is also discussed in the context of professional credentialing exams.
Author Affiliations:
- Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
- Director of Professional Credentialing and Senior Psychometrician, Alpine Testing Solutions, Inc., 51 W. Center Street, #514, Orem, UT 84057, United States
- Vice President, Examination, National Council of Architectural Registration Boards, 1401 H St NW #500, Washington, DC 20005, United States, United States
Title
Fairness Concerns of Discrete Option Multiple Choice ItemsAuthor
Eckerly, Carol; Smith, Russell; and Sowles, John
Date: 2018Author(s): Eckerly, Carol; Smith, Russell; and Sowles, John
Description:Eckerly, Carol; Smith, Russell; and Sowles, John (2018) “Fairness Concerns of Discrete Option Multiple Choice Items,” Practical Assessment, Research, and Evaluation: Vol. 23 , Article 16.
Title
Closing the Loop: Providing Test Developers with Performance Level Descriptors so Standard Setters Can Do Their JobAuthor
Wolkowitz, A., Impara, J. C., & Buckendahl, C. W.
Date: 2017Author(s): Wolkowitz, A., Impara, J. C., & Buckendahl, C. W.
Description:Wolkowitz A.A., Impara J.C., Buckendahl C.W. (2017) Closing the Loop: Providing Test Developers with Performance Level Descriptors So Standard Setters Can Do Their Job. In: Blömeke S., Gustafsson JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham
Available online.
Title
Presentations & WorkshopsAuthor
Foley, B. P.
Date: Jan 3, 2017Author(s): Foley, B. P.
Description:Presentations:
Foley, B. P., Terry, J. E., & Zurn, J. (2016, November [accepted]). Communication strategies for improving stakeholder buy-in. Presentation at the annual conference of the Institute for Credentialing Excellence, Colorado Springs, CO. Foley, B. P., Terry, J. E., & Wilkins, J. R. (2015, October) Small-scale credentialing programs: Balancing security, fairness, and candidate-friendliness. Presentation at the annual conference of the Institute for Credentialing Excellence, Portland, OR.
Buckendahl, C. W., & Foley, B. P. (2015, April). Policy linking as cut score moderation: Considerations for practice. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, Il.
Foley, B. P. (2014, April). Evaluating an impact percentage smoothing vertically moderated standard setting design. Paper presented at the annual meeting of the National Council on Measurement in Education, Philadelphia, PA.
Abernathy, T., Bliss, T.J., Vineyard, R., Wentworth, N., and Foley, B. P. (2013, October). Evaluating the effects of the Common Core State Standards initiative and its associated assessment consortia testing programs on teacher training programs. Panel discussion presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY
Lim, G. S., Buckendahl, C. W., & Foley, B. P. (2012, July). Evaluating test design through complementary standard setting studies. Paper presented at The 8th Conference of the International Test Commission, Amsterdam, Netherlands.
Foley, B. P., & Lupher, D. A. (2012, April). Evaluating the stability of cut scores over time in a statewide assessment system. Paper presented at the annual meeting of the National Council on Measurement in Education, Vancouver, BC.
Foley, B. P. (2011, April). Realistic Expectations: State-level Changes in the Percentage of Proficient Students 2002-2008. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
Workshops:
Foley, B. P. (2015, April). Using visual displays to inform assessment development and validation. Workshop presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
Foley, B. P. (2011, October). Validity by design: A validity-centered assessment development approach. Workshop presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY.
Foley, B. P. (2009, October). Prerequisites for power analysis: Questions you need to ask before deciding the sample size for your study. Workshop presented at the Annual meeting of the Northern Rocky Mountain Educational Research Association, Jackson Hole, WY.
Title
Visual displays of test fraud dataAuthor
Foley, B. P.
Date: 2016Author(s): Foley, B. P.
Description:Foley, B. P. (2016). Visual displays of test fraud data. In G. J. Cizek & J. A. Wollack (Editors). Handbook of quantitative methods for detecting cheating on tests. New York, NY: Routledge.
Title
Getting lucky: How guessing threatens the validity of performance classificationsAuthor
Foley, B. P.
Date: 2016Author(s): Foley, B. P.
Description:Foley, B. P. (2016). Getting lucky: How guessing threatens the validity of performance classifications. Practical Assessment, Research and Evaluation, 21(3).
Available online.
Title
Releasing content to deter cheating: An analysis of the impact on candidate performanceAuthor
Wolkowitz, A., Davis-Becker, S. L., & Gerrow, J. D.
Date: 2016Author(s): Wolkowitz, A., Davis-Becker, S. L., & Gerrow, J. D.
Description:Wolkowitz, A., Davis-Becker, S. L., & Gerrow, J. D. (2016). Releasing content to deter cheating: An analysis of the impact on candidate performance. Journal of Applied Testing Technology, 17(1), 33-40.
Title
Evaluating common item block options when faced with practical constraintsAuthor
Wolkowitz, A. & Davis-Becker, S.
Date: 2015Author(s): Wolkowitz, A. & Davis-Becker, S.
Description:Wolkowitz, A. & Davis-Becker, S. (2015). Evaluating common item block options when faced with practical constraints. Practical Assessment, Research, & Evaluation, 20(19).
Available online.
Title
Tailoring visual displays to improve test score interpretation: Including indicators of uncertaintyAuthor
Foley, B. P.
Date: 2015Author(s): Foley, B. P.
Description:Foley, B. P. (2015). Tailoring visual displays to improve test score interpretation: Including indicators of uncertainty. In M. McCrudden, G. Schraw, & C. Buckendahl (Editors). Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 265-298). Charlotte, NC: Information Age.
Title
Modeling the relationships among measures of teacher quality and student performance in high school geometryAuthor
Foley, B. P.
Date: 2014Author(s): Foley, B. P.
Description:Foley, B. P. (2014). Modeling the relationships among measures of teacher quality and student performance in high school geometry. The Researcher, 26(1), 28-33.
Title
Using visual displays to inform assessment design and developmentAuthor
Foley, B. P., & Buckendahl, C. W.
Date: 2013Author(s): Foley, B. P., & Buckendahl, C. W.
Description:Foley, B. P., & Buckendahl, C. W. (2013). Using visual displays to inform assessment design and development. In G. Schraw, M. McCrudden, & D. Robinson (Editors). Learning through visual displays (pp. 417-445). Charlotte, NC: Information Age.
Title
A Method For Imputing Response Options For Missing Data on Multiple-Choice AssessmentsAuthor
Wolkowitz, A. & Skorupski, W. P.
Date: 2013Author(s): Wolkowitz, A. & Skorupski, W. P.
Description:Wolkowitz, A. & Skorupski, W. P. (2013). A method for imputing response options for missing data on multiple-choice assessments. Educational and Psychological Measurement, 73(6), 1036-1053.
Title
Testing in the movies and on television [Motion Picture]Author
Foley, B. P., Dwyer, A. C., Chuah, D., Rawls, A.
Date: 2013Author(s): Foley, B. P., Dwyer, A. C., Chuah, D., Rawls, A.
Description:Foley, B. P., Dwyer, A. C., Chuah, D., Rawls, A. (Producers), & Foley, B. P. (Director). (2013). Testing in the movies and on television [Motion Picture]. United States: National Council on Measurement in Education.
View online.
Title
Multiple attempts on a nursing admissions examination: Effects on the total scoreAuthor
Wolkowitz, A.
Date: 2011Author(s): Wolkowitz, A.
Description:Wolkowitz, A. (2011). Multiple attempts on a nursing admissions examination: Effects on the total score. Journal of Nursing Education 50(9), 493-501.
Title
High Stakes Uses of Intelligence TestingAuthor
Buckendahl, C. W. & Foley, B. P.
Date: 2011Author(s): Buckendahl, C. W. & Foley, B. P.
Description:Buckendahl, C. W. & Foley, B. P. (2011). High Stakes Uses of Intelligence Testing. In J. Bovaird, K. Geisinger, & C. Buckendahl (Editors). High Stakes Testing in Education – Science and Practice in K-12 Settings [Festschrift to Barbara Plake] (pp. 191-210). Washington, DC: American Psychological Association Press.
Title
Academic Predictors of Success in a Nursing ProgramAuthor
Wolkowitz, A. & Kelley, J.
Date: 2010Author(s): Wolkowitz, A. & Kelley, J.
Description:Wolkowitz, A. & Kelley, J. (2010) Academic predictors of success in a nursing program. Journal of Nursing Education, 49, 498-503.
Title
Study Manual for the Health Occupations Basic Entrance Examination – Version VAuthor
Wolkowitz, A.
Date: 2010Author(s): Wolkowitz, A.
Description:Wolkowitz, A. (Ed.) (2010). Study Manual for the Health Occupations Basic Entrance Examination – Version V. Stilwell, KS: Assessment Technologies Institute®.
Title
Study Manual for the Test of Academic Skills – Version VAuthor
Wolkowitz, A.
Date: 2009Author(s): Wolkowitz, A.
Description:Wolkowitz, A. (Ed.) (2009). Study Manual for the Test of Academic Skills – Version V. Stilwell, KS: Assessment Technologies Institute®.
Title
Learning Strategies: Your Guide to Classroom and Test-taking SuccessAuthor
Wolkowitz, A.
Date: 2009Author(s): Wolkowitz, A.
Description:Wolkowitz, A. (Ed.) (2009). Learning Strategies: Your Guide to Classroom and Test-taking Success. Stilwell, KS: Assessment Technologies Institute®.
Title
Developing Test Forms for Small-Scale Achievement Testing SystemsAuthor
Jones, P., Smith, R., and Talley, D. M.
Date: 2006Author(s): Jones, P., Smith, R., and Talley, D. M.
Description:Jones, P., Smith, R., and Talley, D. M. (2006). Developing test forms for small-scale achievement testing systems. In Downing, S.M., and Haladyna, T.M., Handbook of test development. Mahwah, NJ: Lawrence Erlbaum Associates.