Presented at the ATP Innovations in Testing 2021 Virtual Conference
Pretesting items is a necessity in developing valid, reliable, and fair assessments. This case study examines a scenario of two 76-item/point beta forms being randomly administered to two unintentionally different samples. The difference in form means was over 8 points, with SEMs of 2.25 and 2.23. By form, the average Rasch item measures differed by 0.25 and the average Rasch person measures differed by 0.46. The items on Form A were easier and the beta sample was more able in comparison to Form B. A psychometrician and a certification program representative present how they collaboratively handled discrepant beta forms and samples, with a policy mandate requiring multiple live forms. This presentation reviews the findings from the beta and live forms analyses — including a discussion of handling candidate fairness and the validity of score interpretations from the perspective of both the certification program and policy mandates as well as psychometric implications.
Speakers:
- Jim Henderson, Ping Identity
- Jill van den Heuvel, Alpine Testing Solutions
To learn more about the ATP Innovations in Testing 2021 Virtual Conference, click here.
To view the slides from this session, click the button below: