Do Innovative Item Types Really Work?

Presented at ICE Exchange 2018

Austin, TX

The purpose of innovative item types is to present items in a way that more accurately represent real life situations while also increasing the measurement quality of the exam. Although these item types may be more aligned to real life experiences, is there a tradeoff statistically or security-wise? Do more innovative item types get statistically flagged after being pretested than traditional multiple choice items? If so, why? How difficult is it to train item writers to write effective innovative item types? If innovative item types are effectively written, do they do a better job of discriminating between high and low ability candidates than traditional multiple-choice questions?

This session addresses the strengths and weaknesses of innovative item types from the perspective of a licensing organization, emphasizing their practical and psychometric implications. Participate in a facilitated discussion to learn from and hear about experiences from other organizations.

Speaker(s)
Nick Respecki, NCARB
Brett Foley, Alpine
Amanda Wolkowitz, Alpine