Presented at the Conference on Test Security 2022
The development of score similarity and answer similarity indices in recent years has added to our psychometric arsenal for detecting candidate collusion and content exposure. These indices have been shown to be effective and often are calculated by comparing the expected and observed number of matching item scores/responses for each pair of candidates. These results are useful for identifying pairs of candidates with questionable levels of agreement. However, as candidate volume grows, the number of candidate pairs grows quadratically, resulting in very large numbers of pairs for even modest sample sizes. Given the extreme number of possible comparisons, it can be difficult for non-technical audiences to understand the extent of an exposure problem. In other words, is a security issue negligible, isolated, or widespread? In this session, the presenter will begin by briefly describing best practices for creating exam-security related visual displays. Next, we will present several examples of visual displays that use data from all candidate pairs to effectively communicate the prevalence and extent of collusion and content exposure issues in the testing population. The session is appropriate for both technical and non-technical audiences. Attendees will learn about important factors to consider when creating visual displays, be shown specific examples of effective tools for summarizing collusion/content exposure issues, and learn how scaling group frequencies can help highlight outliers or emphasize common patterns.
Speaker:
- Brett Foley, Alpine Testing Solutions
To learn more about the 2022 Conference on Test Security, click here.
To view the slides from this session, click the button below: