COTS 2019 – Comparing B3 to Answer Similarity Index for Detecting Collusion

Presented at the Conference on Test Security 2019

Miami, FL

Standard Presentation IV

Foley (2019) proposed using a variation of Yen’s Q₃ statistic (1984) for collusion detection, calling it B₃. Where Yen’s Q₃ statistics uses item residual correlations to detect dependencies between pairs of items, the B₃ statistic uses person residual correlations to detect dependencies between pairs of test takers. This paper will compare and contrast the advantages and disadvantages of the B₃ statistic to Answer Similarity Index (ASI) analyses using the generalized binomial model (van der Linden & Sotaridona, 2006). The B₃ statistic has a few potential advantages for detecting collusion. It is readily available via commercial software (e.g. Winsteps, Linacre (2019)), it is generalizable to polytomous models, and it accounts for in both the number of matching scores and the likelihood of the scores given the test takers’ abilities and the difficulty of the individual items. One potential disadvantage of the B₃ statistic is that the underlying distributions for B₃ is not known making it challenging to estimate probabilities. Real data with known exposure problems as well as simulated data will be used to evaluate the proposed statistic and its viability by specifically comparing it to ASI.

Speaker:

• Russell Smith, Ph.D., Alpine Testing Solutions

To view the slides from this session, click the button below: