Prev Next
Close close-icon

2023 ATP Innovations in Testing Global Conference

March 13-15, 2023

We are excited to learn about the possibilities at this year’s ATP conference. Stop by to say hello and start your day with Alpine + a custom hot beverage at our booth in Dallas. We look forward to seeing you all in March!

Be sure to add these sessions to your ATP agenda:

Monday, March 13

  • 11:15 am – 12:05 pm
  • Session Track: Panel Discussions
  • Presenters:
    • Dana Anderson-Pancoe, Tableau Education Services
    • Carl Bowman and Danielle Castelli, CompTIA
    • Jim Henderson, Ping Identity
    • Angelica Mulfinger, Alpine Testing Solutions

Non-traditional item types are not new, but how they are developed and used continues to evolve. When industry professionals share their experiences developing and administering these item types, they provide invaluable learning opportunities for everyone involved. This panel of industry professionals focuses on sharing experiences with developing and administering non-traditional item types (e.g., performance-based items, simulation items, lab items). The panel covers topics ranging from getting started with non-traditional item types (why did they decide to use these item types?) to lessons learned (what are the successes and missteps?). The panel will also share their reflections on current and future value-add of these item types. This panel brings together multiple experiences and perspectives on the development and implementation of non-traditional item types so that other organizations might learn from their experiences, because we are better together.

  • 3:20 pm – 4:10 pm
  • Session Track: Sponsored Speaker Session
  • Presenters:
    • Lisa O’Leary and Corina Owens, Alpine Testing Solutions

With exam security as a pervasive and persistent concern for certification programs; it is critical that credentialing organizations remain vigilant and consistent with regard to protection of program integrity. However, there is often a disconnect between a desire to prevent, detect, and mitigate security threats and the ability to translate that into practical enforcement of policy. During this session, Alpine will simplify those often-complex security analyses and provide concrete examples that will allow for interpretation of advanced security analyses in a way that is both practical and conceptually easy to understand and explain. Accessible approaches to detection of item pre-knowledge, collusion among test takers, and proxy testing will be highlighted along with pragmatic recommendations to support timely and defensible exam actions.

Tuesday, March 14

  • 8:30 am – 9:40 am
  • Session Track: Innovation Fast-Pitch Finalist
  • Presenters:
    • Lisa O’Leary, Alpine Testing Solutions

Integrating Cultural Humility and Mindfulness in Test Item Writing

  • 9:55 am – 11:05 am
  • Session Track: Workshops
  • Presenters:
    • Kim Cavanagh, Gannon University and PAEA
    • Denise Rizzolo, PAEA
    • Brett Foley and Nemanja Dukic, Alpine Testing Solutions

Test item writing is critical to ensure that items are culturally responsible and inclusive. Training subject matter experts to recognize, understand, and appropriately introduce characteristics of race, ethnicity, culture, and identity in test items can provide evidence of exam fairness. This workshop will guide participants through peer review training aimed at uncovering unconscious bias. First, facilitators will provide participants with background information. Next, participants will split into groups. Each group will participate in peer review training and will apply best practices for developing culturally mindful test items. Then, participants will practice the process of peer review in their groups to identify and correct bias in test items. Finally, everyone will come together to share their results, discuss the implications of different training methods, and learn how this process can be complemented by post-exam statistical analysis.

FEATURED SPEAKER SESSION: Real World Implementations of Two, New Collusion Statistics: Approximation Response and Score Similarity Analyses

  • 11:35 am – 12:25 pm
  • Session Track: Panel Discussions
  • Presenters:
    • Patrick Irwin, Palo Alto Networks
    • Danielle Castelli, CompTIA
    • Russell Smith and Corina Owens, Alpine Testing Solutions

Exam security continues to be a primary concern for credentialing organizations. Many organizations struggle to find a solution that is near real-time, automated, and actionable. This panel presentation will describe a relatively easy and efficient method to identify potential collusion. The method is meant to approximate score or response similarity (SSI or RSI) analyses with relatively simple calculations. SSI and RSI are typically computationally intensive, require specialized software and are based on relatively strong assumptions.

During this session, the panelists will begin by providing an accessible introduction to the two new, simplified similarity indices, Approximation RSI and SSI. The remainder of the session will focus on the application of each approximation to two distinctly different certification programs. Specifically, each certification organization will discuss the pros and cons of implementing Approximation RSI and SSI within their programs, how and when these analyses are actionable, and if and how these analyses have changed their approach to security. This session will leave participants with increased knowledge and tools to support timely, computationally less intensive, and actionable results that can be implemented at their own organizations.

Are My Exam Items Really Biased? Bringing DEIA to the Forefront

  • 1:25 pm – 2:15 pm
  • Session Track: Panel Discussions
  • Presenters:
    • Jared Zurn, National Council of Architectural Registration Boards (NCARB)
    • Emily Yunker, Physician Assistant Education Association (PAEA)
    • Amanda Wolkowitz, Data Recognition Corporation (DRC)
    • Susan Cooley and Brett Foley, Alpine Testing Solutions

Detection and prevention of item bias has been researched and discussed for years; however, advancements in and increased awareness of diversity in society has caused the testing industry to revisit the effectiveness of bias review processes. The purpose of this panel discussion is to bridge the classical item bias review processes of the past with current practices and concerns of today. The panel will include experts from different professional testing organizations and a psychometrician. During the first half of the session, the panelists will discuss: 1) changes DEIA have brought to the item development process; 2) effective item bias review methods; and 3) psychometric best practices and statistical methods for detecting item bias. The second half of the session will focus on questions from the audience. Attendees will leave this session with a better understanding of how item bias may unknowingly occur in their exams and ways to both prevent it and address it.

Better Together. Assessments. Learning. Outcomes.

To learn more about the 2023 ATP Innovations in Testing Global Conference, please click here.