2023 I.C.E. Exchange Conference
Stop by to say hello and start each day with a custom, hot beverage with Alpine at Booth #311 in the exhibit hall. We look forward to seeing you again soon!
Tuesday, October 24
Multiple-Choice Item Writing 101
- Pre-Conference Workshop
- 8:00 – 11:00 am MT
- Colorado Hall D
Presenters:
- Lisa O’Leary, Ph.D., VP, Product & Service Management & Senior Psychometrician, Alpine Testing Solutions
- Corina Owens, Ph.D., VP, Client Services & Senior Psychometrician, Alpine Testing Solutions
This workshop is designed for professionals new to the credentialing field. The presenters will share fundamental concepts for how to support subject-matter experts (SMEs) in writing and reviewing items. Attendees will also learn strategies that can help streamline processes without sacrificing psychometric rigor.
Wednesday, October 25
Empowering Excellence: Continuing Education in the Digital Age
- Solutions Showcase Session
- 6:30 – 7:00 pm MT
- Broadmoor Hall F
Presenters:
- Michael Andrews, Product Director, Technology Solutions, Alpine Testing Solutions
- Keith Becraft, VP, Technology, Alpine Testing Solutions
Join us for a thought-provoking presentation on the exciting world of continuing education (CE) and lifelong learning. In today’s rapidly evolving professional landscape, staying ahead of the curve has never been more critical. Our session, “Empowering Excellence: Continuing Education in the Digital World,” will shed light on the multifaceted realm of CE, with a focus on all facets of CE including hybrid and longitudinal CE models, diverse training programs, immersive labs, and much more.
Thursday, October 26
Advancements in Job Analysis and Content Outline Development
- 9:00 – 10:00 am MT
- Colorado Hall D
Presenters:
- Robert Brucia, Ph.D., Psychometrician, American Board of Pediatrics
- Andrew C. Dwyer, Ph.D., Director of Psychometrics, American Board of Pediatrics
- Brett P. Foley, Ph.D., Senior Psychometrician, Client Solutions, Alpine Testing Solutions
Job analysis is one of the most foundational and important elements of any certification program. This session highlights recent research and methodological changes that two organizations have implemented to improve their respective job analysis processes.
Alpine Testing Solutions worked with a large-scale licensure program to update their analysis of practice, with a primary goal of maximizing the breadth of stakeholder input. In this presentation we will describe the study, which consisted of multiple phases and included both qualitative and quantitative methods. Innovative design elements including reviews of position descriptions, electronic bulletin boards, ethnographies, web-based interviews, and social intercepts were combined with traditional elements like large-scale surveys to obtain supporting data from more than 24,000 stakeholders.
The American Board of Pediatrics (ABP) conducted research and implemented several changes that were designed to improve process efficiency and the utility of resulting content outlines. First, we will share research results supporting the use of a single “relevance” rating scale on job analysis surveys in lieu of commonly used “criticality” and “frequency” scales. In addition, we will also share results from an investigation into the relationship between content outline relevance ratings (obtained from the job analysis survey) and item-level relevance ratings (obtained from the ABP’s longitudinal assessment), which may be helpful for cross-validating job analysis results and improving future item development efforts. Other advancements will also be shared, including a set of recently developed content outline guidelines aimed at maximizing the clinical relevance of exam items. In addition to describing these activities, the session will include a discussion of lessons learned, anticipated future directions, and Q&A from the audience.
Smooth Sailing: Mastering Asynchronous Item Writing and Review to Keep Your Program Afloat
- 3:45 – 4:45 pm MT
- Broadmoor Hall D
Presenters:
- Susan Cooley, MA, PMP, Business Development Manager, Alpine Testing Solutions
Are you curious about how credentialing organizations are changing the approach to how subject matter experts (SMEs) write and review exam questions? With staggering technological advancements made in the last few years, organizations are moving towards an asynchronous process that allows SMEs to contribute from anywhere. However, maintaining the quality of these questions is crucial to ensuring the exams maintain a high level of rigor and reliability. Streamlining this asynchronous process and maintaining quality items are critical overarching issues that credentialing organizations need to address to adhere to rigorous psychometric practices and standards.
In this session, hear a case-study about a real world asynchronous item writing and review process that Alpine used to help a credentialing organization develop items for their certification exams. Challenges encountered during this project and the tools and methods used to remediate these issues will be shared, including:
- Empowering SMEs with tailored, on-demand training to help them craft and/or review high-quality items
- Designing an appropriate workflow for items so that they can be effectively remediated
- Enabling SMEs to collaborate with each other asynchronously
- Helping SMEs who are struggling to write quality items
- Gaining SME consensus on item revisions and approval/rejection
- Communicating with SMEs to provide feedback and make sure deadlines are met
- Maintaining SME engagement throughout the process
Stump the Psychometrician!
- 3:45 – 4:45 pm MT
- Colorado Hall C
Presenters:
- Amin Saiar, Ph.D., Senior Director, Psychometrics, Board of Pharmacy Specialties
- Amanda Dainis, Ph.D., MPA, CEO / Lead Psychometrician, Dainis and Company, Inc.
- Brett P. Foley, Ph.D., Senior Psychometrician, Client Solutions, Alpine Testing Solutions
- Moderator: Amanda Wolkowitz, Ph.D., Director, Psychometrics, Licensure and Certification, Data Recognition Corporation
We challenge you to a game of Stump the Psychometrician! In this session, you’ll have the opportunity to ask the tough psychometric questions to which you haven’t been able to get an answer. All psychometric topics are fair game in this game: program design, job analyses, item development, form development, standard setting, equating, fairness, AI and ML, security, accreditation, etc. In addition to getting your tough questions answered, if you stump our panel of psychometricians with a question that none of them can answer, you’ll win a prize! Even better, if two of the expert panelists include the phrase “It depends” when answering your question or if one panelist spends more than 90 seconds answering your question, you’ll get a prize, too! Come prepared to ask your tough questions because this panel of psychometricians who come from a variety of backgrounds and organizations are ready to be challenged in this game and don’t want to be stumped!
The conference for the credentialing community. The name Exchange reflects what is valued most by our annual conference attendees: the exchange of industry trends and best practices with the credentialing community.
To learn more about the I.C.E. Exchange, please click here.