(PO-041) Pre/post COVID-10 Outcomes in a Competency-based Course in Orthodontics

Author: Bilal Chaudhary, B.Sc. – Dental Student, Class of 2022, NYU College of Dentistry
Author: Jae Jun You, B.S. – Dental Student, Class of 2022, NYU College of Dentistry
Author: Lea A. Chilazi, B.S. – Dental Student, Class of 2022, NYU College of Dentistry
Author: Mitchell J. Lipp, D.D.S. – Clinical Professor, NYU College of Dentistry
Submitter: Bilal Chaudhary, B.Sc. – Dental Student, Class of 2022, NYU College of Dentistry

Objective: In a D3 Orthodontics course at NYU Dentistry, changes affecting instruction and assessment occurred after the COVID pandemic. Pre- and post-COVID performance on summative assessments was compared to establish a new baseline.

Methods: Summative assessments for pre- COVID (2018, Nf89) and post-COVID (2020, Nf 98) groups were based on the same four clinical simulation cases (CSC). Evaluative criteria, the basis for critical errors (CE) and the grader were the same. Pre-COVID assessment involved construction of a problem list, treatment objectives, and treatment plan with 0 CE on one CSC to pass. Post-COVID assessment involved five essay prompts (Malocclusion, Skeletal problems, Treatment plan, Space management upper and lower), with 100% correct response based in reason/evidence with 0 CE on at least one CSC to pass. Assessment scores were 0-4: 4-Highest score, 0-did not pass. Pass rates compared using chi-square test (X2) and p-value. Assessment scores compared by percentage (Ngrade/Ntotal), and error bars set at 95% confidence intervals.

Results: Pre-Covid CSC pass rate was 81% (72/89) and post-Covid CSC pass rate was 88% (86/98). Pre-and post-COVID pass rates were not significantly different, X2 (df=1, Nf187) = 1.19, p = 0.2793. Pre-Covid scores (0-4):  0 = 20%, 1 = 37%, 2 = 24%, 3 = 18%, 4 = 1%. Post-Covid scores:  0 = 12%, 1 = 32%, 2 = 29%, 3 = 23%, 4 = 4%.

Conclusion: This project establishes a new outcome-based baseline for future studies. The pass rates for the post-COVID group were higher than the pre-COVID group but not statistically significant. More encouraging was higher performance in the post-COVID group. This may be attributed to the new methods used or a result of the assessment conditions (i.e., remote, non-proctored environment). Future investigations will explore that possibility.