(117) Assessing Student Performance on Open- versus Closed-Book Summative Assessments

Objective : Findings from studies in nursing, medical, and general undergraduate education reveal little difference in performance between open-book and closed-book examination formats. The objective of this study was to investigate if allowing students to use notes during an examination affects student exam performance in an advanced pharmacotherapy elective course.

Methods: The following comparisons were made for Fall 2021 (closed-book) and Fall 2022 (open-book): exam grades, grades based on Bloom's taxonomy categories, and exam item response time by Bloom's category. The following Bloom’s categories were used for analysis: level 1 (knowledge/comprehension), level 2 (analysis/application), and level 3 (evaluation/synthesis). Unpaired t-test was used to compare cohorts.

Results: The difference in average examination scores was significantly higher for open-book as compared to closed-book format (82% vs. 74%, p<.05). Open-book examinations had more level 1 questions as compared to closed-book examinations. Students performed significantly better on level 1 and level 2 questions in open-book as compared to closed-book format (83.2% vs. 74.6% for level 1, p<.05; 79.9% vs. 69.5% for level 2, p<.05). There was no significant difference in examination performance for level 3 questions. The average response time was significantly shorter for level 1 questions for students who took open-book examinations as compared to students who took closed-book examinations (56 seconds vs. 85 seconds, p<.05). The was no significant difference in average response time for level 2 and level 3 questions.

Conclusions: Students performed better on open-book examination level 1 and level 2 questions. Students used less time to answer level 1 questions on the open-book examination. Differences in exam difficulty between the open-book and closed-book formats pose a challenge to interpreting the results.