(371) Using a Collaborative Online Grading Platform to Assess Written Journal Article Critique Assignments

Trainee Category: Fellow

Objective : To assess the impact of using a collaborative online grading platform known as Gradescope to grade written journal article critique assignments in a drug information course.

Methods: Second-year pharmacy students enrolled in the required drug information and literature evaluation course at Purdue University College of Pharmacy are required to submit 3 written journal article critique assignments throughout the semester to help develop their skills in critically analyzing medical literature. The critiques include assessment of the journal, authors, study subjects, study design, endpoints, statistical analysis, results, and impact on practice. Student assignment scores for these assignments in 2017 and 2018 (before Gradescope implementation) and 2020 and 2021 (after Gradescope implementation) were retrospectively reviewed to assess the impact of Gradescope on score variability. The coefficient of variation (COV) was used to evaluate the spread of assignment scores relative to the mean. An F-test was used to compare the pre-Gradescope and post-Gradescope COVs.

Results: The analysis included 1,769 assignment scores. Each assignment was worth a maximum of 40 points. Mean scores for pre-Gradescope assignments #1, #2, and #3 were 30.8±4.4, 31.3±4.4, and 32.4±4.1, respectively, while the mean scores for post-Gradescope assignments #1, #2, and #3 were 31.3±3.8, 33.1±3.8, and 33.5±3.2, respectively. Post-Gradescope COVs were significantly lower than pre-Gradescope COVs at 12.2% and 14.2% (p=.01072​) for assignment #1, 11.5% and 14.2% (p=.00040) for assignment #2, and 9.5% and 12.6% ​(p <.00001​) for assignment #3.

Conclusions: Implementation of Gradescope reduced the variability of journal article critique scores and may be a useful tool to improve consistency in scoring of written assignments.