Senior Associate Dean Texas Tech University Health Sciences Center Lubbock, United States
Objective : Previous data have found repeated testing to be more effective than repeated studying. Using data derived from an assessment model in which students were assessed 8 times in composite exams and then upon final exams, a method for analyzing performance on repeat items is proposed to evaluate “how can repeat item performance be analyzed to measure concept retention?” and “how can such analysis differentiate competencies associated with mastery”?
Methods: In a quasi-experimental model, examinations were analyzed over one semester. For each item repeated in both a composite and final exam, each response at first exposure was compared to response at repeat exposure. Retention rates were calculated (number of correct responses on both exams /number of correct responses on the composite exam only). Venn diagrams were constructed to represent correct responses on first exposure, second exposure, and both. Additional analysis explored the time between first and repeat exposures and the competency each question measured. Competencies associated with multiple items were grouped for analysis of mastery.
Results: 62 repeat items were identified from the first-year Pharmacy curriculum. Average retention rates were 71-97% for each concept area. Time between first and repeat exposure did not influence outcomes. Competency areas associated with a higher degree of mastery included inter-professional collaboration, mathematics, leadership and population-based care. Competency areas associated with less degree of mastery were Pharmaceutical sciences, laws and regulations, and biomedical sciences.
Conclusions: This analysis illustrates a method of measuring concept retention. Analysis of performance on repeat items can identify competencies associated with mastery.
