(143) Evaluating students’ experience of a technology designed to support learning how to detect medication-related problems

Objective : To evaluate perceived benefits and limitations of a new medication-related problem detection tool (MRPDT) that was designed to support students learning to systematically analyze complex health records.

Methods: A mixed methods descriptive research design was utilized. Two previously validated 5-point Likert scales—the Systems Usability Scale (SUS) and Reduced Instructional Materials Motivation Survey (RIMMS)—and an open-ended questionnaire were used to evaluate perceptions of the benefits and limitations of the MRPDT from multiple viewpoints, including usability, motivational design, and overall learning experience. Quantitative data were analyzed via descriptive statistics. SUS ratings were converted to an overall score ranging from 0-100 as described in the relevant methods literature. Qualitative data were analyzed using a two-cycle inductive coding process to identify themes.

Results: This study was conducted in a required P2 pharmacy practice course (n=79 students). RIMMS mean scores were above the midpoint for all subscales, indicating acceptable motivational design: attention (3.64±1.19), relevance (3.92±1.07), confidence (3.94±1.09), and satisfaction (3.48±1.25). Additionally, the SUS score was 66.9 out of 100. Based on extensive previous research, this indicates acceptable system usability with room for improvement. Finally, qualitative analysis revealed 5 themes related to students’ overall learning experience. Theme 1: The MRPDT Mediates Sensemaking Efforts; Theme 2: The MRPDT Orients a Systematic Discovery Process; Theme 3: The MRPDT Supports Internalizing a Systematic Process; Theme 4: Aspects of the MRPDT UX Design Create Friction for Analyzing Patient Information; and Theme 5: Preference for Case-Specific Rather than General Process for Case Analysis.

Conclusions: The MRPDT benefitted students by supporting their process for analyzing health records. Identification of limitations in the current design provide direction for future research intended to improve and further validate the tool.