(369) Exploring the use of artificial intelligence (AI) assisted feedback for written assignments in pharmacy education

Trainee Category: Fellow

Objective : To explore pharmacy faculty perceptions about adoption of AI-assisted technology to provide student feedback for natural language written assignments.

Methods: An exploratory qualitative design was used to conduct semi-structured interviews with faculty members who coordinate drug information courses within pharmacy programs across the United States. Study participants were recruited via the AACP Connect Drug Information and Library Sciences (DILS) Section. Potential study subjects were asked to participate in a 45-minute to one-hour virtual interview via Microsoft Teams with a target sample size of 10 to 15 educators. An interview guide was developed using the Technology, Individual, Pedagogy, and Enabling Conditions (TIPEC) framework. Questions focused on faculty perceptions of AI-assisted feedback/grading, including potential barriers to implementation of this technology in courses with natural written language assignments. The interview guide was pilot tested with an experienced academician after which minor modifications were made to improve the clarity of questions. Data analysis will use an inductive coding approach to identify themes and patterns that appear in the aggregate responses.

Results: Seven pharmacy academicians have been interviewed to date. Data collection is ongoing and will continue until saturation is reached. Preliminary themes that have emerged include a lack of trust in the accuracy of AI feedback, concerns of poor interrater reliability between AI and faculty feedback, and AI limitations in providing detailed explanation in feedback for complex topics.

Conclusions: Final results and conclusions are pending. The results of this project will be used to assist with implementation of an AI-assisted feedback program for draft assignments within a drug information course.