(387) Let’s ChatGPT: Implementation of a ChatGPT-Generated Social Determinants of Health Activity

Trainee Category: Fellow

Objective : To evaluate the implementation of a ChatGPT-generated social determinants of health activity in team-based pharmacy education

Methods: The studied activity occurred within a required first-year, skills-based pharmacy course at The University of Tennessee Health Science Center. Participants completed a ChatGPT-generated, in-class activity on social determinants of health (SDOH) within groups. Instructors asked the software to generate a set of learning objectives, an in-class activity, assessment strategies, and assessment tools for SDOH knowledge acquisition. Instructors reframed various logistical aspects, including allotted activity time and correction of generated data, ensuring appropriate implementation within the course. Pre- and post-assessment data collected included knowledge-based multiple-choice and Likert-type questions. Data were analyzed using quantitative statistics.

Results: Data collected (n=95; 100% response rate) showed mean scores on the pre- and post-assessments were 86.1 (SD 15.6; 95% CI 83.0-89.2) and 88.7 (SD 13.9; 95% CI 85.9-91.5), respectively. For Likert-type questions, 94.7% (n=90) agreed, "The quality of this in-class activity was on-par with other in-class activities in the course," and 90.5% (n=86) agreed, "This in-class exercise was just as impactful to my professional development as other in-class activities." When asked, "Regarding quality, I feel the in-class activity was __ than other in-class activities," 93.6% (n=89) reported the activity was "similar, somewhat better, or far better."

Conclusions: These findings suggest that the ChatGPT-generated activity has congruency with instructor-generated activities. Per student perceptions, the generated activity was similar to or better than previous activities in the course, with the added benefit of decreasing planning time. However, there were noticeable disadvantages, such as the generation of incorrect assessment strategies. Future research is needed to assess ChatGPT's utility in lecture-based settings.