Artificial Intelligence in Clinical Informatics

The objectives for this course are for participants to understand:

• Learn the history and terminology of AI, and understand its major branches. 
• Develop intuition for how AI algorithms work
• Examine the use cases of AI in healthcare  
• Understand the core decision points when developing any AI system
• Understand important considerations when designing an AI tool for clinical implementation
• Design an AI project and communicate via a proposal
• Develop skills to leverage generative AI tools such as Large Language Models to advance research and clinical operations work.

None

Each week, new material is introduced via in-person lecture. A Laboratory session immediately follows, providing students with time to work on problem sets/activities with supervision and assistance from course leaders.

Lecture:
A lecture covering the core course topics.
Time: Thursdays, 2:45 PM - 3:45 PM, beginning April 3

Computer Laboratory:
Students will work on problem sets/activities with supervision and assistance from course leaders. Also an opportunity to ask questions related to class projects
Time: Thursday, 3:45 PM - 4:45 PM, beginning April 3

All course materials and handouts will be posted on the course's online syllabus. Learners will need to request access to several resources 2 weeks prior to the course in preparation for the course, including the UCSF Research Analysis Environment (RAE), the UCSF de-identified Clinical Data Warehouse, UCSF Versa, and possibly MIMIC IV. Instructions on how to do so will be sent out prior to the start of the course..

All course materials and handouts will be posted on the course's online syllabus.

Grades will be based on total points achieved on two course projects (50 % each). 

There will be 2 mini class projects, one for each major segment of the course (traditional ML/AI, generative AI). Each is expected to take you around 15-20 hours to complete. Assignments will be submitted to course directors for review.

Traditional ML/AI project

The goal of this project is twofold: (1) for learners to create a thorough 5-page single space proposal for a clinical AI training, evaluation, and implementation process and (2) to repurpose the Python code from the course labs to work with a clinical data set of their choosing. Some suggested datasets will be provided. The primary learning outcome for the first half of this course is to think critically about design decisions across the AI/ML development and implementation pipeline in clinical informatics. Students will therefore detail the data source (including summary statistics about its properties), modeling procedures, pre-translation evaluation strategy with primary endpoints justified based on the clinical application, real-world implementation strategy, and post-translation evaluation strategy. Part 1: Students will follow the ML-PICO framework outlined by Liu et al., requiring the identification of the type of ML problem, population, identification of gold standard labels, crosschecking for ML best practices, and evaluated outcomes. Throughout the proposal, students will also be required to address each of the 3 grand challenges of AI in clinical informatics: fairness/bias, interpretability/explainability, and/or privacy/security. Part 2: Students will repurpose the Python code from their labs to create a baseline machine learning model using the dataset they chose.

Generative AI project

The goal of this project is to have learners develop skills in using Large Language models to perform tasks using clinical notes such as: structured data extraction, cohort discovery, outcome labeling, etc. Learners will use prompt engineering to create a prompt that can be used to perform one of the above tasks related to a clinical area of interest that they have, either for research or operational purposes. Learners at UCSF will be able to do this through the deidentified CDW and Versa. Learners outside of UCSF will instead use the MIMIC-IV database in combination with Gemini.

Students not in full-year TICR Programs who satisfactorily pass all course requirements will, upon request, receive a Certificate of Course Completion.

Only UCSF students (defined as individuals enrolled in UCSF degree or certificate programs) will receive academic credit for courses. Official transcripts are available to UCSF students only. A Certificate of Course Completion will be available upon request to individuals who are not UCSF students and satisfactorily pass all course requirements.

UCSF Graduate Division Policy on Disabilities