Training procedures and regulatory policies for safe machine learning models in healthcare

February 19, 2020
3:00 to 4:00pm

Jeng J. Feng, PhD Student, University of Washington

Before we can realize the potential of machine learning (ML) in healthcare, we need to ensure the safety of these models for medical decision-making.  In this talk, we explore 1) how to learn safer ML models and 2) how to allow modifications to them over time.  As a first step to learning safer models, I will discuss the need for fail-safes in ML.  Typical models give predictions for all inputs, which can be overconfident or even misleading for out-of-sample inputs.  I introduce a penalization method for learning models that can abstain from making predictions and apply it to predict patient outcomes in the intensive care unit.  In the second half of the talk, we discuss how the US FDA might regulate modifications to ML-based Software as a Medical Device (SaMD) without hindering innovation.  To this end, we consider policies that can automatically evaluate proposed modifications without human intervention.  I define a framework for evaluating the error rates of such policies and show that policies without error rate guarantees are prone to "bio-creep."  I then show how to protect against it by combining group sequential and online hypothesis testing methods.


Social hour following: 4-5pm

Event Type: 
Biostatistics and Bioinformatics Seminar