Deep Doc | Issue 2

Dealing with Dataset Shift in Medicine

Hi everyone,

Thanks for the overwhelming support. Our goal is to provide quality news content regarding machine learning in medicine for you in this and coming versions. 

There are 6 updates we highlight in this issue:

1. Dataset shift is one of the main challenges for AI model generalization. For example, in a clinical setting, training data may differ from the data used by the model to provide diagnostic, prognostic, or treatment advice. Finlayson et al. have published letters in the New England Journal of Medicine outlining how to identify and potentially mitigate familiar sources of dataset shift in machine learning systems. 

2. Casto et al. have considered causal reasoning to tackle different types of shifts in medical imaging. Their paper published in Nature provides a causal perspective on addressing the common challenges in machine learning for medical imaging: the scarcity of high-quality annotated data and mismatch between training and test dataset. 

3. In a recent paper published at Lancet, Seah et al. tried to assess radiologists' performance with and without the assistance of deep learning models. Compared to an AUC of 0.713 from unassisted radiologists, radiologists assisted by the model achieved AUC 0.808. Interestingly, the model alone achieved an AUC of 0.957. Surprised? 

4. In another work published at JAMA Network, Chi et al. tried to answer if developing a novel artificial intelligence (AI) system to organize patient health records would improve physicians' ability to extract patient information. They found that the AI technology developed helps physicians extract relevant patient information in a shorter time.

5. AlphaFold's method paper and code have (finally) been released. If you are interested to learn more about the technical aspects of the method and how it can be generalized to other molecular problems, Columbia University's Professor Mohammed AlQuaishi's recent blog might be helpful. 

6. The World Health Organization (WHO) has published extensive guidance a while ago on Ethics & Governance of Artificial Intelligence for Health. The report identifies the ethical challenges and risks with artificial intelligence in health, six consensus principles to ensure AI works to the public benefit of all countries. 

Events:

Stanford University's Center for Artificial Intelligence in Medicine & Imaging (AIMI) organized the BOLD-AIR Summit on August 3 and 4, 2021. The conference recordings will be available on their website and the AIMI YouTube Channel 1-2 weeks after the event.