LANS Seminar

Seminar Title: Rethinking Training Configurations for Deep Learning Models in Genomic
Speaker: Rohit Tripathy, Postdoctoral Researcher, Cold Spring Harbor Laboratory

Date/Time: June 22, 2022 / 10:30 am – 11:30 am
Location: See meeting URL on the cels-seminars website (requires Argonne login)

Host: Julie Bessac and Vishwas Rao

Description: A critical barrier to the widespread deployment of deep neural networks (DNNs) in regulatory genomics is the lack of DNN model interpretability. Conventional wisdom in parametric data-driven modeling posits that there is an inherent trade-off between the model complexity and interpretability. While the complex hierarchical structure of DNNs enables learning of rich, nonlinear patterns from data, it effectively renders the model as an uninterpretable black-box. Post-hoc explanation techniques such as gradient-based attribution methods reveal input features (or motifs) which drive the predictions made by a trained DNN. Unfortunately, while post-hoc attribution methods allow us to derive feature-level importance scores for the predictions of our models, such scores are noisy and their dependence on model architecture and training configurations is not fully understood. The standard deep learning model selection approach, based on optimizing a held-out validation dataset classification metric, discourages learning excessively complex models that might overfit to the training error. It is, however, unclear if such a process leads to models with accurate post-hocattribution maps, due to the lack of ground truth attributions.

In this talk, we discuss observations from recent experiments on the interplay between model performance and explainability in genomics. First, using a synthetic regulatory genomics dataset with known attributions, we demonstrate that the classic model selection process does not optimize for the post-hoc explainability of a DNN. In fact, we show that there is effectively no correlation between model performance and model interpretability. We hypothesize that such behavior owes to the non-robust nature of the learned functions, and show that we can simultaneously improve classification and interpretability of our models through an appropriate tuning of model regularizers. Specifically, we show consistent improvement through a particular schedule for the learning rate and batch normalization, which we call warm-up regularization.

Please note that the meeting URL for this event can be seen on the cels-seminars website, which requires an Argonne login.