{"id":3010,"date":"2022-03-07T11:33:55","date_gmt":"2022-03-07T17:33:55","guid":{"rendered":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/?post_type=tribe_events&#038;p=3010"},"modified":"2022-06-17T12:27:19","modified_gmt":"2022-06-17T17:27:19","slug":"lans-seminar-34","status":"publish","type":"tribe_events","link":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/event\/lans-seminar-34\/","title":{"rendered":"LANS Seminar"},"content":{"rendered":"<p><strong>Seminar Title<\/strong>: Rethinking Training Configurations for Deep Learning Models in Genomic<br \/>\n<strong>Speaker<\/strong>: Rohit Tripathy, Postdoctoral Researcher, Cold Spring Harbor Laboratory<\/p>\n<p><strong>Date\/Time<\/strong>: June 22, 2022 \/ 10:30 am &#8211; 11:30 am<br \/>\n<strong>Location<\/strong>: See meeting URL on the cels-seminars website\u00a0(requires Argonne login)<\/p>\n<p><strong>Host<\/strong>: Julie Bessac and Vishwas Rao<\/p>\n<hr \/>\n<p><strong>Description<\/strong>: A critical barrier to the widespread deployment of deep neural networks (DNNs) in regulatory genomics is the lack of DNN model interpretability. Conventional wisdom in parametric data-driven modeling posits that there is an inherent trade-off between the model complexity and interpretability. While the complex hierarchical structure of DNNs enables learning of rich, nonlinear patterns from data, it effectively renders the model as an uninterpretable black-box.\u00a0<em>Post-hoc<\/em>\u00a0explanation techniques such as gradient-based attribution methods reveal input features (or\u00a0<em>motifs<\/em>) which drive the predictions made by a trained DNN. Unfortunately, while\u00a0<em>post-hoc<\/em>\u00a0attribution methods allow us to derive feature-level importance scores for the predictions of our models, such scores are noisy and their dependence on model architecture and training configurations is not fully understood. The standard deep learning model selection approach, based on optimizing a held-out validation dataset classification metric, discourages learning excessively complex models that might overfit to the training error. It is, however, unclear if such a process leads to models with accurate\u00a0<em>post-hoc<\/em>attribution maps, due to the lack of ground truth attributions.<\/p>\n<p>In this talk, we discuss observations from recent experiments on the interplay between model performance and explainability in genomics. First, using a synthetic regulatory genomics dataset with known attributions, we demonstrate that the classic model selection process does not optimize for the\u00a0<em>post-hoc<\/em>\u00a0explainability of a DNN. In fact, we show that there is effectively no correlation between model performance and model interpretability. We hypothesize that such behavior owes to the non-robust nature of the learned functions, and show that we can simultaneously improve classification and interpretability of our models through an appropriate tuning of model regularizers. Specifically, we show consistent improvement through a particular schedule for the learning rate and batch normalization, which we call\u00a0<em>warm-up regularization<\/em>.<\/p>\n<p><em>Please note that the meeting URL for this event can be seen on the <\/em>cels-seminars website<em>, which requires an Argonne login.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Seminar Title: Rethinking Training Configurations for Deep Learning Models in Genomic Speaker: Rohit Tripathy, Postdoctoral Researcher, Cold Spring Harbor Laboratory Date\/Time: June 22, 2022 \/ 10:30 am &#8211; 11:30 am Location: See meeting URL on the cels-seminars website\u00a0(requires Argonne login) &hellip; <a href=\"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/event\/lans-seminar-34\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":43,"featured_media":0,"template":"","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_tribe_events_status":"","_tribe_events_status_reason":"","footnotes":"","_members_access_role":[],"_members_access_error":""},"tags":[],"tribe_events_cat":[2],"class_list":["post-3010","tribe_events","type-tribe_events","status-publish","hentry","tribe_events_cat-seminar","cat_seminar"],"acf":[],"_links":{"self":[{"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tribe_events\/3010","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tribe_events"}],"about":[{"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/types\/tribe_events"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/users\/43"}],"version-history":[{"count":3,"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tribe_events\/3010\/revisions"}],"predecessor-version":[{"id":3100,"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tribe_events\/3010\/revisions\/3100"}],"wp:attachment":[{"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/media?parent=3010"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tags?post=3010"},{"taxonomy":"tribe_events_cat","embeddable":true,"href":"https:\/\/wordpress.cels.anl.gov\/lans-seminars\/wp-json\/wp\/v2\/tribe_events_cat?post=3010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}