‘SAM as an optimal relaxation of Bayes’ with Emtiyaz Khan

We are pleased to have Emtiyaz Khan, a team leader at the RIKEN Center for Advanced Intelligence Project (AIP) in Tokyo, join us for this joint ART-AI and University of Bath AI group seminar.

ART-AI and University of Bath AI group seminar

We are pleased to have Emtiyaz Khan, a team leader at the RIKEN Center for Advanced Intelligence Project (AIP) in Tokyo, join us for this joint ART-AI and University of Bath AI group seminar. This seminar will take place online on Thursday 24th November 2022, 09.15am-10.05am (GMT). Olga Isupova, lecturer in AI, Computer Science, University of Bath will be chairing. For joining instructions please e-mail [email protected].

Title

SAM as an optimal relaxation of Bayes

Abstract

Sharpness-aware minimization (SAM) and related adversarial deep-learning methods can drastically improve generalization, but their underlying mechanisms are not yet fully understood. In this talk, I will show that adversarial-loss (or max-loss) used in SAM is in fact the best convex relaxation of the expected-loss used in SAM. Using this, we can derive exact condition when SAM is equivalent to minimizing a Bayesian objective. We use this connection to obtain reasonable uncertainty estimates from SAM, while often getting some boost in accuracy too. By connecting adversarial and Bayesian methods, our work opens a new path to robustness. Joint work with Thomas Möllenhoff (https://www.thomasmoellenhoff.net/) Preprint: https://arxiv.org/abs/2210.01620

Bio

Emtiyaz Khan (also known as Emti) is a team leader at the RIKEN Center for Advanced Intelligence Project (AIP) in Tokyo where he leads the Approximate Bayesian Inference Team. Previously, he was a postdoc and then a scientist at Ecole Polytechnique Fédérale de Lausanne (EPFL), where he also taught two large machine learning courses and received a teaching award. He finished his PhD in machine learning from University of British Columbia in 2012. The main goal of Emti’s research is to understand the principles of learning from data and use them to develop algorithms that can learn like living beings. For the past 10 years, his work has focused on developing Bayesian methods that could lead to such fundamental principles. The approximate Bayesian inference team now continues to use these principles, as well as derive new ones, to solve real-world problems. Speakers webpage: https://emtiyaz.github.io/


Event Info

Start Date 24.11.2023
End Date 24.11.2022
Start Time 9:15pm
End Time 10:05am

Add to Google Calendar