UAI 2017 - Tutorials

Methods and models for large-scale optimization

John C. Duchi, Stanford University

In this tutorial, we cover a biased view of recent movements in optimization and its intersection with machine learning and statistics, with an eye toward applying these algorithms. Broadly, we believe these advances lie in two major areas: the first is in large-scale stochastic optimization problems, where the growing size in dataset - both in sample size and dimension - due to the amount of information we track and collect has led to substantial research in the optimization literature. The second is a more recent trend toward non-convex optimization problems. While, as Rockafellar notes, the justifiable dogma in optimization is that "the great watershed in optimization isnt between linearity and nonlinearity, but convexity and nonconvexity," a number of problems in statistics that are nominally non-convex are in fact still (with high probability) easy to solve using simple gradient-based and other methods. Unifying both of these trends is that the inherent randomness in the problems we solve - sampling randomness from collecting data - lends itself to simple optimization methods with convergence guarantees that are, at least, good enough. Our tutorial addresses these trends in three parts: first, by covering methods for large-scale stochastic optimization, second, by discussing optimization models that arise in large-scale problems, and third, by showing how non-convex problems arising from various nice stochastic problems are in practice (and in theory) relatively easy.

Representing and comparing probabilities with (and without) kernels

Arthur Gretton, University College London

The purpose of this tutorial is to provide an introduction to distribution embeddings and their applications, with a focus on recent tools (from 2014-2017) developed to represent and compare probabilty distributions. The first part of the talk will focus entirely on the representation of probability distributions, using both kernel methods and explicit features. The focus will be on designing kernels or features to make two distributions as distinguishable as possible. The second part of the talk will focus on more sophisticated applications of distribution representations: model criticism (using Stein's method to test against a parametric model), learning from probabilities as inputs, testing for independence and testing multi-way interaction. I might also touch on the problem of testing where the inputs are time series.

Deep Generative Models

Shakir Mohamed and Danilo Rezende, DeepMind

This tutorial will be a review of recent advances in deep generative models. Generative models have a long history at UAI and recent methods have combined the generality of probabilistic reasoning with the scalability of deep learning to develop learning algorithms that have been applied to a wide variety of problems giving state-of-the-art results in image generation, text-to-speech synthesis, and image captioning, amongst many others. Advances in deep generative models are at the forefront of deep learning research because of the promise they offer for allowing data-efficient learning, and for model-based reinforcement learning. At the end of this tutorial, audience member will have a full understanding of the latest advances in generative modelling covering three of the active types of models: Markov models, latent variable models and implicit models, and how these models can be scaled to high-dimensional data. The tutorial will expose many questions that remain in this area, and for which there remains a great deal of opportunity from members of the UAI community.

Machine learning in healthcare

Suchi Saria, Johns Hopkins University


Golden Sponsor

Golden Sponsor

Startup Sponsor

Media Sponsor