---

Tutorial 1 (8am-9.50am)
Copulas in Machine Learning

Gal Elidan

Abstract:

From high-throughput biology to astronomy to financial markets, a wide variety of complex high-dimensional domains are inherently continuous.  The statistical copula framework is a powerful mechanism for constructing multivariate real-valued distributions. The essence of the framework is that it allows us to separate the choice of the univariate marginals from that of the dependency structure, as captured by the copula function. This provides great modeling flexibility that often leads to substantial quantitative and qualitative advantages. Indeed, there has been a dramatic growth of academic and practical interest in copulas in recent years, with applications ranging from mainstream economics to hydrologic flood analysis. Copulas have even been famously accused of "bringing the world financial system to its knees" (Wired Magazine, 2009). Yet, the study and use of copulas for high dimensional data is still in its infancy.

While studied in statistics for many years, it is somewhat remarkable that the general purpose "distribution generating" framework of copulas was only recently noticed by machine learning researchers in general and the probabilistic graphical models community in particular. Accordingly, the first part of the tutorial aims to draw the attention of the community to this important framework and will cover: (i) a motivation and introduction to copulas; (ii) copulas as flexible multivariate models and dependence functions; (iii) copula models: examples, properties, advantages, visualization; (iv) inference for copula models; (v) copula constructions. The second part of the tutorial will provide a partial survey of recent copula-based works in machine learning as a teaser for further research.

Biographical details

Gal Elidan is a faculty member of the Statistics Department at the Hebrew University of Jerusalem. He received his Ph.D. from the Hebrew University in 2005 under the supervision of Prof. Nir Friedman. Prior to the Hebrew University, he was a postdoctoral scholar in Prof. Daphne Koller's machine learning group in Stanford University. His research interests include representation, inference and in particular structure learning of probabilistic graphical models and their application to bioinformatics, machine vision and medical diagnosis. His most recent works focus on coping with complex, non-Gaussian and multimodal domains via copula-based representations.

---

Tutorial 2 (10am-11.50am)
Determinantal Point Processes

Alex Kulesza and Ben Taskar

Abstract:

Many real-world problems involve negative interactions; we might want search results to be diverse, sentences in a summary to cover distinct aspects of the subject, or objects in an image to occupy different regions of space.  However, traditional structured probabilistic models tend deal poorly with these kinds of situations; Markov random fields, for example, become intractable even to approximate.

In this tutorial we will define and describe determinantal point processes (DPPs), which behave in a complementary fashion: while they cannot encode positive interactions, they define expressive models of negative correlations that come with surprising and elegant algorithms for many types of inference, including normalization, conditioning, marginalization, and sampling.  While DPPs have been studied by mathematicians for over 35 years and play an important role in random matrix theory, we will show how they can also be used as models for real-world data.

Our goal in this tutorial is to translate the technically dense mathematical literature and provide the audience with a comprehensible, intuitive, and practical introduction to DPPs. We will also summarize some of the ways in which recent work in machine learning has made use of DPPs for modeling real data, including learning, and discuss connections to other topics of interest to the community like spectral clustering, compressive sensing, and submodular functions.

Biographical details:

Alex Kulesza is a PhD student at the University of Pennsylvania's Department of Computer and Information Science, advised by Fernando Pereira and Ben Taskar. His primary research interests are in machine learning algorithms and theory, particularly structured models.  He has worked on applications in natural language processing, computer vision, and computational finance.

Ben Taskar received his bachelor's and doctoral degree in Computer Science from Stanford University. After a postdoc at the University of California at Berkeley, he joined the faculty at the University of Pennsylvania Computer and Information Science Department in 2007, where he currently co-directs PRiML: Penn Research in Machine Learning. His research interests include machine learning, natural language processing and computer vision. He has been awarded the Sloan Research Fellowship, the NSF CAREER Award, and selected for the Young Investigator Program by the Office of Naval Research and the DARPA Computer Science Study Group. His work on structured prediction has received best paper awards at NIPS and EMNLP conferences.  He previously presented tutorials at NIPS 2007, UAI 2005 and ACL 2005.

---

Tutorial 3 (1pm-2.50pm)
Graphical Models for Causal Inference

Karthika Mohan and Judea Pearl

Abstract:

Recent years have witnessed rapid developments in the field of Causality. This tutorial focuses on the role of graphical models in Causal Inferencing and aims to apprise the participants of the fundamental ideas as well as the latest developments in Causality. We shall describe the use of graphical models, mostly Directed Acyclic Graphs (DAGs), as (i) a formal language for expressing cause-effect relationships and (ii) a mathematical tool for predicting the effect of actions/policies. We will demonstrate the primacy of causal over probabilistic models and cover in depth the conditions that insure identifiability of causal effects in semi-Markovian models. Finally we shall discuss cutting-edge algorithms for finding minimal separators and finding Markov equivalence. Topics covered would include interventions, identifiability, counterfactuals and maximal ancestral graphs among others.

Biographical details:

Karthika Mohan is a PhD student in Computer Science at the University of California, Los Angeles (UCLA). She is advised by Prof. Judea Pearl and is a member of the Cognitive Systems Laboratory at UCLA. Her areas of interest include causal inference, probabilistic reasoning, graphical models and transfer learning. Prior to joining UCLA she was a graduate student at the International Institute of Information Technology - Hyderabad (IIIT-H), India, where she primarily worked on Speech-to-speech translation systems and Indic Language OCRs.

Judea Pearl is a professor of computer science and statistics at University of California, Los Angeles, where he directs the Cognitive Systems Laboratory and conducts research in artificial intelligence, human reasoning and philosophy of science. He authored three books including Causality (Cambridge University Press, 2000, 2009) which pioneered many developments in causal reasoning. 

---

Tutorial 4 (3pm-4.50pm)
Continuous-Time Markov Models

Christian Shelton and Gianfranco Ciardo
Abstract:

Many real-world systems evolve in continuous-time.  Events are not regulated by a global clock, but rather proceed asynchronously.  Although theory, algorithms, and applications of discrete-time Markov models are
relatively wide-spread in artificial intelligence, their counterparts in continuous-time have received much less attention.

This tutorial will introduce continuous-time Markov processes.  The first part will provide the mathematical background.  We will discuss their parameterization, semantics, and basic learning and inference procedures. We will also discuss their relative advantages compared with discrete-time Markov processes.

The second section will cover decision-diagram-based compact representations of continuous-time processes.  These representations have grown out of the queueing theory and verification literatures. We will present factored representations based on Kronecker algebra, matrix diagrams, and edge-valued decision diagrams, paying particular attention to their efficient algorithmic manipulation and use.

The third section will cover continuous-time Bayesian networks, a representation of continuous-time processes analogous to dynamic Bayesian networks for discrete time.  We will present their representation, parameter estimation, and inference, focusing on how such calculations can be more efficient than in the discrete-time case.

Biographical details:

Christian Shelton is an Associate Professor of Computer Science and Engineering at the University of California, Riverside.  His research interests are in machine learning, particularly dynamic systems. He received his BS from Stanford and his PhD from MIT.  He returned to Stanford for his post-doc work.  He has been a faculty member at UC Riverside since 2003.  He was the managing editor of JMLR from 2003 through 2008, currently serves on the Editorial Board of JAIR, and is the lead programmer for CTBN-RLE (http://rlair.cs.ucr.edu/ctbnrle/), a library for continuous-time Bayesian network representations and algorithms.

Gianfranco Ciardo is a Professor in the Department of Computer Science and Engineering at the University of California, Riverside.  Previously, he has been on the faculty at the College of William and Mary, Williamsburg, Virginia, a Visiting Professor at the University of Torino, Italy, and at the Technical University of Berlin, Germany, and has held research positions at HP Labs (Palo Alto, California), ICASE (NASA Langley Research Center, Hampton, Virginia), Software Productivity Consortium (Herndon, Virginia), and CSELT (Torino, Italy).  He received a Laurea from the University of Torino, Italy, and a PhD from Duke University. He has been on the editorial board of IEEE Transactions on Software Engineering and is on the editorial board of Transactions on Petri Nets and Other Models of Concurrency.  He was keynote speaker at PNPM'01, ATPN'04, EPEW/WS-FM'05, and PDMC 2009.  He is interested in algorithms and tools for logic and stochastic analysis of discrete-state models, symbolic model checking, performance and reliability evaluation of complex hardware/software systems, Petri nets, and Markov +models.