Workshop Overview A graphical model, or Bayesian network, encodes probabilistic relationships among variables. Techniques based on these models are becoming increasingly important in data analysis applications of many types. In areas such as foreign-language translation, microchip manufacturing, and drug discovery, the volume of data can slow progress because of the difficulty of finding causal connections or dependencies. The new Bayesian methods enable these tangled interconnections to be sorted out and produce useful tools for handling large data sets. Google is already using these techniques to find and take advantage of patterns of interconnections between Web pages, and Bill Gates has been quoted as saying that expertise in Bayesian networks is an essential part of Microsoft's competitive advantage, particularly in such areas as speech recognition. (Bayesian networks now pervade Microsoft Office.) Recently, the MIT Technology Review named Bayesian networks as one of the top ten emerging technologies. Go to the talk on An Introduction to Probabilistic Graphical Models and Their Lyapunov Functions and Algorithms for Inference and Learning (by Prof. Brendan J. Frey) Go to the talk on Graphical Models for Linear Systems, Codes and Networks (by Prof. Ralf Koetter) Go to the talk on Graphical Models, Exponential Families and Variational Inference (by Prof. Michael I. Jordan) Remark: This workshop was held on February 19, 2004 as part of the Computational Sciences Lecture Series (CSLS) at the University of Wisconsin-Madison.

An Introduction to Probabilistic Graphical Models and Their Lyapunov Functions and Algorithms for Inference and Learning By Prof. Brendan J. Frey (Probabilistic and Statistical Inference Group, Electrical and Computer Engineering, University of Toronto, Canada) Slides of talk in PDF | Video [WMV] ABSTRACT: Many problems in science and engineering require that we take into account uncertainties in the observed data and uncertainties in the model that is used to analyze the data. Probability theory (in particular, Bayes rule) provides a way to account for uncertainty, by combining the evidence provided by the data with prior knowledge about the problem. Recently, we have seen an increasing abundance of data and computational power, and this has motivated researchers to develop techniques for solving large-scale problems that require complex chains of reasoning applied to large datasets. For example, a typical problem that my group works on will have 100,000 to 1,000,000 or more unobserved random variables. In such large-scale systems, the structure of the probability model plays a crucial role and this structure can be easily represented using a graph. In this talk, I will review the definitions and properties of the main types of graphical model, and the Lyapunov functions and optimization algorithms that can be used to perform inference and learning in these models. Throughout the talk, I will use a simple example taken from the application area of computer vision, to demonstrate the concepts.

Graphical Models for Linear Systems, Codes and Networks By Prof. Ralf Koetter (Coordinated Science Laboratory and Department of Electrical Engineering, University of Illinois, Urbana-Champaign, USA) Slides of talk in PDF | Video [WMV] ABSTRACT: The use of graphical models of sytems is a well established technique to characterize a represented behavior. While these models are often given by nature in some cases it is possible to choose the underlying graphical framework. If in addition the represented behavior satisfies certain linearity requirements, surprising structural properties of the underlying graphical models can be derived. We give an overview over a developing structure theory for linear systems in graphical models and point out numerous directions for further research. Examples of applications of this theory are given that cover areas as different as coding, state space models and network information theory.

Graphical Models, Exponential Families and Variational Inference By Prof. Michael I. Jordan (Department of Computer Science, University of California Berkeley, USA) Slides of talk in PDF | Video [WMV] ABSTRACT: The formalism of probabilistic graphical models provides a unifying framework for the development of large-scale multivariate statistical models. Graphical models have become a focus of research in many applied statistical and computational fields, including bioinformatics, information theory, signal and image processing, information retrieval and machine learning. Many problems that arise in specific instances---including the key problems of computing marginals and modes of probability distributions---are best studied in the general setting. Exploiting the conjugate duality between the cumulant generating funciton and the entropy for exponential families, we develop general variational representations of the problems of computing marginals and modes. We describe how a wide variety of known computational algorithms---including mean field, sum-product and cluster variational techniques---can be understand in terms of these variational representations. We also present novel convex relaxations based on the variational framework. We present applications to problems in bioinformatics and information retrieval. [Joint work with Martin Wainwright]