Efficient global point cloud alignment using Bayesian nonparametric mixtures J. Straub, T. Campbell, J. P. How, and J. W. Fisher III IEEE Conference on Computer Vision and Pattern Recognition, 2017

Streaming, distributed variational inference for Bayesian nonparametrics T. Campbell, J. Straub, J. W. Fisher III, and J. P. How Advances in Neural Information Processing Systems, 2015

Small-variance nonparametric clustering on the hypersphere J. Straub, T. Campbell, J. P. How, and J. W. Fisher III IEEE Conference on Computer Vision and Pattern Recognition, 2015

Dynamic clustering via asymptotics of the dependent Dirichlet process mixture T. Campbell, M. Liu, B. Kulis, J. P. How, and L. Carin Advances in Neural Information Processing Systems, 2013

Simultaneous clustering on representation expansion for learning multimodel MDPs T. Campbell, R. H. Klein, A. Geramifard, and J. P. How Reinforcement Learning and Decision Making, 2013

Multiagent planning with Bayesian nonparametric asymptotics Master's thesis Massachusetts Institute of Technology, 2013

Automated Bayesian Approximation

One of the most important recent developments in the Bayesian paradigm has been the shift towards automation:
rather than having to develop, code, and tune model-specific algorithms, practitioners now have ``black-box'' implementations
that require only a basic specification of the model as inputs. This level of automation enables experts and nonexperts alike
to use more sophisticated models, facilitates faster exploratory modeling and analysis, and helps ensure experimental reproducibility.
My research leverages the redundancy of data in large datasets to obtain a small, weighted subset
of the data (called a Bayesian coreset) that can be used in place of the full dataset in a standard posterior inference algorithm.
Coresets not only provide scalable batch inference, but also are naturally suited to streaming and parallel data modalities,
and have theoretical guarantees on the worst-case deviation of the coreset log-likelihood from that of the full dataset.

Categories:
theory algorithms applications modeling

Completely random measures (CRMs)—and their normalizations (NCRMs)—are a rich source of Bayesian nonparametric priors
such as the beta, gamma, and Dirichlet processes. However, tractable inference with (N)CRM-based priors typically requires
a finite approximation of the infinite random measure. My research provides a standardized, black-box methodology for the development of
finite approximations of (N)CRMs through truncation of sequential representations. This work has led to the characterization
of two major classes of sequential (N)CRM representation
that can be used for simulation and inference—series representations
and superposition representations—along with generalized truncation error and computational
complexity analyses for each class.

Categories:
theory algorithms applications modeling

Bayesian nonparametrics provides a wealth of models that capture the latent structure of
streaming combinatorial data. In this context, exchangeability—roughly, that the order of observations is
irrelevant for inference—is a useful assumption that simplifies inference significantly.
My research in this area focuses on the consequences of this and similar assumptions for Bayesian modeling.
For example, my recent work on exchangeability in combinatorial trait allocations—a class of models that generalizes
clustering, feature allocations, topic models, networks, and more—has characterized the full set of exchangeable trait allocations,
developed and characterized a subset particularly amenable to MCMC and variational inference algorithms, and established the first
direct connection between related theory for clustering, feature allocations, and networks.
I have used this general theory to characterize the distribution of all edge-exchangeable networks,
a model recently developed by myself and collaborators for realistic sparse graph sequences.

Categories:
theory algorithms applications modeling

Keywords: exchangeability, paintbox, probability function, frequency model, sequence, trait allocation, cluster, feature, combinatorial

One of the major challenges of streaming inference is that algorithms can only examine data
at most a small number of times before it must be discarded. My research has provided a number
of different techniques for handling this problem. In both parametric and nonparametric unsupervised models,
I have developed a technique for merging variational minibatch posteriors while correctly accounting
for component mismatch. The method is streaming, distributed, asynchronous, learning-rate-free,
and truncation-free. For clustering temporally evolving data, I have developed a k-means-like algorithm
based on a small-variance analysis of a nonparametric mixture model with Markov time dependence.
Bayesian coresets—small, weighted data subsets built as a preprocessing step—also naturally admit
streaming and parallel construction.

Categories:
theory algorithms applications modeling

Inference for Bayesian nonparametric clustering models is often not computationally feasible when dealing with high-rate data streams. Using small-variance analysis—in which maximum a-posteriori inference is converted to a hard assignment problem by considering the limit of the joint probability as component likelihood variances tend to 0—my research has developed k-means-like cost functions and corresponding minimization algorithms for fast clustering. These algorithms possess much of the flexibility of original Bayesian nonparametric models upon which they are based, but also have the scalability of k-means-like algorithms.

In order to get reliable and robust solutions to optimization problems, we need to account
for uncertainty in problem data. Bayesian nonparametric mixture models provide a flexible
framework for capturing the distribution of uncertainty as a weighted sum of relatively simple distributions.
My work in this area has taken advantage of this to formulate a flexible class of uncertainty sets for
robust optimization based on a union of ellipsoids derived from multivariate Gaussian mixture components,
and to formulate an alignment objective for 3D point cloud data from robotic perception based on multivariate
Gaussian convolution.