Home
From Probabilistic Programming
PROBABILISTIC-PROGRAMMING.org
This website serves as a repository of links about probabilistic programming languages, including both academic research spanning theory, algorithms, modeling, and systems, as well as implementations, evaluations, and applications. If you would like to contribute to this site, please contact Daniel Roy or David Wingate. (The site is being revamped and so we welcome suggestions.)
Contents |
The probabilistic programming approach
Probabilistic graphical models provide a formal lingua franca for modeling and a common target for efficient inference algorithms. Their introduction gave rise to an extensive body of work in machine learning, statistics, robotics, vision, biology, neuroscience, artificial intelligence (AI) and cognitive science. However, many of the most innovative and useful probabilistic models published by the AI, machine learning, and statistics community far outstrip the representational capacity of graphical models and associated inference techniques. Models are communicated using a mix of natural language, pseudo code, and mathematical formulae and solved using special purpose, one-off inference methods. Rather than precise specifications suitable for automatic inference, graphical models typically serve as coarse, high-level descriptions, eliding critical aspects such as fine-grained independence, abstraction and recursion.
PROBABILISTIC PROGRAMMING LANGUAGES aim to close this representational gap, unifying general purpose programming with probabilistic modeling; literally, users specify a probabilistic model in its entirety (e.g., by writing code that generates a sample from the joint distribution) and inference follows automatically given the specification. These languages provide the full power of modern programming languages for describing complex distributions, and can enable reuse of libraries of models, support interactive modeling and formal verification, and provide a much-needed abstraction barrier to foster generic, efficient inference in universal model classes.
We believe that the probabilistic programming language approach within AI has the potential to fundamentally change the way we understand, design, build, test and deploy probabilistic systems. This approach has seen growing interest within AI over the last 10 years, yet the endeavor builds on over 40 years of work in range of diverse fields including mathematical logic, theoretical computer science, formal methods, programming languages, as well as machine learning, computational statistics, systems biology, probabilistic AI.
Academic research
A growing body of literature studies probabilistic programming from an array of perspectives.
- Church
A collection of papers on Church, but also on computable probability theory. - BLOG
A collection of papers on BLOG. - Approximate Bayesian Computation
A distinct but related body of work on likelihood-free methods, developed originally to tackle statistical queries in population genetics. - your links here
Existing languages and their implementations
- BLOG
Bayesian logic (BLOG) is a first-order probabilistic modeling language. It is designed for making inferences about real-world objects that underlie some observed data: for instance, tracking multiple people in a video sequence, or identifying repeated mentions of people and organizations in a set of text documents. BLOG makes it (relatively) easy to represent uncertainty about the number of underlying objects and the mapping between objects and observations. - Church
Church is a universal probabilistic programming language, well suited for potentially recursive generative models (Goodman, Mansinghka, Roy, Bonawitz and Tenenbaum, 2008). Church is a derivative of the programming language Scheme with probabilistic semantics. - HANSEI
HANSEI is the the embedded domain-specific language for probabilistic programming: for writing potentially infinite discrete-distribution models and performing exact inference, importance sampling and inference of inference. - your language here
Tutorials
- Probabilistic Models of Cognition Tutorial
A tutorial in Church using examples from Cognitive Science and AI.
Workshops
- NIPS*2008 Workshop on Probabilistic Programming
- 'your workshop here
