Probabilistic Graphical Models

A Concise Tutorial

Jacqueline Maasch1, Willie Neiswanger2, Stefano Ermon3, Volodymyr Kuleshov1
1Cornell Tech, 2University of Southern California, 3Stanford University

About this tutorial

Probabilistic graphical modeling is a branch of machine learning that uses probability distributions to describe the world, make predictions, and support decision-making under uncertainty. Underlying this modeling framework is an elegant body of theory that bridges two mathematical traditions: probability and graph theory. This framework provides compact yet expressive representations of joint probability distributions, yielding powerful generative models for probabilistic reasoning.

This tutorial provides a concise introduction to the formalisms, methods, and applications of this modeling framework. After a review of basic probability and graph theory, we explore three dominant themes:

  • The representation of multivariate distributions as graphical models.
  • Algorithms for learning model parameters and graphical structures from data.
  • Algorithms for inference, both exact and approximate.


Note: This tutorial is currently undergoing peer review. We welcome readers to submit GitHub issues with suggested edits, comments, or critiques.

Authors

Jacqueline Maasch

PhD Candidate
Cornell Tech


Jacqueline Maasch is interested in AI reasoning and the theory, methods, and applications of causal graphical modeling.

 

Willie Neiswanger

Assistant Professor
University of Southern California


Willie Neiswanger works at the intersection of machine learning, decision making, generative AI, and AI-for-science.

 

Stefano Ermon

Associate Professor
Stanford University


Stefano Ermon develops principled generative AI methods motivated by concrete real-world applications.

 

Volodymyr Kuleshov

Assistant Professor
Cornell Tech


Volodymyr Kuleshov focuses on machine learning and its applications in scientific discovery, health, and sustainability.

 

What's inside

Representation
✦

How to specify a model.

General techniques for parameterizing multivariate probability distributions and representing them in the intuitive visual language of graphs.

 

A Markov random field and its factors.

Inference
✦

How to ask the model questions.

Inference algorithms for answering useful questions, such as: What is the most probable explanation for a given hypothesis, and how certain are we?

 

Message passing.

Learning
✦

How to fit a model to real-world data.

Methods for learning directed, undirected, and latent variable models, with an emphasis on structure learning and parameter learning.

 

A weighted mixture of exponential family distributions.

Download the full tutorial

This 200-page tutorial reviews the theory and methods of representation, learning, and inference in probabilistic graphical modeling. As an accompaniment to this tutorial, we provide links to exceptional external resources that provide additional depth.

   

Download tutorial chapters

Chapter 1: Introduction.

In this brief introduction, we provide a high-level overview of what to expect from this tutorial.

Chapter 2: Preliminaries.

Probability Set theory Random variables Graph theory

This chapter covers the basics of probability theory and graph theory, which provide the mathematical foundations of probablistic graphical modeling.

   

Chapter 3: Representation.

Markov random fields Bayesian networks Factor graphs Conditional random fields

In this chapter, we focus on general techniques for parameterizing probability distributions with relatively few parameters. We explore how the resulting models can be elegantly described as directed and undirected graphs.

   

Chapter 4: Exact Inference.

Variable elimination Belief propagation in trees Junction tree algorithm Exact MAP inference

In this chapter, we focus on exact probabilistic inference in graphical models. Though exact inference is NP-hard in the general case, tractable solutions can be obtained for certain kinds of problems. As illustrated throughout this chapter, the tractability of an inference problem depends heavily on the structure of the graph that describes the probability of interest.

   

Chapter 5: Approximate Inference.

Sampling-based methods Variational methods Approximate MAP inference

This chapter introduces the two main families of approximate inference algorithms: (1) sampling methods, which produce answers by repeatedly generating random numbers from a distribution of interest, and (2) variational methods, which formulate inference as an optimization problem.

   

Chapter 6: Learning.

Learning theory Learning in directed models Learning in undirected models Learning in latent variable models Bayesian learning

This chapter introduces methods for fitting predictive models on real data. We highlight two main tasks: (1) structure learning, where we want to infer variable dependencies in the graphical model, and (2) parameter learning, where the graph structure is known and we want to estimate the factors.

   

Chapter 7: Discussion – The Variational Autoencoder.

Deep generative latent variable models Auto-encoding variational bayes The variational autoencoder

In this concluding discussion, we present a highly influential deep probabilistic model: the variational autoencoder (VAE). Using the VAE as a case study, we draw connections among ideas from throughout this tutorial and demonstrate how these ideas are useful in machine learning research.

   

How to cite this work

Please cite our work using the following BibTeX.


    @misc{maasch2025pgm,
        title={Probabilistic Graphical Models: A Concise Tutorial}, 
        author={Jacqueline Maasch and Willie Neiswanger and Stefano Ermon and Volodymyr Kuleshov},
        year={2025},
        eprint={2507.17116},
        archivePrefix={arXiv},
        primaryClass={cs.LG},
        url={https://arxiv.org/abs/2507.17116}, 
    }