Probabilistic Graphical Model
A probabilistic graphical model is a framework that uses a graph to describe the structure of a joint probability distribution over many random variables, with nodes standing for variables and edges for direct probabilistic dependence, so that a distribution too large to write down can be specified, reasoned about, and learned through small local pieces. It is an umbrella term: Bayesian networks, Markov random fields, factor graphs, and their many relatives are all members. What unites them is the bargain that missing edges assert conditional independence, and conditional independence is what makes high-dimensional probability tractable.
The field's standard reference is Daphne Koller and Nir Friedman's 2009 textbook Probabilistic Graphical Models: Principles and Techniques, which organizes the subject into three questions: representation, inference, and learning. Representation asks how a graph encodes a distribution, inference asks how to compute marginals or the most probable configuration given evidence, and learning asks how to estimate parameters and structure from data. This three-part decomposition is useful because the same questions recur whatever the graph type, and because algorithms developed for one model family often transfer to another. The framework grew out of the convergence of statistical physics, statistics, and artificial intelligence during the 1980s, with Judea Pearl's work on Bayesian networks as the principal AI-side contribution.
The two classical families differ in the kind of graph and in what the edges mean. Directed models, the Bayesian networks, use a directed acyclic graph and factor the joint distribution into conditional probabilities of each variable given its parents, which gives them a natural causal-generative reading. Undirected models, the Markov random fields, descend from the Ising model of ferromagnetism, studied by Ernst Ising in 1925, and define the joint as a normalized product of non-negative potential functions over cliques of the graph. The Hammersley-Clifford theorem, with Julian Besag's 1974 account, links the Markov property of such a graph to this factorized form. The normalizer, called the partition function, is a sum over all configurations and is the source of many computational headaches, since it is generally intractable to compute exactly. Chain graphs and other hybrids mix both edge types.
Factor graphs, presented in Frank Kschischang, Brendan Frey, and Hans-Andrea Loeliger's 2001 paper "Factor graphs and the sum-product algorithm," unify the families by making the factorization itself the graph. A bipartite graph connects variable nodes to factor nodes, and the sum-product algorithm passes messages between them, computing exact marginals when the graph is a tree. The same algorithm, discovered independently in different fields, appears as the forward-backward algorithm for hidden Markov models, as Pearl's belief propagation, and as decoding algorithms in error-correcting codes. When run on graphs with loops it is only approximate, yet loopy belief propagation decodes turbo and low-density parity-check codes remarkably well, a surprising link between coding theory and graphical models.
Many well-known models are graphical models under another name. Hidden Markov models and Kalman filters are chain-structured, with discrete and continuous latent states respectively. Conditional random fields, introduced by John Lafferty, Andrew McCallum, and Fernando Pereira in 2001, are undirected models of labels conditioned on inputs, long used for sequence labeling. Latent Dirichlet allocation, by David Blei, Andrew Ng, and Michael Jordan in 2003, is a directed topic model. Boltzmann machines, from Geoffrey Hinton and Terrence Sejnowski in the mid 1980s, are Markov random fields with hidden units, the ancestors of restricted Boltzmann machines used in early deep learning. Geman and Geman's 1984 paper on Gibbs sampling in image restoration bound Markov random fields to the sampling methods still used for them.
Pitfalls cluster around inference cost and interpretation. Exact inference in general graphs is NP-hard, so a model that is elegant to write may be unusable without approximation, and approximations such as loopy propagation or mean-field variational inference carry no general guarantee of accuracy. Another misconception is that a graph shows causation or that undirected edges say "correlation"; in fact the edges describe conditional dependence given the other variables, a subtler claim. The graph is also a modeling assumption, not a fact read from data, and learning its structure is much harder than fitting its parameters.
The relevance today is that graphical models supplied the language, still in use, of latent-variable modeling, whose descendants include variational autoencoders and diffusion models. Probabilistic programming systems such as Stan and PyMC let practitioners write down a generative model and rely on automatic inference, and the field's emphasis on structure, uncertainty, and explicit assumptions remains an alternative to purely end-to-end learned systems where data are scarce or decisions are high-stakes.
Read probabilistic graphical model as the idea that a graph is a compact promise about independence: by choosing which variables may interact directly, a model turns an impossible joint distribution into local factors, and Bayesian networks, Markov random fields, and factor graphs are three notations for that same promise.