Bayesian Network
A Bayesian network is a probabilistic model that represents a set of random variables and their conditional dependencies as a directed acyclic graph, in which each node holds a conditional probability table giving the variable's distribution given its parents, so that a complex joint distribution is factored into small, local, easily specified pieces. The graph encodes which variables directly influence which, and the numbers quantify how strongly. Once built, the network answers queries: given what has been observed, what is the updated probability of everything else?
The term was coined by Judea Pearl in 1985, in the paper "Bayesian networks: a model of self-activated memory for evidential reasoning," and the framework was developed fully in his 1988 book Probabilistic Reasoning in Intelligent Systems. Pearl's motivation was to give artificial intelligence a principled way to handle uncertainty at a time when expert systems were using ad hoc certainty factors. He named the model for Thomas Bayes because the core operation is updating beliefs by Bayes' rule, and because he stressed the subjective, evidence-driven view of probability. His earlier work on belief propagation in trees, around 1982, supplied the first efficient inference algorithm, and he received the ACM Turing Award in 2011 for fundamental contributions to probabilistic and causal reasoning in AI.
The central mathematical fact is the factorization: the joint probability of all variables equals the product, over every node, of its probability given its parents. For a diagnostic network with a disease node pointing to two symptom nodes, the joint distribution is the prior on the disease times the chance of each symptom given the disease, which needs far fewer numbers than a full joint table, whose size doubles with each added binary variable. This compactness holds because the graph asserts conditional independencies: each variable is independent of its non-descendants given its parents. The naive Bayes classifier is the simplest special case, a single parent with conditionally independent children, and hidden Markov models and Kalman filters are chain-structured relatives.
D-separation, a criterion formalized by Pearl and by Thomas Verma in 1988, reads independence directly off the graph without any arithmetic. Information flows along a path between two variables unless the path is blocked. A chain or a fork is blocked by observing its middle node, but a collider, a node with two arrowheads meeting at it, is the reverse: it blocks the path until the collider or one of its descendants is observed, at which point it opens. This opening is the "explaining away" in the figure, where learning of an earthquake lowers belief in a burglary, and it is the source of selection-bias and Berkson-style paradoxes. Two graphs that share the same d-separation structure are Markov equivalent and cannot be distinguished from observational data alone.
Inference and learning are the practical difficulties. Exact inference in general networks is NP-hard, as Gregory Cooper showed in 1990, although the junction tree algorithm of Steffen Lauritzen and David Spiegelhalter in 1988 is efficient when the graph has small treewidth. Large networks rely on approximations: sampling methods, loopy belief propagation, and variational inference. Parameters can be learned from data with the EM algorithm when variables are missing, and structure can be learned too, though this is far harder. Historic applications include medical diagnosis systems such as Pathfinder, developed by David Heckerman and colleagues, probabilistic help in Microsoft's software from Heckerman and Eric Horvitz's group, fault diagnosis, and early spam filters.
The two most common misconceptions concern the name and the arrows. First, a Bayesian network is not inherently Bayesian in the statistical sense; it is a model of conditional independence and can be fitted by frequentist maximum likelihood. Second, the arrows are not automatically causal: a network with arrows pointing from symptoms to disease can encode the same distribution as one pointing the other way. Pearl's later work on structural causal models and the do-calculus, set out in his 2000 book Causality, is what adds interventions and counterfactuals on top of the graph, and it is why the representation remains central to causal inference today.
Read a Bayesian network as a map of who can tell you something about whom: the arrows factor a huge joint distribution into local tables, d-separation says exactly when evidence on one variable becomes irrelevant to another, and only with extra causal assumptions do the arrows mean more than that.