Hidden Markov Model
A hidden Markov model is a probabilistic model of a sequence in which an unobserved, or hidden, state evolves according to a Markov chain and each state emits an observable symbol according to its own distribution, so that what can be seen is a noisy, indirect view of a process whose true state must be inferred. Two assumptions define it: the next hidden state depends only on the current one, and each observation depends only on the hidden state that produced it. These simplifications make exact inference cheap, and they made the model the workhorse of sequence analysis for decades.
The mathematics grew out of work by Leonard Baum and colleagues at the Institute for Defense Analyses in the late 1960s, and its three canonical problems were made famous by Lawrence Rabiner's 1989 tutorial in the Proceedings of the IEEE. The evaluation problem asks how likely an observation sequence is under a model, and the forward algorithm solves it in time proportional to the sequence length times the square of the number of states, instead of the exponential cost of summing over all state paths. The decoding problem asks for the single most likely hidden state sequence, answered by the Viterbi algorithm, which Andrew Viterbi published in 1967 for decoding convolutional error-correcting codes. The learning problem asks for the parameters that best explain the data, and the Baum-Welch algorithm, from Baum and coauthors around 1970, solves it iteratively. It is an instance of expectation maximization, a framework Dempster, Laird, and Rubin generalized in 1977.
Speech recognition is the application that made hidden Markov models famous. In the 1970s James Baker at Carnegie Mellon, whose DRAGON system used the approach, and Frederick Jelinek's group at IBM independently showed that treating speech as a noisy channel decoded with an HMM beat knowledge-based rule systems. Phonemes were modeled as small left-to-right state chains whose emissions were acoustic feature vectors, commonly modeled by Gaussian mixtures, and the chains were concatenated into words and scored against a language model. Systems such as CMU's Sphinx, from Kai-Fu Lee's 1988 work, and the Cambridge HTK toolkit carried the GMM-HMM recipe through the 1990s and 2000s. The same machinery also drove part-of-speech tagging, handwriting recognition, and bioinformatics: profile HMMs, introduced by Anders Krogh, David Haussler and colleagues in 1994, power protein-family searches in tools such as HMMER, and GENSCAN, from Chris Burge and Samuel Karlin in 1997, used a generalized HMM for gene finding.
Neural networks displaced the HMM in speech not by discarding it at first but by replacing its emission model. The 2012 paper "Deep neural networks for acoustic modeling in speech recognition," by Geoffrey Hinton and colleagues from several labs, showed that deep networks predicting HMM states outperformed Gaussian mixtures, creating the hybrid DNN-HMM era. Soon after, end-to-end approaches dropped the explicit HMM: Alex Graves's connectionist temporal classification in 2006 and the later recurrent-transducer and attention-based sequence-to-sequence models learn the alignment directly, and large transformer systems such as OpenAI's Whisper in 2022 recognize speech with no HMM at all.
The contrast with neural sequence models clarifies what an HMM is good and bad at. An HMM with N states can remember only which of N states it is in, about log2 N bits of history, whereas a recurrent network or transformer carries a high-dimensional distributed representation and can model long-range dependencies. In exchange, an HMM has exact, efficient inference, a transparent state structure, trainable behavior with small datasets, and calibrated probabilities. The Markov assumption is also the pitfall: durations of states are implicitly geometric, so realistic phoneme or gene lengths are modeled poorly unless the model is extended, as in hidden semi-Markov models.
Common misconceptions concern the word hidden and the training procedure. Hidden does not mean secret or meaningful: the states are latent variables whose semantics, if any, must be assigned afterward, and unsupervised Baum-Welch often finds states that fit the data without matching the categories a human intended. Baum-Welch also converges only to a local optimum, so initialization matters. Today the HMM survives in compact embedded systems, in bioinformatics, in finance regime modeling, and as a teaching case for all of probabilistic inference, being the simplest chain-structured probabilistic graphical model and sharing its forward-backward computation with the sum-product algorithm.
Read hidden Markov model as a small, exactly solvable machine for guessing what is really going on behind a noisy sequence: a Markov chain of hidden states plus per-state emissions, three clean algorithms for evaluating, decoding, and learning, and a history in which neural networks first replaced its emission model and then the model itself.