bayesian statistics
Bayesian statistics is the framework of learning from data that treats unknown quantities as random variables and uses the mathematics of conditional probability to update beliefs as evidence arrives. Where the frequentist tradition views a parameter as a fixed but unknown constant to be estimated, the Bayesian tradition assigns it a prior distribution that encodes what is known before seeing data, combines that prior with the likelihood of the observed data through Bayes' theorem, and produces a posterior distribution that represents all remaining uncertainty. The result is a coherent and honest accounting of what the data have and have not told us, which is why Bayesian methods have become central to semiconductor engineering wherever data are scarce, prior physical knowledge is valuable, or a full predictive distribution rather than a single point is needed. This document develops bayesian statistics from its axioms through to the modern computational machinery, and then to the machine-learning and process applications that run in a modern fab and design house, including device model calibration, yield and reliability estimation, metrology, statistical process control, and uncertainty-aware deep learning. A Bayesian way of thinking gives an engineer a single framework for combining physics-informed priors with measurement data and for stating conclusions as probabilities, which is exactly the rigor that turning billions of near-identical devices into a dependable yield and a trustworthy performance number demands.
**Bayes' theorem is the engine of Bayesian statistics, relating the prior, the likelihood, and the posterior.** For unknown parameter $\theta$ and observed data $D$, Bayes' theorem states $P(\theta \mid D) = P(D \mid \theta) P(\theta) / P(D)$, where $P(\theta)$ is the prior distribution, $P(D \mid \theta)$ is the likelihood of the data under the parameter, and $P(D)$ is the marginal likelihood or evidence that normalizes the posterior $P(\theta \mid D)$ so that it integrates to one. The theorem, published posthumously by Thomas Bayes in 1763 and given its modern form by Pierre-Simon Laplace, turns the probability of data given a hypothesis into the probability of a hypothesis given data, which is the reversal that makes statistical learning possible. The posterior distribution is proportional to the prior times the likelihood, a concise statement that encapsulates the entire discipline: the posterior is what you believe after the data, formed by updating what you believed before with what the data say.
**The prior distribution encodes what is known about a parameter before the data arrive.** A prior is a probability distribution over the parameter that represents current belief, and it can be informative, carrying substantial prior knowledge from physics or past experiments, or weakly informative, capturing only a rough range, or uninformative, expressing near-ignorance and letting the data dominate. The choice of prior is the most philosophically distinctive and practically consequential decision in Bayesian analysis, because it determines how much the posterior is pulled toward prior belief relative to the data. An informative prior based on a validated compact-model parameter or a well-known film-thickness distribution can dramatically improve estimates from small samples, while a poorly chosen prior can bias a result even when the data are abundant. The value of the prior is precisely what attracts engineers to Bayesian methods: it is the formal place where domain knowledge enters the analysis. A prior can encode a hard physical bound, such as a threshold voltage that must be positive or a film thickness that cannot be negative, by giving zero density outside the feasible region, and it can encode a smoothness or a known range from a validated model. The sensitivity of the conclusion to the prior is something a careful analyst checks by repeating the analysis under different priors, and when the posterior barely changes across reasonable priors the data have overwhelmed the prior, while when it changes strongly the prior is doing real work. The discipline of stating a prior explicitly is itself a benefit, because it forces the assumptions into the open where they can be challenged, rather than leaving them implicit and unexamined.
**The likelihood is the probability of the observed data under a candidate parameter value, and it is where the model lives.** Given a statistical model that says how data are generated, the likelihood $L(\theta) = P(D \mid \theta)$ measures how plausible the actual observed data are for each possible parameter value, and it is the component of Bayes' theorem that is updated by the measurements. For independent observations the likelihood is a product of per-observation densities, and its logarithm is a sum that is far easier to work with numerically. The likelihood embodies the modeling assumptions, such as whether a measurement error is normal or whether defect counts follow a Poisson process, so a wrong model produces a wrong likelihood and a misleading posterior. In practice the likelihood is often the most important and most carefully justified part of a Bayesian analysis, because it is the mechanism by which the data speak. Choosing the likelihood is the same act as choosing the noise model, and getting it wrong can dominate the posterior far more than the choice of prior. For a thickness measurement the likelihood is usually taken to be normal around the true value, for a defect count it is Poisson, for a pass-fail outcome it is Bernoulli, and for a time-to-failure it is Weibull or lognormal, and each choice carries its own assumptions about how the data arise. A likelihood that is too narrow makes the posterior overconfident about parameters, while one that is too wide wastes the information in the data, so the model must be chosen to match the actual measurement error. In semiconductor metrology, where measurement error is often well understood from tool qualification, the likelihood can be specified with confidence, and that confidence flows through to the posterior.
**Conjugate priors keep the posterior in the same distributional family as the prior, giving exact closed-form updates.** A conjugate prior is one such that the posterior has the same functional form as the prior, so that updating is a simple matter of changing the prior's parameters, which makes Bayesian analysis analytically tractable for a wide class of models. The Beta prior is conjugate to the Binomial likelihood, the Gamma prior to the Poisson likelihood, and the Normal prior to the Normal likelihood, so that the posterior of a beta distribution after binomial data is another beta with updated parameters. This tractability is what made Bayesian analysis feasible long before modern computers, and it remains the fastest and most transparent way to update a model with streaming data. An engineer tracking a defect rate with a beta prior can update the posterior after every lot with a few arithmetic steps, which is the cleanest possible demonstration of sequential Bayesian learning. Because the conjugate family is closed, the mean of the beta posterior is a weighted combination of the prior mean and the sample proportion, with the weights set by how much information each carries, so the update is transparent and easily audited. For a Poisson count with a gamma prior, the posterior gamma has its shape and rate incremented by the count and the sample size, again a simple arithmetic update, and the same pattern repeats for the normal-normal pair. These closed-form updates are not merely a historical convenience; they remain the fastest and most robust way to do streaming Bayesian inference, and they are the natural starting point for any engineer learning the framework. The intuition built on a conjugate pair transfers directly to understanding why numerical methods are needed for the harder problems that lack a closed form.
**A credible interval is the Bayesian analog of a confidence interval and has the intuitive interpretation a confidence interval lacks.** A 95 percent credible interval for a parameter is a region of the posterior that contains 95 percent of the posterior probability, so one can say directly that, given the data and the prior, the parameter lies in the interval with 95 percent probability. The highest posterior density interval is the shortest such region, and the equal-tailed interval places 2.5 percent of the posterior mass in each tail, with the two agreeing for symmetric posteriors. Because the posterior is a genuine probability distribution over the parameter, the credible interval's interpretation is far more natural than the repeated-sampling interpretation of a confidence interval, which is one of the main practical attractions of the Bayesian approach. When an engineer needs to state how confident she is that a yield or a device parameter lies in a range, a credible interval expresses exactly that.
**The maximum a posteriori (MAP) estimate and the posterior mean are the two default point summaries of a posterior distribution.** The maximum a posteriori estimate is the parameter value that maximizes the posterior density, the mode of the posterior, and it reduces to the maximum likelihood estimate when the prior is uniform, while the posterior mean is the expected value of the parameter under the posterior. These two summaries coincide for symmetric, unimodal posteriors but can differ for skewed ones, and the choice between them depends on the loss function of the decision problem. The posterior mean minimizes squared-error loss, the posterior median minimizes absolute-error loss, and the mode maximizes posterior probability, so the right point summary follows from the cost of being wrong in each direction. In a process model calibrated with an informative prior, the posterior mean of each parameter is the natural point estimate, with the posterior standard deviation giving its uncertainty.
**The Bayes factor compares the evidence for two models or hypotheses by the ratio of their marginal likelihoods.** For competing hypotheses $H_1$ and $H_2$ with prior probabilities, the Bayes factor is $B_{12} = P(D \mid H_1)/P(D \mid H_2)$, the ratio of how well each model explains the data, and it is multiplied by the prior odds to obtain the posterior odds. A Bayes factor greater than one favors the first hypothesis, with values beyond about ten providing strong evidence and values beyond one hundred decisive evidence, according to the calibration popularized by Harold Jeffreys. Unlike a p-value, the Bayes factor can quantify evidence in favor of the null hypothesis and can accumulate evidence as more data arrive, rather than only ever arguing against the null. For comparing whether a new process truly improves yield or whether a model with an extra term is warranted, the Bayes factor offers a principled, symmetric measure of support. The posterior odds, which combine the Bayes factor with the prior odds, are the quantity that actually guides a decision, and they naturally update as more data accumulate, so a sequence of experiments that each favor one hypothesis will drive the odds decisively in that direction. In practice the Bayes factor must be computed by integrating the likelihood over the prior, which requires care because a diffuse prior on a parameter that matters can deflate the evidence for a model, a phenomenon sometimes called the Bartlett paradox. The practical remedy is to use sensible, weakly informative priors and to report the Bayes factor together with the sensitivity to the prior. Used this way, the Bayes factor is a far more balanced tool than a p-value for comparing the support that data lend to competing explanations.
**Bayesian model comparison and model averaging handle the choice among competing models in a unified way.** The marginal likelihood or model evidence $P(D) = \int P(D \mid \theta) P(\theta) d\theta$ summarizes how well a model explains the data after averaging over its parameters, and models with higher evidence are preferred, with a built-in penalty for complexity that is the Bayesian counterpart of Occam's razor. Bayesian model averaging goes further and weights the predictions of several models by their posterior probabilities, so that the final predictive distribution reflects model uncertainty rather than betting on a single model. This is particularly valuable when several process or device models fit the data comparably, because the averaged prediction is more robust than the prediction of any single winner. The Bayesian framework thus treats model selection and prediction as one coherent exercise rather than two separate ad hoc procedures.
**The sequential nature of Bayesian updating means the posterior of one analysis is the prior of the next.** Because the posterior is proportional to the prior times the likelihood, the posterior obtained after the first batch of data can be used unchanged as the prior before the second batch, so that the same posterior is reached whether the data are processed all at once or in sequence. This property, a consequence of the associativity of Bayesian conditioning, is the theoretical foundation of online and streaming learning, where the model is refined continuously as new measurements arrive. In a fab, a model of a defect rate or a process drift can be updated lot by lot or wafer by wafer, with each update incorporating the new data while retaining everything learned before. Sequential updating is what makes Bayesian methods naturally suited to the continuous stream of data that manufacturing generates.
**Hierarchical Bayesian models share statistical strength across related groups by placing a prior on the group-level parameters.** In a hierarchical model, the parameters of several related groups, such as the mean defect densities of different tools or the offset parameters of different wafers, are assumed to be drawn from a common distribution whose own parameters, called hyperparameters, are given a hyperprior. The model then estimates the group-level distribution from the data, which partially pools the estimates across groups, so that a group with little data borrows strength from groups with more data, and this shrinkage toward the group mean is a principled answer to the small-sample problem. Hierarchical structure is the natural Bayesian way to model nested variation in semiconductor manufacturing, where wafers nest within lots and lots nest within time. The result is more stable and more honest estimates for every level of the hierarchy.
**Markov chain Monte Carlo sampling turns the posterior into a set of samples when exact formulas are unavailable.** For any model complex enough that the posterior cannot be computed in closed form, Markov chain Monte Carlo methods generate a sequence of parameter values that converge to the posterior distribution, and averages over the samples approximate integrals under the posterior. The Metropolis-Hastings algorithm proposes a new parameter value and accepts or rejects it according to a rule that guarantees the chain's stationary distribution is the posterior, and it works with only the unnormalized posterior, which is essential because the normalizing evidence $P(D)$ is usually intractable. The samples are not independent, so diagnostics and thinning are needed, but the approach is general and makes Bayesian analysis possible for essentially any model. The development of Markov chain Monte Carlo, beginning with Metropolis and extended by Hastings, Geman and others, is the development that turned Bayesian statistics from a theory into a practical computational discipline.
**Gibbs sampling updates one parameter at a time using conditional distributions and is simple and widely used.** When the full conditional distribution of each parameter given all the others is known, Gibbs sampling draws each parameter in turn from that conditional, and the sequence of draws converges to the joint posterior. Gibbs sampling is particularly attractive for hierarchical and graphical models where the full conditionals are tractable, and it forms the core of many probabilistic programming systems. Hamiltonian Monte Carlo improves on random-walk methods by using gradient information to propose distant, efficient moves that explore the posterior with far fewer samples, and the No-U-Turn Sampler adaptively chooses its step size, giving the modern default for many problems. The availability of these efficient samplers in libraries such as Stan, PyMC, and JAX is what lets an engineer fit a Bayesian model to realistic semiconductor data.
**Variational inference turns sampling into optimization and scales Bayesian methods to very large models.** Instead of drawing samples, variational inference approximates the posterior with a simpler distribution chosen from a family, and it finds the member of that family closest to the true posterior by maximizing the evidence lower bound, which is equivalent to minimizing a divergence between the approximation and the posterior. The result is a fast, deterministic approximation that is especially valuable when the data are enormous or the model is deep, as in Bayesian deep learning, where exact sampling is impractical. Variational inference trades a small amount of approximation error for a large gain in speed and scalability, making it the workhorse of modern Bayesian machine learning. The choice of the approximating family and the optimization method determine the quality of the approximation, and mean-field and structured approximations offer different trade-offs.
**Approximate Bayesian computation sidesteps an intractable likelihood by simulating the data-generating process directly.** When the likelihood is impossible to evaluate but the model can be simulated, approximate Bayesian computation generates candidate parameters, simulates data, and retains those parameters whose simulated data are close to the observed data, thereby producing samples from an approximate posterior without ever computing the likelihood. The tolerance on the distance between simulated and observed data controls the approximation, with smaller tolerances giving more accurate posteriors at greater computational cost. Approximate Bayesian computation is valuable in semiconductor contexts where the forward model, such as a device or process simulation, is expensive but usable as a simulator, and where a tractable likelihood is unavailable. The method demonstrates how the Bayesian posterior can be recovered by simulation alone when the likelihood is the obstacle.
**Bayesian and frequentist methods answer different questions and are chosen by the needs of the problem.** The frequentist treats the parameter as fixed and asks what long-run frequencies would arise across repeated sampling, producing confidence intervals and p-values with their subtle repeated-sampling interpretation, while the Bayesian treats the parameter as random and produces a posterior that directly quantifies uncertainty about the parameter given the data and the prior. The two frameworks agree in many large-sample settings, where the posterior is often similar to the likelihood-based inference, but they diverge when prior information is strong, when the sample is small, or when a direct probability statement about a parameter is needed. Many practitioners use both: frequentist methods for routine, standardized monitoring and Bayesian methods when a prior is valuable, a full predictive distribution is needed, or a hierarchical structure must be honored. The honest analyst knows which framework produced a number and what its interpretation is, because a credible interval and a confidence interval mean different things.
**Decision theory connects the posterior to action by specifying the cost of being wrong in each direction.** In a Bayesian decision problem, the posterior distribution is combined with a loss function that assigns a cost to each possible decision and each true state, and the optimal decision minimizes the expected loss under the posterior. Different loss functions yield different optimal point estimates, with squared-error loss giving the posterior mean, absolute-error loss giving the posterior median, and zero-one loss giving the posterior mode, so the choice of summary is not arbitrary but follows from the decision's economics. Decision theory unifies estimation, testing, and prediction under a single principle, and it makes the engineering trade-offs explicit, such as the cost of shipping a marginal lot versus the cost of scrapping a good one. By framing a statistical question as a decision, the Bayesian approach forces the analyst to state what matters, which is often the most valuable step of all.
**Bayesian neural networks and their practical approximations bring uncertainty to deep learning.** A Bayesian neural network places a prior over the network weights and computes a posterior over them, producing a predictive distribution that reflects both the uncertainty in the weights and the noise in the data, but exact inference over millions of weights is intractable. Practical approximations include Monte Carlo dropout, which interprets dropout applied at test time as sampling from an approximate posterior and averages many stochastic forward passes, and deep ensembles, which train several independent networks and treat their disagreement as epistemic uncertainty. These methods turn an ordinary trained network into a calibrated predictor that knows when it is uncertain, which is essential when a model is asked to extrapolate to an unfamiliar design. In semiconductor design automation, such uncertainty-aware models can flag an out-of-distribution input for human review instead of silently producing an overconfident prediction.
**Calibration measures whether a model's stated confidence matches its observed accuracy.** A model is well calibrated when, among all predictions made with 90 percent confidence, roughly 90 percent are correct, and poorly calibrated models systematically overstate or understate their certainty. Calibration curves plot observed accuracy against claimed confidence, and the expected calibration error summarizes the deviation from the ideal diagonal, with methods such as temperature scaling adjusting a model's confidence to improve calibration. Calibration is the bridge between having uncertainty estimates and being able to trust them, because an uncertainty number that does not match reality is worse than none. In reliability and yield contexts, a well-calibrated model is the prerequisite for using its predictive interval as a release criterion, since the stated coverage must genuinely hold.
**Bayesian optimization is the application of Bayesian statistics to the efficient search for the optimum of an expensive function.** Bayesian optimization builds a probabilistic model, typically a Gaussian process, of an expensive black-box objective, and uses an acquisition function that balances exploration of uncertain regions against exploitation of promising regions to choose where to evaluate next. Each evaluation updates the surrogate model, and the sequence of evaluations converges to the optimum far faster than random search, making the method ideal for tuning process recipes and hyperparameters where each evaluation is an expensive experiment or training run. The prior over the objective and the posterior after observations give both the predicted optimum and the uncertainty about it, so the search is guided by Bayesian reasoning throughout. In semiconductor engineering, Bayesian optimization tunes etch, deposition, and implant recipes and searches hyperparameter spaces for machine-learning models.
**Bayesian methods in the fab combine physics-informed priors with measured data for calibration, yield, and control.** A compact device model can be calibrated by placing priors on its parameters based on physical expectations and updating them with measured current-voltage data, yielding parameter posteriors that capture both the best fit and its uncertainty. A defect rate or yield can be modeled with a conjugate beta or gamma prior and updated lot by lot, giving an always-current posterior for the true rate rather than a fixed point estimate. Run-to-run process control can be framed as a Bayesian state estimation that updates a belief about the process state with each measurement and chooses a recipe to correct it, and statistical process control can be augmented with Bayesian change point detection that flags when the process has likely drifted. Wherever an engineer must combine prior knowledge with sparse, noisy data, the Bayesian framework provides the principled way to do it.
**The marginal likelihood and the evidence are what let Bayesian methods score models and detect change.** The Bayes theorem, applied at every level of a model, is what makes the whole framework cohere: whether the parameter is a scalar defect rate, a vector of device parameters, or a set of network weights, the same rule of prior times likelihood over evidence produces the posterior that Bayesian statistics is built on. The evidence $P(D)$ is the probability of the observed data under a model averaged over its parameters, and its comparison across models underpins model selection and averaging, while its role as the normalizing constant of the posterior makes it central to every Bayesian computation. In a change-point model, the posterior probability that a change occurred at each time is computed by Bayesian updating, and the location with the highest posterior mass marks the likely drift, which is the basis for detecting process excursions. Model checking through posterior predictive tests compares simulated data from the posterior to the observed data to reveal whether the model adequately captures the process. These uses show that the evidence and the predictive distribution are not side effects but the operational heart of Bayesian analysis.
**The likelihood principle and the coherence of Bayesian updating give the framework a deep theoretical foundation.** The likelihood principle states that all evidence about a parameter in a set of data is contained in the likelihood function, so two experiments with proportional likelihoods carry the same evidence, a principle that Bayesian methods respect and frequentist procedures can violate. The subjective interpretation of probability, championed by de Finetti and Savage, justifies treating a parameter as a random variable with a personal degree of belief, while Cox's theorem derives the rules of probability from a few reasonable desiderata for reasoning under uncertainty. These foundations, though philosophical, have practical consequences, because they guarantee that Bayesian methods are internally consistent and that rational agents who agree on the prior and the likelihood will agree on the posterior. An engineer who internalizes this coherence can reason about evidence, and not just compute with formulas.
**Bayesian methods shine exactly where frequentist methods struggle: small samples, strong priors, and hierarchical data.** When a new process has been run only a few times, a Bayesian analysis with a physics-informed prior yields a posterior that reflects both the scarce data and the prior knowledge, while a frequentist analysis of the same few points has enormous uncertainty and little to say. When measurements are hierarchical, with wafers within lots within tools, the Bayesian hierarchical model pools information across levels in a way that a flat frequentist analysis cannot naturally express. When a decision requires a direct probability statement about a parameter or a cost-weighted choice, the Bayesian posterior and decision theory provide exactly that. The engineering value of Bayesian statistics is therefore concentrated in the hardest, most data-poor, most consequential decisions, which is precisely where semiconductors demand the most care.
| Prior | Likelihood | Posterior | Typical Semiconductor Use |
|---|---|---|---|
| Beta(α, β) | Binomial(x; n, θ) | Beta(α+x, β+n−x) | defect rate, die pass fraction |
| Gamma(α, β) | Poisson(k; θ) | Gamma(α+k, β+n) | defect count per die, yield |
| Normal(μ₀, σ₀²) | Normal(θ; μ, σ²) | Normal(…, …) | film thickness, CD, Vt mean |
| Normal(μ₀, σ₀²) | Normal(θ; μ, σ²) known | Normal(…) | device parameter calibration |
| Gamma(α, β) | Exponential(x; θ) | Gamma(α+n, β+Σx) | time-to-failure, reliability |
| Dirichlet(α) | Categorical | Dirichlet(α + counts) | wafer zone / bin proportions |
| Aspect | Bayesian | Frequentist |
|---|---|---|
| Parameter | random variable with distribution | fixed unknown constant |
| Uncertainty | posterior / credible interval | confidence interval, p-value |
| Prior | explicit, required | avoided or implicit |
| Update | posterior → prior sequentially | no sequential formalism |
| Small data + prior | principled strength borrowing | large uncertainty |
| Hierarchy | natural hierarchical models | awkward |
| Interpretation of interval | direct probability statement | repeated-sampling coverage |
| Main tools | MCMC, variational, conjugate | MLE, tests, likelihood |
**The computation of the posterior is the central practical task, and modern tools make it routine.** For conjugate models the posterior is available in closed form, for moderate models Markov chain Monte Carlo draws samples, and for very large or deep models variational inference optimizes an approximation, with each approach appropriate to a different regime of complexity. Probabilistic programming languages such as Stan, PyMC, and NumPyro automate the specification of the model and the sampling or optimization, so that an engineer writes the model and the prior and the tool returns the posterior samples and diagnostics. The convergence diagnostics, such as the potential scale reduction factor near one and an adequate effective sample size, tell the user whether the samples can be trusted. The practical consequence is that Bayesian analysis has moved from a specialized art to a standard, accessible tool in the engineer's toolkit.
```flowchart
A[Model + prior P(θ)] --> B[Observe data D]
B --> C[Write likelihood P(D|θ)]
C --> D[Form posterior ∝ prior × likelihood]
D --> E{Conjugate / tractable?}
E -->|Yes| F[Closed-form posterior]
E -->|No| G[Sampling: MCMC / HMC / NUTS]
E -->|No, huge model| H[Variational inference]
F --> I[Posterior samples / analytic]
G --> I
H --> I
I --> J[Point estimate + credible interval]
I --> K[Posterior predictive P(y* | D)]
J --> L[Decision under loss function]
K --> L
L --> M[Report uncertainty & update prior next round]
M --> B
```
**Bayesian statistics is a framework for reasoning under uncertainty, not merely a set of formulas.** It turns the intuitive act of learning from evidence into a precise, coherent procedure in which prior knowledge and data are combined through the rules of probability to yield a posterior that fully describes what is known. The same machinery that calibrates a device model from a handful of measurements also tunes a recipe through Bayesian optimization, flags a process drift with change-point detection, and gives a deep network the honesty to say when it does not know. An engineer who thinks in Bayesian terms never separates the estimate from its uncertainty, always asks what prior belief is being assumed, and always understands that a probability statement about a parameter is a statement of belief under the model. Read bayesian statistics through a coherent-learning-and-decision lens rather than a formula-memorization lens.