Analog AI Chip
An analog AI chip is a processor that performs the core arithmetic of neural networks — multiplying inputs by weights and summing the results — using continuous physical quantities such as electrical current rather than digital logic operating on binary numbers. Its defining idea is in-memory computing: the network's weights are stored as the conductance of memory devices, and the multiplication happens where the weight lives, so the data movement between separate memory and processor that dominates the energy cost of digital inference is largely avoided. It is a research and early-product area, and it targets inference more than training.
The computation is physics doing the multiply-accumulate (MAC). Memory devices sit at the crossings of a grid of horizontal and vertical wires called a crossbar array. Each device holds a weight as a conductance G. Input values are applied as voltages V along the rows, and by Ohm's law each device passes a current I = G × V — a multiplication. By Kirchhoff's current law the currents of all devices in a column add together on the column wire — an accumulation. A single read step therefore produces a whole matrix-vector product in parallel, which is exactly the operation that dominates fully connected layers and, in a transformer, the projection matrices. Because negative weights are needed, designs typically use a pair of devices per weight or a reference column.
Devices that can hold a tunable conductance without power are the enabling technology. Candidates include phase-change memory (PCM), where partially crystallizing a chalcogenide material sets its resistance, resistive RAM or memristors, flash cells, and magnetic or ferroelectric devices. IBM Research has published many prototypes with PCM: Ambrogio and colleagues reported in Nature in 2018 on training networks using analog memory with accuracy comparable to software on selected tasks; Joshi and colleagues showed in 2020 that carefully programmed PCM could run deep network inference with accuracy close to digital baselines; and Le Gallo and colleagues described a 64-core mixed-signal PCM chip for inference in a 2023 Nature Electronics paper. These are laboratory demonstrations on specific networks, not evidence that the approach is ready to replace GPUs.
Analog computing pays for its efficiency with imprecision. Programming a device lands on its target conductance only approximately; read operations add thermal and shot noise; and PCM in particular shows conductance drift, where resistance slowly changes after programming following a roughly power-law pattern. Wire resistance, device-to-device variation and nonlinear device response distort the sums, and an eight-bit-class result requires careful analog design. Common mitigations include hardware-aware training that injects noise during training so the network learns to tolerate it, periodic recalibration or drift compensation at the output, and writing weights more than once and verifying them.
Analog arrays do only part of the job, and overhead can erode the gain. Converters between digital and analog domains (DACs and ADCs) consume substantial area and energy; nonlinear functions, normalization, softmax and the dynamic attention between tokens are awkward in analog and are usually done in digital logic on the same chip. Writing to devices is slow and wears them out, which is why analog arrays suit inference with fixed weights better than training or fast-changing data such as a growing KV cache. Evaluation must be end-to-end: a headline figure for the array alone, in operations per watt, may not survive once converters, data routing and the digital parts are included.
Misconceptions are common because "analog" sounds like a revival of old analog computers. It does not mean the whole chip is analog, nor that results are random: modern designs keep a digital interface and treat the analog core as a noisy but characterizable accelerator. Nor does it mean lower accuracy is acceptable in general — neural networks tolerate some noise, but tolerance has to be measured per model and per task. Digital in-memory designs and neuromorphic spiking chips pursue related goals through different means, and the winner for any given workload is an open engineering question rather than a settled one.
Read analog AI chip as a trade of exactness for locality: let conductances and currents perform the multiply-accumulate where the weights are stored, then spend careful engineering on noise, drift and converters to keep the answer good enough.