audio inpainting

Audio inpainting fills in missing, corrupted, or intentionally removed portions of audio signals with plausible content that sounds natural and seamlessly blends with surrounding audio, analogous to image inpainting for visual data. Audio inpainting addresses scenarios where audio data is degraded or incomplete: packet loss in VoIP and streaming (network dropouts causing gaps), clipping repair (reconstructing audio peaks that exceeded recording limits), noise/artifact removal (replacing corrupted segments with clean reconstructions), intentional redaction filling (generating plausible audio to replace bleeped or censored portions for natural listening flow), and historical recording restoration (filling in damaged portions of archival audio). Technical approaches include: signal processing methods (linear prediction, autoregressive modeling — extrapolating from surrounding audio using statistical properties of the signal), dictionary-based methods (sparse representation using overcomplete dictionaries — representing the missing segment as a sparse combination of learned audio atoms), deep learning methods (neural networks trained to predict missing audio given context — using architectures like WaveNet, temporal convolutional networks, or U-Nets operating on spectrograms), and diffusion-based methods (applying denoising diffusion models conditioned on the known surrounding audio — current state-of-the-art for perceptual quality). The difficulty varies significantly with gap length: short gaps (under 20ms) are relatively easy to fill using interpolation, medium gaps (20-100ms) require more sophisticated statistical modeling, and long gaps (over 100ms — corresponding to phonemes or notes) require semantic understanding of the audio content to generate plausible fills. For music, the model must maintain rhythm, harmony, and timbral consistency. For speech, it must generate phonetically plausible content that maintains the speaker's voice characteristics and utterance prosody. Evaluation uses both objective metrics (signal-to-noise ratio, PESQ for speech quality) and subjective listening tests.