AI Video Generation
AI video generation is the industry and set of products built on models that synthesize moving images, usually from a text prompt, a still image or an existing clip, producing a few seconds to about a minute of footage per request. The underlying technique, text-to-video, has its own entry; this entry concerns the products, the companies and the practical question of how good the results actually are, which is a moving target. A convincing short shot is now cheap to obtain, while a coherent scene that holds together across many shots remains difficult, and the gap between those two facts defines the current state of the field.
The product timeline compresses into about two years. Meta announced Make-A-Video in September 2022 and Google announced Imagen Video the next month, both as research demonstrations with clips of a few seconds. Runway, a company with roots in creative tooling, released Gen-1 in early 2023 for transforming existing video and Gen-2 in mid-2023 as a public text-to-video and image-to-video service, and Stability AI published Stable Video Diffusion as open weights in November 2023. OpenAI announced Sora on 15 February 2024 with sample clips of up to a minute that were far more coherent than earlier public examples, though it was not released to the general public then; a faster version, Sora Turbo, became available to ChatGPT subscribers in December 2024. Google presented Veo at its I/O conference in May 2024, followed by Veo 2 in December 2024, and Chinese companies, including Kuaishou with Kling from June 2024, released competitive services. Later versions added higher resolution, longer clips and synchronized audio.
The common recipe is diffusion over compressed video, with a transformer in place of the older convolutional backbone. OpenAI's technical report accompanying Sora, titled "Video generation models as world simulators," described compressing video into a latent space and cutting it into spacetime patches, which act as tokens for a diffusion transformer, so one model can be trained on clips of varied resolution, aspect ratio and length. Attention across patches in both space and time is what keeps an object consistent from one frame to the next, and it is also why compute cost grows quickly with clip length and resolution. Earlier systems often generated sparse keyframes and filled in the gaps, or added temporal layers to image diffusion models, which tended to produce more flicker and drifting detail.
Quality has improved dramatically, but the limits are specific and persistent. Faces and textures are often convincing, and camera motion and lighting can look cinematic, yet the models are not physics engines: objects may merge, appear or vanish, hands and gait go wrong, a glass may not shatter correctly, and text on signs frequently scrambles. Characters and settings are hard to keep identical across separate generations, which makes narrative work hard without tricks like reference images, image-to-video chaining and manual compositing. Control is the other bottleneck, since a text prompt is a blunt instrument for specifying exact blocking, timing or camera path, and generation is expensive enough that creators typically run many attempts and keep the best. The viral early example of a person eating spaghetti, generated by a 2023 open model, became a benchmark meme precisely because progress between versions was easy to see.
The industry is developing along two lines: professional production tools and mass-market creation. Runway worked with studios and in 2024 announced a partnership with Lionsgate, while directors and agencies have used generated footage for storyboards, previsualization, effects plates and advertising spots; consumer apps focus on short social clips and image animation. Concerns include the sourcing of training video, the effect on stunt, stock footage and visual effects work, and misuse, where realistic synthetic video can be used for disinformation or impersonation. Companies typically add visible watermarks and metadata credentials such as C2PA to mitigate this, though both can be removed. The 2023 Hollywood union agreements addressed digital replicas and AI more broadly, and regulations on labeling synthetic media are emerging in several jurisdictions.
A frequent misconception is that these models understand a scene physically, or that a one-minute demo reflects typical output. Showcase clips are selected from many attempts, and failures are less often shown. Sound, editing, continuity and story still require conventional skills and tools, and the most reliable use today is short, stylized or establishing shots where small errors are tolerable.
Read AI video generation as a fast-maturing production tool with a visible ceiling: diffusion over spacetime patches made short clips convincing, but physical consistency, character continuity, control and cost still decide whether a given shot can be used.