RAG-Sequence is a variant of Retrieval-Augmented Generation (RAG) that retrieves a set of relevant documents for a given query and generates a complete response sequence conditioned on each retrieved document independently, then marginalizes over the document distribution at the full-sequence level to produce the final output. This approach treats each retrieved document as a complete context for generation, selecting the most probable complete response across all document-conditioned generations.

Why RAG-Sequence Matters in AI/ML: RAG-Sequence produces coherent, single-source responses that maintain narrative consistency by generating each candidate response from a single document context, then selecting the best overall response through sequence-level marginalization.

• Sequence-level marginalization — The probability of a complete output sequence y is: p(y|x) = Σ_z p(z|x) · p(y|x, z), where each candidate response is generated entirely conditioned on a single retrieved document z; the final output maximizes this marginalized probability • Narrative coherence — Since each candidate response is generated from a single document context, RAG-Sequence produces more coherent, stylistically consistent outputs compared to RAG-Token's per-token document mixing, which can create disjointed responses • Beam search implementation — For each retrieved document, a separate beam search generates top candidate sequences; then candidates across all documents are re-scored using the marginalized probability, selecting the globally best response • Factual consistency — Conditioning the entire response on a single retrieved passage reduces the risk of combining contradictory facts from different sources, maintaining logical consistency within each generated response • Retriever-generator pipeline — A DPR (Dense Passage Retrieval) bi-encoder retrieves the top-k documents, which are then individually fed to a BART generator; the final output is selected by marginalizing over retrieval scores and generation probabilities

RAG-Sequence provides retrieval-augmented generation with strong narrative coherence by generating complete responses conditioned on individual retrieved documents and selecting the best through sequence-level marginalization, making it ideal for tasks where factual consistency and stylistic coherence within the response are more important than multi-source information synthesis.