Structured Representations are latent state encodings that explicitly organize information into compositional data structures — graphs, sets, trees, or relational tables — rather than compressing everything into flat, unstructured vectors — enabling neural networks to capture the inherent relational, hierarchical, and compositional structure of the data domain, supporting systematic generalization to novel combinations that flat representations fundamentally cannot achieve.

What Are Structured Representations?

• Definition: A structured representation is any internal neural network state that maintains explicit organizational structure beyond a single fixed-dimensional vector. This includes graph representations (nodes connected by typed edges), set representations (unordered collections of entity vectors), tree representations (hierarchical parent-child structures), and relational representations (entities linked by named relations).
• Contrast with Flat Vectors: A standard neural network encodes a scene with 5 objects as a single 1024-dimensional vector — all object identities, attributes, and relationships are compressed and entangled. A structured representation encodes the same scene as a set of 5 node vectors plus edge connections between them — preserving the discrete entity structure and enabling independent manipulation of each object.
• Inductive Bias: Choosing a structured representation format is an architectural inductive bias statement — a graph representation says "the world consists of entities with pairwise relationships," a tree representation says "the world has hierarchical organization," and a set representation says "the world contains unordered entities with independent attributes."

Why Structured Representations Matter

• Variable Cardinality: Flat vectors have fixed dimensionality — they cannot naturally handle scenes with varying numbers of objects. Structured sets and graphs naturally accommodate variable numbers of entities by adding or removing nodes, enabling generalization from "3 objects" training to "10 objects" testing without architectural changes.
• Systematic Generalization: The critical failure mode of flat representations is the inability to systematically generalize to novel combinations. A model trained on "red circle" and "blue square" as flat vectors may not understand "red square" because the attribute-object binding is implicit. Structured representations with separate object and attribute nodes generalize systematically because composition is explicit.
• Relational Reasoning: Answering questions about relationships ("Which object is between A and C?") requires explicit relational structure that flat vectors cannot reliably provide. Graph representations with typed edges naturally support multi-hop relational reasoning through message passing.
• Causal Inference: Causal reasoning requires an explicit structural causal model — a directed graph where edges represent causal relationships. Models operating on flat vectors cannot distinguish correlation from causation because the representational format lacks the structural vocabulary for causal direction.

Types of Structured Representations

Structured Representations are organized thoughts — replacing the "everything in one bag" approach of flat vectors with explicitly organized data structures that mirror the compositional, relational, and hierarchical structure of reality, enabling the systematic generalization that flat neural networks notoriously lack.