Compositional Networks are neural architectures explicitly designed to solve problems by assembling and executing sequences of learned sub-functions that mirror the compositional structure of the input — reflecting the fundamental principle that complex meanings, visual scenes, and reasoning chains are built from the systematic combination of simpler primitives, just as "red ball on blue table" is composed from independent concepts of color, object, and spatial relation.

What Are Compositional Networks?

• Definition: Compositional networks decompose a complex task into a structured sequence of primitive operations, where each operation is implemented by a trainable neural module. The composition structure — which modules execute in what order — is determined by the input (typically parsed into a symbolic program or tree structure) rather than being fixed for all inputs.
• Compositionality Principle: Human cognition is fundamentally compositional — we understand "red ball" by composing "red" and "ball," and we can immediately understand "blue ball" by substituting "blue" without learning a new concept. Compositional networks embody this principle architecturally, learning primitive concepts that can be freely recombined to understand novel combinations.
• Program Synthesis: Many compositional networks operate by first parsing the input (question, instruction, scene description) into a symbolic program (e.g., Filter(red) → Filter(sphere) → Relate(left) → Filter(green) → Filter(cube)), then executing each program step using a corresponding neural module. The program structure provides the composition; the neural modules provide the perceptual grounding.

Why Compositional Networks Matter

• Systematic Generalization: Standard neural networks fail at systematic generalization — they can learn "red ball" and "blue cube" from training data but struggle with "red cube" if it was never seen, because they learn holistic patterns rather than compositional rules. Compositional networks generalize systematically because they compose independent primitives: if "red" and "cube" are learned separately, "red cube" is automatically available.
• CLEVR Benchmark: The CLEVR dataset (Compositional Language and Elementary Visual Reasoning) became the standard testbed for compositional visual reasoning: "Is the red sphere left of the green cube?" requires composing spatial, color, and shape filters. Neural Module Networks achieved near-perfect accuracy by parsing questions into module programs, while end-to-end models struggled with complex compositions.
• Data Efficiency: Compositional networks require less training data because they learn reusable primitives rather than holistic patterns. Learning N objects × M colors × K relations requires O(N + M + K) examples compositionally, versus O(N × M × K) examples holistically — an exponential reduction.
• Interpretability: The module execution trace provides a complete explanation of the reasoning process. For "How many red objects are bigger than the blue cylinder?", the trace shows: Filter(red) → FilterBigger(Filter(blue) → Filter(cylinder)) → Count — a step-by-step reasoning path that can be verified and debugged by humans.

Key Compositional Network Architectures

Compositional Networks are syntactic solvers — treating complex reasoning as grammatical assembly of logic primitives, enabling neural networks to achieve the systematic generalization that comes naturally to human cognition but has long eluded monolithic end-to-end learning approaches.