queue-based contrastive learning

**Queue-Based Contrastive Learning** is the **MoCo-style approach where negative samples are maintained in a FIFO queue** — new batch representations are enqueued while the oldest are dequeued, providing a large, consistent pool of negatives with controlled staleness. **How Does the Queue Work?** - **Enqueue**: After each forward pass, the current batch's key representations (from the momentum encoder) are added to the queue. - **Dequeue**: The oldest entries are removed. - **Queue Size**: Typically 4096-65536. Independent of batch size. - **Consistency**: Momentum encoder (slowly updated EMA) ensures the queue entries are reasonably consistent. **Why It Matters** - **Decoupling**: Batch size can be small (256) while the effective number of negatives is large (65K). - **MoCo v1/v2**: The queue is the key innovation of MoCo, enabling SOTA performance on standard GPUs. - **vs. SimCLR**: SimCLR requires batch size 4096-8192 (needs many GPUs). MoCo achieves similar results with batch size 256 + queue. **Queue-Based Contrastive Learning** is **the conveyor belt of negatives** — continuously refreshing a large pool of comparison samples for effective contrastive training on modest hardware.