Glossary

constrained optimization Lagrange multipliers KKT conditions convex optimization duality interior point methods penalty methods semidefinite programming constraint qualifications

Bayesian inference posterior approximation Markov Chain Monte Carlo variational inference MCMC Hamiltonian sampling importance sampling model comparison hierarchical models stochastic optimization

Generative Adversarial Networks GANs minimax objective Wasserstein GAN WGAN GP conditional GAN mode collapse Fréchet Inception Distance FID generative models

Variational Autoencoders VAEs Evidence Lower Bound ELBO reparameterization trick latent space continuous latent variables KL divergence generative modeling

Reinforcement Learning Markov Decision Processes MDPs Bellman equations dynamic programming policy iteration value iteration Monte Carlo temporal difference learning Q-learning

Deep Reinforcement Learning Deep Q-Networks DQN Policy Gradients REINFORCE Actor-Critic A2C PPO Soft Actor-Critic SAC continuous control value function approximation

Transfer Learning fine-tuning domain adaptation feature extraction pre-trained models domain adversarial neural networks DANN maximum mean discrepancy MMD zero-shot cross-domain generalization

Meta-Learning Few-Shot Learning Model-Agnostic Meta-Learning MAML Prototypical Networks Matching Networks Relation Networks episodic training optimization-based meta-learning

Graph Neural Networks GNNs Graph Convolutional Networks GCN Graph Attention Networks GAT Message Passing Neural Networks MPNN graph representation node classification spatial spectral graph convolutions

Transformers Natural Language Processing self-attention multi-head attention BERT GPT autoregressive models positional encoding seq2seq pre-training fine-tuning language models

Convolutional Neural Networks Computer Vision convolutions pooling feature maps Residual Networks ResNet object detection instance segmentation spatial invariance receptive field

Information Theory Shannon entropy mutual information Kullback-Leibler KL divergence cross-entropy rate-distortion theory source coding channel capacity rate distortion optimization

Probabilistic Graphical Models PGMs Bayesian networks Markov random fields MRFs factor graphs belief propagation sum-product algorithm junction tree variational inference structure learning

Recommendation Systems collaborative filtering matrix factorization Singular Value Decomposition SVD neural collaborative filtering NCF content-based filtering cold-start hybrid recommenders precision recall ranking metrics

101000

lstm and recurrent neural networks capturing sequential dependencies, lstm recurrent neural networks capturing sequential dependencies, lstm recurrent, neural networks capturing, 103m 10d 50k 7m 929k arima, lstm recurrent neural

101001

sequence-to-sequence models in nlp encoder-decoder architectures and attention, sequence to sequence models in nlp encoder decoder architectures and attention, sequence sequence models nlp encoder decoder architectures attention, models

101002

diffusion models denoising score matching and generative modeling, diffusion models denoising score matching generative modeling, diffusion models, denoising score matching, 10k 123k 30k 32gb 3d 3m, diffusion models denoising

101003

federated learning distributed training privacy and communication efficiency, federated learning distributed training privacy communication efficiency, federated learning, distributed training privacy, 100gb 100k 100mb 10gb 10k 10m

101004

continual learning catastrophic forgetting elastic weight consolidation and task-incremental learning, continual learning catastrophic forgetting elastic weight consolidation and task incremental learning, continual learning catastrophic

101005

active learning uncertainty sampling query strategies and annotation efficiency, active learning uncertainty sampling query strategies annotation efficiency, active learning, uncertainty sampling query, 100k 10k 120k 15k 1k 1m

101006

explainability and interpretability shap lime attention and saliency, explainability interpretability shap lime attention saliency, explainability interpretability, shap lime attention, 10k ai bert cam clip compas, lime attention saliency

101007

adversarial robustness adversarial examples attacks defenses and verification, adversarial robustness adversarial examples attacks defenses verification, adversarial robustness, adversarial examples attacks, 10k ai cdf cifar clip cnn

101008

time series forecasting arima exponential smoothing prophet and deep learning, time series forecasting arima exponential smoothing prophet deep learning, time series, forecasting arima exponential, 100k 10k 10m 24k acf aic, prophet deep

101009

anomaly detection isolation forest lof autoencoders and statistical methods, anomaly detection isolation forest lof autoencoders statistical methods, anomaly detection, isolation forest lof, 100d 100k 10k 284k 494k 50d, detection isolation

101010

gaussian processes and kernel methods rbf svm bayesian inference, gaussian processes kernel methods rbf svm bayesian inference, gaussian processes, kernel methods rbf, 100k 10k 1m 800mb 80gb bfgs, gaussian processes kernel, methods rbf svm

101011

multi-task learning parameter sharing negative transfer and representation learning, multi task learning parameter sharing negative transfer and representation learning, mult, multi-task learning parameter, learning parameter sharing

101012

quantum machine learning bridging quantum computing and ml algorithms, quantum machine learning bridging quantum computing ml algorithms, quantum machine, learning bridging quantum, 2d 3g api cnot cobyla h2, quantum machine learning

101013

causal inference causal discovery from association to causation, causal inference causal discovery association causation, causal inference, causal discovery association, 2sls acic api ate bart cate, causal inference causal, inference causal

101014

self-supervised learning contrastive methods learning from unlabeled data, self supervised learning contrastive methods learning from unlabeled data, self supervised learning contrastive methods learning unlabeled data, self supervised

101024

multimodal learning vision-language models from clip to cross-modal reasoning, multimodal learning vision language models from clip to cross modal reasoning, multimodal learning vision language models clip cross modal reasoning, visio

101025

model compression quantization pruning and knowledge distillation, model compression quantization pruning knowledge distillation, model compression, quantization pruning knowledge, ar awq bert c4 ce fp16, model compression quantization

101026

multi-agent reinforcement learning game-theoretic learning, multi agent reinforcement learning game theoretic learning, multi agent, reinforcement learning game, 2d ai api ctde dqn ii, multi-agent reinforcement learning

101027

retrieval-augmented generation vector search for llms, retrieval augmented generation vector search for llms, retrieval augmented generation vector search llms, retrieval augmented, generation vector search, ai ann beir bge bm25 dpr

101028

physics-informed neural networks neural odes, physics informed neural networks neural odes, physics informed, neural networks neural, 1d 3d cde cifar gpu l2, physics-informed neural networks & neural odes, physics-informed neural networks

101029

automl hyperparameter optimization bayesian search, automl hyperparameter, optimization bayesian search, 1d 2d amlb api asha aws, automl hyperparameter optimization & bayesian search, automl hyperparameter optimization, bayesian search

101030

speech recognition audio representation learning, speech recognition, audio representation learning, asr cer ctc gpu lium llm, speech recognition & audio representation learning, speech recognition audio, recognition audio representation

101031

object detection semantic segmentation, object detection, detection semantic, 2d 3d ade20k ce cnn coco, object detection & semantic segmentation, object detection semantic, detection semantic segmentation, semantic segmentation

101032

in-context learning prompt engineering, in context learning prompt engineering, context learning prompt engineering, context learning, learning prompt, ai ape api bbh big gpt, in-context learning & prompt engineering, in-context learning

101033

generative adversarial networks adversarial generative modeling, generative adversarial, networks adversarial generative, 1d cifar dcgan esrgan fid gan, generative adversarial networks & adversarial generative modeling, adversarial networks

101034

mixture-of-experts sparse model scaling, mixture of experts sparse model scaling, mixture experts sparse model scaling, mixture experts, sparse model scaling, arc gsm8k math mbpp mmlu, mixture-of-experts & sparse model scaling, sparse model

101035

state-space models long-context sequence architectures mamba, state space models long context sequence architectures mamba, state space, models long context, a1 a2 arc cuda dna gpu, state-space models long-context, state-space models

101036

neural radiance fields novel view synthesis, neural radiance, fields novel view, 2d 3d aabb back colmap front, neural radiance fields & novel view synthesis, neural radiance fields, radiance fields novel, novel view synthesis, fields novel

101037

offline reinforcement learning imitation learning, offline reinforcement, learning imitation learning, 1d actual bc cql d4rl down, offline reinforcement learning & imitation learning, offline reinforcement learning, reinforcement learning

101038

vision transformers masked image modeling, vision transformers, masked image modeling, 1d 2d 300m bert clip coco, vision transformers & masked image modeling, vision transformers masked, transformers masked image, transformers masked

101039

direct preference optimization llm alignment, direct preference, optimization llm alignment, 1d ai dpo gpt hh higher, direct preference optimization & llm alignment, direct preference optimization, preference optimization llm, llm alignment

101040

speculative decoding kv-cache optimization, speculative decoding kv cache optimization, speculative decoding, kv cache optimization, gb gpu gqa kv llm mha, speculative decoding & kv-cache optimization, speculative decoding kv-cache

101041

equivariant neural networks geometric deep learning, equivariant neural, networks geometric deep, 2d 3d casp cnn egnn fails, equivariant neural networks & geometric deep learning, equivariant neural networks, neural networks geometric

101042

world models model-based reinforcement learning, world models model based reinforcement learning, world models model reinforcement learning, world models, model reinforcement learning, 2d cem combo cpu dmc dqn, world models model-based

101043

chain-of-thought reasoning test-time compute scaling, chain of thought reasoning test time compute scaling, chain thought reasoning test time compute scaling, chain thought, reasoning test time, 32l agi ai aime api arc, reasoning test-time

101044

neural scaling laws emergent abilities, neural scaling, laws emergent abilities, 6n 6nd bfgs big bnsl gpt, neural scaling laws & emergent abilities, neural scaling laws, scaling laws emergent, scaling laws, laws emergent, emergent abilities