Gradient-Based Pruning
model optimization
Gradient-Based Pruning is a more principled pruning criterion — using gradient information (or second-order derivatives) to estimate the impact of removing a weight on the loss…
15,410 technical terms and definitions
model optimization
Gradient-Based Pruning is a more principled pruning criterion — using gradient information (or second-order derivatives) to estimate the impact of removing a weight on the loss…
model optimization
Gradient-Based Pruning is pruning strategies that rank parameters using gradient-derived importance signals - It leverages optimization sensitivity to remove low-impact…
gradient boosting sequential residuals prediction, gradient boosting sequential, boosting sequential residuals, sequential residuals prediction, gradient boosting
Gradient Boosting Sequential Residuals Prediction with Machine Learning Introduction & Motivation Gradient Boosting Sequential Residuals Prediction with Machine Learning addresses…
Gradient Boosting Uncertainty Introduction & Motivation Gradient Boosting Uncertainty addresses a central problem in uncertainty methods and prediction intervals: how to Quantify…
distributed training
Gradient bucketing is the grouping of many small gradient tensors into larger communication chunks before collective operations - it improves network efficiency by reducing…
activation recomputation, memory optimization training
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
activation checkpointing, memory efficient training, recomputation training, checkpointing deep learning
Gradient checkpointing and gradient accumulation are the two techniques that let you train a model that does not fit in memory. They attack different halves of the training memory…
activation recomputation, memory efficient training, checkpoint segment, rematerialization
Gradient Checkpointing (Activation Recomputation) is the memory optimization technique for training deep neural networks that trades compute for memory by storing only a subset…
model training
Gradient clipping caps gradient magnitude to prevent exploding gradients that destabilize training. The problem: Large gradients cause huge weight updates, loss spikes, or NaN…
training techniques
Gradient Clipping is operation that limits gradient magnitude to a fixed norm before optimization updates - It is a core method in modern semiconductor AI serving…
Gradient Clipping Strategies Introduction & Motivation Gradient Clipping Strategies addresses a central problem in optimization algorithm tuning and improvement: how to Optimize…
gradient compression communication-efficient prediction, gradient compression communication-efficient, compression communication-efficient prediction
Gradient Compression Communication-Efficient Prediction with Machine Learning Introduction & Motivation Gradient Compression Communication-Efficient Prediction with Machine…
gradient compression and quantization for communication efficiency, gradient compression quantization, compression quantization communication, quantization communication efficiency
Gradient Compression and Quantization for Communication Efficiency Introduction & Motivation Gradient Compression and Quantization for Communication Efficiency addresses a central…
distributed training
Gradient compression techniques is the communication-reduction methods that lower distributed training bandwidth demand by encoding or sparsifying gradients - they reduce…
top k sparsification, gradient sparsity training, magnitude based pruning, sparse gradient communication
Gradient Compression Techniques are the family of methods that reduce gradient communication volume by transmitting only the most important gradient components — using…
Gradient Descent Algorithm Variants Introduction & Motivation Gradient Descent Algorithm Variants addresses a central problem in optimization algorithm tuning and improvement: how…
model training
Gradient Flow Preservation is a design principle for pruning and sparse training — ensuring that removing weights does not disrupt the backpropagation signal, keeping gradient…
gradient flow stability in deep networks prediction, gradient flow stability, flow stability deep, stability deep networks, deep networks prediction, gradient flow
Gradient Flow Stability in Deep Networks Prediction with Machine Learning Introduction & Motivation Gradient Flow Stability in Deep Networks Prediction with Machine Learning…
Evolution Strategies for Gradient-Free Meta-Learning Introduction & Motivation Evolution Strategies for Gradient-Free Meta-Learning addresses a central problem in fewshot…
Nelder-Mead Simplex Direct Search Algorithm Introduction & Motivation Nelder-Mead Simplex Direct Search Algorithm addresses a central problem in optimization algorithm tuning…
Score-CAM Gradient-Free Visual Activation Mapping Introduction & Motivation Score-CAM Gradient-Free Visual Activation Mapping addresses a central problem in explainability…
ai safety
Gradient Masking is a phenomenon where a defense accidentally or intentionally makes the model's gradients uninformative — causing gradient-based attacks to fail while the model…
Gradient Masking Pitfalls Introduction & Motivation Gradient Masking Pitfalls addresses a central problem in robustness of advrobust models under adversarial conditions: how…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
Gradient Norm Expectation for Deep Model Active Selection Introduction & Motivation Gradient Norm Expectation for Deep Model Active Selection addresses a central problem…
generative models
Gradient Penalty is a regularization technique used primarily in GAN training (WGAN-GP) — penalizing the norm of the discriminator's gradient with respect to its input, enforcing…
distributed training
Gradient quantization for communication reduces the precision of gradient tensors before transmitting them between workers in distributed training, dramatically reducing network…
domain adaptation
The Gradient Reversal Layer (GRL) is the ingenious mathematical trick at the beating heart of Adversarial Domain Adaptation (specifically DANN), functioning as a simple, custom…
Gradient Similarity and Dot-Product Task Affinity Introduction & Motivation Gradient Similarity and Dot-Product Task Affinity addresses a central problem in transfer learning…
Gradient Surgery for Multi-Task Learning (PCGrad) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) addresses a central problem in multitask learning…
Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) addresses a central…
Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) addresses a central…
PCGrad Projected Conflicting Gradients Multi-Task Introduction & Motivation PCGrad Projected Conflicting Gradients Multi-Task addresses a central problem in optimization algorithm…
distributed training
Gradient synchronization is the distributed operation that aligns per-worker gradients into a shared update before parameter step - it ensures data-parallel replicas remain…
GradNorm Loss Balancing Introduction & Motivation GradNorm Loss Balancing addresses a central problem in multitask learning across several related prediction objectives: how…
GradNorm Loss Balancing (Extended Study) Introduction & Motivation GradNorm Loss Balancing (Extended Study) addresses a central problem in multitask learning across several…
GradNorm Loss Balancing (Extended Study) Introduction & Motivation GradNorm Loss Balancing (Extended Study) addresses a central problem in multitask learning across several…
Gradual Domain Adaptation along Intermediate Trajectories Introduction & Motivation Gradual Domain Adaptation along Intermediate Trajectories addresses a central problem…
defects
Grain Boundaries are interfaces separating crystallites (grains) of the same material that have different crystallographic orientations — they are regions of atomic disorder where…
grain boundary analysis, grain boundary character distribution, gbcd, five parameter grain boundary, grain boundary misorientation, grain boundary plane characterization
A grain boundary is only a few atomic spacings wide, yet a connected network of those interfaces can govern the resistivity of an interconnect, the lifetime of a solder joint…
grain boundary density prediction, grain boundary density, boundary density prediction, grain boundary, boundary density, density prediction
Grain Boundary Density Prediction with Machine Learning Introduction & Motivation Grain Boundary Density Prediction with Machine Learning addresses a central problem in grain…
defects
Grain Boundary Energy is the excess free energy per unit area associated with the disordered atomic arrangement at a grain boundary compared to the perfect crystal interior — this…
high-angle grain boundary, defects, crystal defects
High-Angle Grain Boundary (HAGB) is a grain boundary with a misorientation angle exceeding approximately 15 degrees, where the atomic structure is fundamentally disordered…
grain boundary, boundary modeling, csl dft gb gbdb ml
Grain Boundary Modeling Introduction & Motivation Predicting grain boundary properties and segregation enables design of materials with optimized microstructure and mechanical…
defects
Grain Boundary Segregation is the thermodynamically driven accumulation of solute atoms (dopants, impurities, or alloying elements) at grain boundaries where the disordered atomic…
beol
Grain Growth in Copper is the microstructural evolution process where small copper grains coalesce into larger ones — driven by the reduction of grain boundary energy, occurring…
Gram Matrix for Style Representation Introduction & Motivation Gram Matrix for Style Representation addresses a central problem in generative generation…
grammar-based fuzzing
Grammar-Based Fuzzing Introduction & Motivation Grammar-Based Fuzzing addresses a central problem in verification process control and grammar-coverage-percentage prediction: how…
graph neural networks
Grammar-Based Generation is graph generation constrained by production grammars that encode valid construction rules - It guarantees syntactic validity by restricting generation…