neural architecture search
**Neural architecture search automatically explores neural-network structures under task and deployment objectives.** NAS can discover layer types, topology, width, depth, resolution, attention, sparsity, and operator choices that outperform hand tuning, especially when accelerator latency or memory is part of the objective. A professional machine-learning claim specifies the task, data distribution, split strategy, model and training recipe, inference constraints, comparison baseline, uncertainty, and failure cost. Accuracy on one benchmark is not a deployment specification. Quality, latency, throughput, memory, energy, robustness, privacy, maintainability, and human workflow must be evaluated together under the intended operating distribution. A NAS result is inseparable from its search space, weight-training method, performance estimator, search algorithm, compute budget, and final retraining. Large search spaces can contain invalid or operationally unsupported models.
**Architecture and operating mechanism.** The search space encodes candidate operations and connections; a controller or optimizer proposes architectures; a performance estimator trains, shares weights, predicts quality, or uses low-fidelity proxies; a cost model supplies latency or energy; an archive retains Pareto candidates. RL controllers treat validation reward as feedback, evolution mutates and selects populations, differentiable NAS relaxes discrete choices into continuous weights, one-shot supernets share parameters among subnetworks, and predictor-based methods learn architecture-to-performance mappings. The complete system includes data loaders, tokenizers or preprocessors, model execution, memory hierarchy, accelerators, interconnect, postprocessing, policy filters, APIs, caches, observability, and human escalation. Optimization is credible only when it preserves the relevant behavior and measures end-to-end cost rather than an isolated kernel or ideal operation count. Final retrained quality, search cost in accelerator-hours, wall time, number of candidates, rank correlation of proxy and final quality, target-device latency, memory, energy, parameter count, MACs, compilation success, robustness, and run variance matter. Results should report task-appropriate quality metrics alongside calibration, subgroup behavior, worst-case or tail latency, tokens or samples per second, model and activation memory, training compute, serving cost, energy, data volume, and confidence intervals across seeds or resamples. Ablations isolate causal contributions; controlled baselines prevent extra data or compute from being mislabeled as an algorithmic gain.
**Implementation, acceleration, and failure modes.** Weight sharing reduces cost but couples candidates; progressive shrinking trains elastic width/depth/kernel choices; latency lookup tables approximate hardware; compiler-in-the-loop measurement captures fusion and memory; constraints eliminate unsupported tensors or operators. Search overfits the validation set, proxies misrank architectures, shared weights favor certain paths, latency models miss compiler behavior, FLOPs poorly predict memory-bound time, retraining loses gains, and reported search cost may omit supernet development or failed trials. Hardware-aware objectives measure batch-specific latency, SRAM/HBM traffic, tensor-core utilization, quantization, operator fusion, DVFS, thermal throttling, and compiler support on the actual target. Multi-objective search yields a Pareto frontier rather than one universal architecture. Engineering must include interfaces, numerical or physical limits, concurrency, resource contention, error propagation, and safe behavior when assumptions are violated. Data collection, licensing, filtering, labeling, pretraining, adaptation, evaluation, deployment, monitoring, feedback, rollback, and retirement form one lifecycle. Dataset and model versions, feature definitions, prompts, random seeds, dependency locks, accelerator kernels, quantization, and serving configuration must be traceable for a result to be reproducible or auditable.
**Evaluation, assurance, and deployment.** Reserve final test data, repeat search seeds, fully retrain selected candidates with matched recipes, compare against tuned manual baselines, report total compute, measure target hardware distributions, ablate search components, and publish the space and selection rule. Data pipeline, augmentation, optimizer, distillation, quantization, compiler, runtime, batch, concurrency, and serving policy can contribute more than topology. Architecture search should co-design these without attributing every improvement to structure. Search budgets, shared-cluster quotas, reproducible manifests, license-compatible operators, dataset rights, safety evaluation, and selection approvals keep automated exploration accountable. Verification uses leakage-resistant splits, out-of-distribution and stress tests, adversarial and abuse cases, calibration analysis, slice evaluation, human review where judgment matters, hardware-in-the-loop measurement, and shadow or canary deployment. Offline scores are compared with online behavior and user impact; monitoring distinguishes input drift, concept drift, pipeline faults, and deliberate manipulation. Data collection, licensing, filtering, labeling, pretraining, adaptation, evaluation, deployment, monitoring, feedback, rollback, and retirement form one lifecycle. Dataset and model versions, feature definitions, prompts, random seeds, dependency locks, accelerator kernels, quantization, and serving configuration must be traceable for a result to be reproducible or auditable. Results should report task-appropriate quality metrics alongside calibration, subgroup behavior, worst-case or tail latency, tokens or samples per second, model and activation memory, training compute, serving cost, energy, data volume, and confidence intervals across seeds or resamples. Ablations isolate causal contributions; controlled baselines prevent extra data or compute from being mislabeled as an algorithmic gain.
| NAS strategy | Search signal | Cost tendency | Strength | Primary weakness |
|---|---|---|---|---|
| Reinforcement learning | Controller reward | Historically very high | Flexible discrete spaces | Credit/sample efficiency |
| Evolutionary | Population fitness | High but parallel | Robust irregular search | Many evaluations |
| Differentiable/DARTS | Gradient relaxation | Low-medium | Fast optimization | Relaxation/proxy bias |
| One-shot supernet | Shared weights | Medium upfront | Many cheap subnet estimates | Ranking interference |
| Predictor-based | Learned performance model | Data-dependent | Efficient candidate scoring | Extrapolation error |
```svg
```
**Selection and practical use.** Use NAS when architecture space and deployment constraints are valuable and repeatable enough to amortize search; use manual or simple scaling when data, objectives, or target hardware change faster than the search can validate. Mobile vision, speech, recommendation, language-model blocks, edge AI, accelerator dataflows, operator fusion, and chip floorplanning use architecture-search ideas. The complete system includes data loaders, tokenizers or preprocessors, model execution, memory hierarchy, accelerators, interconnect, postprocessing, policy filters, APIs, caches, observability, and human escalation. Optimization is credible only when it preserves the relevant behavior and measures end-to-end cost rather than an isolated kernel or ideal operation count. A professional machine-learning claim specifies the task, data distribution, split strategy, model and training recipe, inference constraints, comparison baseline, uncertainty, and failure cost. Accuracy on one benchmark is not a deployment specification. Quality, latency, throughput, memory, energy, robustness, privacy, maintainability, and human workflow must be evaluated together under the intended operating distribution. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.