Data labeling assigns task-relevant annotations, judgments, or preferences to raw examples for model training and evaluation. Labels define what supervised systems learn and how performance is judged across classification, detection, segmentation, language, audio, ranking, safety, and human-preference tasks. Labels are measurements produced by people, instruments, policies, heuristics, or models—not infallible ground truth. Ontology, instructions, context, annotator expertise, uncertainty, disagreement, provenance, and downstream use determine quality. A professional responsible-AI claim identifies affected people, intended benefit, prohibited use, decision authority, data provenance, model capability, foreseeable misuse, uncertainty, recourse, monitoring, and accountable owner. Fairness, privacy, transparency, safety, accessibility, autonomy, and reliability can conflict and require explicit tradeoffs rather than a single ethics score.

Architecture, representation, and operating mechanism. A labeling program defines schema and examples, samples and secures data, routes tasks by skill, captures annotations and confidence, measures agreement, reviews/adjudicates, audits slices, versions the dataset, exports formats, trains models, and returns errors to guidelines and sampling. Classification assigns tags; detection draws boxes; segmentation traces masks; NLP marks spans/relations or produces text; audio transcribes/time-aligns; ranking compares candidates; RLHF-like work records preferences. Model-assisted prelabels accelerate work but can anchor annotators. Agreement, consensus/adjudication rate, gold-task accuracy, per-class error, boundary/IoU quality, label latency, throughput, cost, rework, abstention, coverage, annotator drift, subgroup disagreement, privacy incidents, and downstream model utility matter. Interfaces, defaults, incentives, human workflow, automation level, tool permissions, business policy, organizational governance, and downstream action often determine harm more than the model score. Defense in depth limits consequence when predictions are wrong or misused. Evaluation combines task utility with subgroup and intersectional performance, calibration, harmful-error severity, robustness, privacy risk, explanation fidelity, human override, complaint and appeal outcomes, incident rate, latency, cost, and uncertainty. Aggregate accuracy can conceal systematic harm, and a fairness metric chosen after seeing results can rationalize rather than govern.

Implementation, infrastructure, and failure modes. Clear guidelines and edge-case examples, training/certification, pilot rounds, hidden gold checks, overlapping labels, expert escalation, calibrated consensus, uncertainty/abstain, active-learning queues, prelabels, audit sampling, dataset versioning, and annotator feedback build quality. High-resolution images/video/3D require responsive rendering, GPU prelabels, streaming and storage; audio needs synchronized playback; secure VDI or on-prem systems may protect data. Tool latency and ergonomics directly affect accuracy and labor. Ambiguous ontology forces guesses, class imbalance hides rare labels, low pay/time pressure harms work, prelabels anchor errors, majority vote erases legitimate ambiguity, gold tasks are unrepresentative, annotator demographics or trauma are ignored, and train/test leakage occurs. Engineering includes data movement, finite precision, concurrency, resource contention, security boundaries, error propagation, and deterministic behavior when assumptions fail. Problem selection, impact assessment, collection, consent or lawful basis, labeling, training, evaluation, deployment, monitoring, feedback, incident response, update, retention, deletion, and retirement form one lifecycle. Decisions, datasets, model cards, approvals, exceptions, and user communications remain traceable.

Evaluation, governance, and deployment. Pilot and revise guidelines, compare expert and independent labels, analyze disagreement rather than only average it, inspect each class/slice, replay known cases, audit prelabel acceptance, test exports/transforms, measure downstream sensitivity, and monitor drift over time. Collection, privacy review, task design, platform, workforce, quality control, adjudication, dataset registry, sampling, training, evaluation, error analysis, and correction form a feedback loop. Annotation budget should target information and harm, not raw volume. Fair compensation, worker wellbeing, informed task conditions, sensitive-content support, access, minimization, consent/lawful basis, regional transfer, retention, audit, conflict of interest, and documented uncertainty are responsible-data requirements. Assurance combines documentation, data and label audits, red teaming, robustness and privacy tests, subgroup evaluation, causal or counterfactual analysis where appropriate, human-factors studies, accessibility testing, external review, incident exercises, and post-deployment monitoring. Technical tests do not replace legal, domain, or community judgment. Problem selection, impact assessment, collection, consent or lawful basis, labeling, training, evaluation, deployment, monitoring, feedback, incident response, update, retention, deletion, and retirement form one lifecycle. Decisions, datasets, model cards, approvals, exceptions, and user communications remain traceable. Evaluation combines task utility with subgroup and intersectional performance, calibration, harmful-error severity, robustness, privacy risk, explanation fidelity, human override, complaint and appeal outcomes, incident rate, latency, cost, and uncertainty. Aggregate accuracy can conceal systematic harm, and a fairness metric chosen after seeing results can rationalize rather than govern.

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Data Labeling — Human Annotation at Scale</text>
  <text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">acquire raw data → design taxonomy → annotate → review → deliver training-ready datasets for supervised learning</text>

  <!-- === TOP: Pipeline === -->
  <rect x="25" y="62" width="710" height="90" rx="6" fill="#080d14" stroke="#233043" stroke-width="1.2"/>
  <text x="380" y="80" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">Annotation Pipeline</text>

  <rect x="40" y="94" width="80" height="28" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
  <text x="80" y="111" fill="#6ee7b7" font-size="8" text-anchor="middle">Raw data</text>

  <path d="M123,108 L140,108" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="138,105 144,108 138,111" fill="#8b98a5"/>

  <rect x="147" y="94" width="80" height="28" rx="3" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.8"/>
  <text x="187" y="108" fill="#93c5fd" font-size="7.5" text-anchor="middle">Taxonomy</text>
  <text x="187" y="118" fill="#6b7684" font-size="6.5" text-anchor="middle">label schema</text>

  <path d="M230,108 L247,108" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="245,105 251,108 245,111" fill="#8b98a5"/>

  <rect x="254" y="94" width="80" height="28" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="0.8"/>
  <text x="294" y="108" fill="#fbbf24" font-size="7.5" text-anchor="middle">Pre-label (AI)</text>
  <text x="294" y="118" fill="#6b7684" font-size="6.5" text-anchor="middle">model-assist</text>

  <path d="M337,108 L354,108" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="352,105 358,108 352,111" fill="#8b98a5"/>

  <rect x="361" y="94" width="80" height="28" rx="3" fill="#1a1520" stroke="#a78bfa" stroke-width="0.8"/>
  <text x="401" y="108" fill="#c4b5fd" font-size="7.5" text-anchor="middle">Human annotate</text>
  <text x="401" y="118" fill="#6b7684" font-size="6.5" text-anchor="middle">correct + label</text>

  <path d="M444,108 L461,108" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="459,105 465,108 459,111" fill="#8b98a5"/>

  <rect x="468" y="94" width="80" height="28" rx="3" fill="#1a0f0f" stroke="#f87171" stroke-width="0.8"/>
  <text x="508" y="108" fill="#f87171" font-size="7.5" text-anchor="middle">QA review</text>
  <text x="508" y="118" fill="#6b7684" font-size="6.5" text-anchor="middle">consensus check</text>

  <path d="M551,108 L568,108" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="566,105 572,108 566,111" fill="#8b98a5"/>

  <rect x="575" y="94" width="80" height="28" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
  <text x="615" y="108" fill="#6ee7b7" font-size="7.5" text-anchor="middle">Export</text>
  <text x="615" y="118" fill="#6b7684" font-size="6.5" text-anchor="middle">JSONL/COCO</text>

  <text x="380" y="143" fill="#6b7684" font-size="8" text-anchor="middle">Model-assisted labeling: AI pre-labels → humans correct → 3-10× faster than manual</text>

  <!-- === MIDDLE LEFT: Label types === -->
  <rect x="25" y="160" width="350" height="128" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="200" y="178" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Annotation Types</text>

  <text x="45" y="198" fill="#60a5fa" font-size="8.5" font-weight="600">Classification</text>
  <text x="140" y="198" fill="#8b98a5" font-size="8.5">assign category (sentiment, topic)</text>
  <text x="45" y="214" fill="#34d399" font-size="8.5" font-weight="600">Bounding box</text>
  <text x="140" y="214" fill="#8b98a5" font-size="8.5">object detection (COCO format)</text>
  <text x="45" y="230" fill="#fbbf24" font-size="8.5" font-weight="600">Segmentation</text>
  <text x="140" y="230" fill="#8b98a5" font-size="8.5">pixel-level masks (instance/semantic)</text>
  <text x="45" y="246" fill="#c4b5fd" font-size="8.5" font-weight="600">NER / spans</text>
  <text x="130" y="246" fill="#8b98a5" font-size="8.5">named entities, relation extraction</text>
  <text x="45" y="262" fill="#f87171" font-size="8.5" font-weight="600">Preference pairs</text>
  <text x="155" y="262" fill="#8b98a5" font-size="8.5">A ≻ B comparisons (RLHF training)</text>
  <text x="45" y="278" fill="#8b98a5" font-size="8.5" font-weight="600">Keypoints / pose</text>
  <text x="155" y="278" fill="#8b98a5" font-size="8.5">skeleton joints, facial landmarks</text>

  <!-- === MIDDLE RIGHT: Quality control === -->
  <rect x="390" y="160" width="345" height="128" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="562" y="178" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Quality Control Methods</text>

  <text x="410" y="198" fill="#60a5fa" font-size="8.5" font-weight="600">Multi-annotator consensus</text>
  <text x="410" y="214" fill="#8b98a5" font-size="8.5">3+ labels per item, majority vote or MACE</text>
  <text x="410" y="232" fill="#34d399" font-size="8.5" font-weight="600">Gold standard questions</text>
  <text x="410" y="248" fill="#8b98a5" font-size="8.5">known-answer items sprinkled in batches</text>
  <text x="410" y="266" fill="#fbbf24" font-size="8.5" font-weight="600">Inter-annotator agreement</text>
  <text x="410" y="282" fill="#8b98a5" font-size="8.5">Cohen's κ, Krippendorff's α (target &gt; 0.8)</text>

  <!-- === BOTTOM: Tools and platforms === -->
  <rect x="25" y="298" width="710" height="105" rx="5" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="380" y="316" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Annotation Platforms</text>

  <text x="95" y="340" fill="#c4b5fd" font-size="9.5" text-anchor="middle" font-weight="600">Scale AI</text>
  <text x="95" y="354" fill="#8b98a5" font-size="8" text-anchor="middle">largest, RLHF focus</text>
  <text x="95" y="366" fill="#6b7684" font-size="7.5" text-anchor="middle">managed workforce</text>

  <text x="225" y="340" fill="#fbbf24" font-size="9.5" text-anchor="middle" font-weight="600">Label Studio</text>
  <text x="225" y="354" fill="#8b98a5" font-size="8" text-anchor="middle">open-source, self-host</text>
  <text x="225" y="366" fill="#6b7684" font-size="7.5" text-anchor="middle">flexible, ML-assisted</text>

  <text x="355" y="340" fill="#34d399" font-size="9.5" text-anchor="middle" font-weight="600">Labelbox</text>
  <text x="355" y="354" fill="#8b98a5" font-size="8" text-anchor="middle">enterprise, auto-label</text>
  <text x="355" y="366" fill="#6b7684" font-size="7.5" text-anchor="middle">vision + NLP + video</text>

  <text x="485" y="340" fill="#60a5fa" font-size="9.5" text-anchor="middle" font-weight="600">CVAT</text>
  <text x="485" y="354" fill="#8b98a5" font-size="8" text-anchor="middle">open-source, Intel</text>
  <text x="485" y="366" fill="#6b7684" font-size="7.5" text-anchor="middle">computer vision focus</text>

  <text x="615" y="340" fill="#f87171" font-size="9.5" text-anchor="middle" font-weight="600">Surge AI</text>
  <text x="615" y="354" fill="#8b98a5" font-size="8" text-anchor="middle">expert annotators</text>
  <text x="615" y="366" fill="#6b7684" font-size="7.5" text-anchor="middle">NLP + LLM evals</text>

  <text x="380" y="393" fill="#fbbf24" font-size="8.5" text-anchor="middle">Trend: LLMs as annotators (GPT-4 labels) → human review → 5-20× cheaper, comparable quality for simple tasks</text>

  <!-- Key insight -->
  <rect x="25" y="411" width="710" height="22" rx="3" fill="#0b1220" stroke="#233043" stroke-width="0.8"/>
  <text x="380" y="426" fill="#fbbf24" font-size="9" text-anchor="middle">Data quality &gt; data quantity: 10K well-labeled examples often beat 1M noisy ones — invest in annotation guidelines.</text>

  <text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">The model is only as good as its labels — data labeling is the unglamorous foundation of every supervised AI system.</text>
</svg>

Selection and practical application. Use domain experts for consequential or technical judgments, crowds for well-specified scalable tasks, model assistance for speed with bias controls, weak/self-supervision for bootstrapping, and active learning to prioritize expensive labels. Medical records, autonomous scenes, semiconductor defects, speech, translation, moderation, search relevance, recommendations, document AI, assistants, and preference alignment depend on labels. Interfaces, defaults, incentives, human workflow, automation level, tool permissions, business policy, organizational governance, and downstream action often determine harm more than the model score. Defense in depth limits consequence when predictions are wrong or misused. A professional responsible-AI claim identifies affected people, intended benefit, prohibited use, decision authority, data provenance, model capability, foreseeable misuse, uncertainty, recourse, monitoring, and accountable owner. Fairness, privacy, transparency, safety, accessibility, autonomy, and reliability can conflict and require explicit tradeoffs rather than a single ethics score. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.