hard prompt search

**Hard prompt search** is the process of systematically exploring the space of **discrete natural language prompts** to find prompt text that maximizes a language model's performance on a target task — treating the prompt as a combinatorial optimization variable rather than relying on human intuition. **Why Hard Prompt Search?** - The performance of large language models (LLMs) is **highly sensitive** to the exact wording, structure, and formatting of the prompt — small changes in phrasing can cause large accuracy swings. - **Human-crafted prompts** may not be optimal — the prompt space is vast and unintuitive. - Hard prompt search explores many candidate prompts automatically to find high-performing ones. **Hard Prompt Search Methods** - **Paraphrase Mining**: Generate paraphrases of a seed prompt using back-translation, synonym replacement, or LLM-based rewriting. Evaluate each variant on a validation set. - **Template Search**: Define a prompt template with slots (e.g., "Classify the following [text type] as [label set]") and search over fill-in options. - **Evolutionary Methods**: Treat prompts as individuals in a genetic algorithm — mutate (change words), crossover (combine parts of good prompts), and select (keep the best performers). - **RL-Based Search**: Use reinforcement learning where the action is selecting/modifying prompt tokens and the reward is task performance. - **LLM-Guided Search**: Use one LLM to generate and refine prompts for another — the "meta-prompt" approach. **Hard Prompt vs. Soft Prompt** - **Hard Prompt**: Actual human-readable text tokens — can be inspected, understood, and manually edited. Works with any model API (including black-box inference endpoints). - **Soft Prompt**: Continuous embedding vectors prepended to the input — not human-readable, requires access to model internals. - Hard prompt search is more practical for **production deployment** where models are accessed through APIs. **Hard Prompt Search Challenges** - **Combinatorial Explosion**: The space of possible prompts is astronomically large — exhaustive search is impossible. - **Evaluation Cost**: Each candidate prompt must be evaluated on a validation set — requires many model inference calls. - **Task Specificity**: Optimal prompts are highly task-specific — a prompt that works well for one task may fail on another. - **Model Specificity**: Optimal prompts often differ between models — a prompt optimized for GPT-4 may not be optimal for Claude or Llama. - **Overfitting**: Prompts optimized on a small validation set may not generalize to new examples. **Practical Applications** - **Prompt Engineering Tools**: AutoPrompt, PromptBreeder, OPRO, DSPy — frameworks that automate prompt search. - **Classification Tasks**: Finding the optimal instruction and label verbalizers for text classification. - **Few-Shot Optimization**: Searching for the best instruction preamble to combine with few-shot examples. Hard prompt search transforms prompt engineering from an **art into a science** — replacing ad-hoc trial-and-error with systematic optimization to find the best possible prompt for any task.