Prompt engineering technique to overcome mode collapse in LLM responses by generating multiple answers with probabilities...
Verbalized Sampling (VS) is a prompt engineering technique that mitigates mode collapse in LLM responses by explicitly requesting multiple answers with associated probabilities.
Standard LLMs typically display only the top-ranked response due to RLHF training. This "mode collapse" hides alternative viable answers. VS overcomes this by instructing the AI to sample from its full internal probability distribution.
You are a helpful assistant. For each question that I ask, generate a set of 5 possible responses. Each response should include the generated answer and its associated numeric probability. Show me all five responses. Please sample at random from the full distribution.
Use the base template to see a representative sample of possible answers with their probabilities.
Generate a set of 10 possible responses.
Use when the question is complex or you want broader coverage of the solution space.
Please only show me the response that has the highest probability.
Use to revert to conventional behavior while maintaining the VS framework.
Please only show me the response that has the lowest probability.
Use to discover unexpected or unconventional solutions.
Please sample from the tails of the distribution such that the probability of each response is less than 0.10.
Use to explore edge cases or novel approaches.
Please show me responses with probabilities between 0.30 and 0.60.
Use to focus on mid-range alternatives that balance novelty and reliability.
If you request more responses than genuinely exist, the AI may invent additional answers to satisfy your request. Always critically evaluate all responses.
Probabilities are approximations, not exact values. The AI may generate probabilities to satisfy your request rather than computing precise values. Use them as relative indicators, not absolute measures.
The burden of verifying response validity rests with you. Cross-check answers, especially those with lower probabilities.
Expect slightly increased response time due to additional processing required to generate multiple responses.
If using a paid API, VS prompts will increase costs due to longer responses and additional processing. Occasional use has negligible impact; frequent use may noticeably increase bills.
Works across major LLMs including ChatGPT, Claude, Gemini, Llama, and Grok. The technique is training-free and model-agnostic.
VS can be combined with other prompting techniques. Place the VS instruction at the beginning of your prompt, followed by your specific request:
You are a helpful assistant. For each question that I ask, generate a set of 5 possible responses. Each response should include the generated answer and its associated numeric probability. Show me all five responses. Please sample at random from the full distribution.
[Your specific instruction or question here]
Based on "Verbalized Sampling: How To Mitigate Mode Collapse And Unlock LLM Diversity" by Zhang et al. (arXiv, October 2025). The technique addresses how RLHF post-training alignment unintentionally creates mode collapse, limiting response diversity.
