Decision Under Uncertainty: Confidence, Risk, and Verification
Decision Under Uncertainty: Confidence, Risk, and Verification
Why uncertainty is unavoidable
LLMs don’t “know” they are wrong. You must engineer uncertainty signals: low evidence, conflicting sources, missing inputs.
Risk-aware decisions
When risk is high, verify. Examples: payment changes, compliance statements, destructive actions.
Confidence thresholds
Practical rule: if confidence < threshold, do one of:
- Call a verification tool
- Ask the user a clarifying question
- Escalate to human review
