Why Anthropic Chose Examples Over Stricter Schemas for Tool Calls
TRIGGER
JSON schemas define structural validity but can't express usage patterns—models were generating malformed tool calls because they couldn't infer date formats (YYYY-MM-DD vs ISO 8601?), ID conventions (UUID vs 'USR-12345'?), or when to populate optional nested fields from schema alone.
APPROACH
Anthropic added `input_examples` field to tool definitions containing 1-5 concrete sample calls showing realistic data. Input: array of example parameter objects alongside the schema. Output: model learns format conventions, ID patterns, and optional parameter correlation from examples. For a create_ticket tool, three examples showed: (1) critical bug with full contact/escalation and tight SLA, (2) feature request with reporter but no escalation, (3) internal task with title only. Results: accuracy improved from 72% to 90% on complex parameter handling in internal testing.
PATTERN
“Schemas say "string"; examples show "USR-12345" not UUID. The model learns conventions from patterns, not type constraints. Add 3-5 examples alongside tool definitions.”
✓ WORKS WHEN
- Tools have complex nested structures where valid JSON doesn't imply correct usage
- Many optional parameters with non-obvious inclusion patterns
- APIs have domain-specific conventions not expressible in JSON Schema (ID formats, date styles)
- Similar tools exist where examples clarify which to use (create_ticket vs create_incident)
- Parameter errors are a measurable bottleneck in current tool use
✗ FAILS WHEN
- Simple single-parameter tools with obvious usage
- Standard formats Claude already understands (URLs, emails, UUIDs)
- Constraints are better enforced by schema validation than examples
- Token budget is extremely tight and 1-5 examples per tool adds unacceptable overhead
- Tools change frequently and examples would require constant maintenance