Max Length

Max length — max_tokens on most APIs, max_output_tokens on some — caps how many tokens the model may generate in a single response. It’s a hard ceiling, not a target: generation simply stops when the count is reached, mid-sentence if necessary. The model doesn’t see the limit or write more concisely because of it; if you want shorter answers, you say so in the prompt. The cap and the response share space with your input inside the model’s overall context window.

This parameter is your cost and latency circuit breaker. Output tokens are the expensive ones, and generation time scales with their count — so an unbounded response is an unbounded bill and an unbounded wait. A runaway generation loop, a model that decides to enumerate 500 examples, a prompt-injected request for an essay: max_tokens bounds the damage from all of them. On Anthropic’s API it’s required on every request; elsewhere it’s optional but you should treat it as mandatory in production.

In practice, set it per endpoint based on what a legitimate response actually needs — a classification label might get 16 tokens, a summary 300, a code generation task several thousand — with headroom so normal answers never get clipped. Then always check the response’s finish reason: length (OpenAI) or max_tokens (Anthropic) means truncation, which your code should detect and handle. Stopping gracefully at a point you choose is the next topic, stopping criteria.