vllm serve with flags built from your environment variables. The tables below list common settings, but you’re not limited to them:
- Any environment variable named after a
vllm serveflag in upper snake case becomes that flag. For example,MAX_MODEL_LEN=4096becomes--max-model-len 4096. - Boolean variables take
trueorfalse. VLLM_EXTRA_ARGSpasses extra flags straight tovllm serve. They’re added last, so they override any earlier setting. Use it for flags that don’t have an environment variable, for exampleVLLM_EXTRA_ARGS="--tool-call-parser qwen3_coder --enable-auto-tool-choice".
LLM settings
These variables control the core language model configuration.LoRA settings
Configure LoRA (Low-Rank Adaptation) adapters for your model.Speculative decoding settings
Configure speculative decoding to improve performance. Pass the whole configuration as one JSON string inSPECULATIVE_CONFIG, which vLLM parses directly.
For example, to use a draft model with three speculative tokens:
SPECULATIVE_MODEL and NUM_SPECULATIVE_TOKENS, are no longer read. See Removed variables.
System performance settings
Configure GPU memory and system resource utilization.Tokenizer settings
Customize tokenizer behavior and chat templates.OpenAI compatibility settings
Configure OpenAI API compatibility features.TOOL_CALL_PARSER tells vLLM how to interpret a model’s tool-call output. If the parser doesn’t match the model’s format, tool calls may not be detected (or may error during parsing).Serverless and concurrency settings
Configure concurrency, timeouts, and logging for Serverless deployments. The worker readsMAX_CONCURRENCY, the two timeouts, VLLM_EXTRA_ARGS, and VLLM_CONFIG_FILE itself. The logging variables go to vLLM.
Advanced settings
Additional configuration options for specialized use cases.Docker build arguments
These arguments are used when building a custom worker image, for example to bake a model in. They’re set at build time, so changing them on an endpoint has no effect.Removed variables
The worker no longer reads the following variables. Setting one has no effect. Variables named after a flag that vLLM removed also log a warning at startup.
For the complete list, see the worker’s configuration reference.