VLLM adds Day-0 support for Shanghai AI Lab's Intern-S2-397B

The Shanghai AI Laboratory's new model lands in vLLM 0.22.1 with a promise of long-horizon scientific agents, but the spec sheet makes clear this is a machine that demands serious iron: 397 billion total parameters, only 17 billion active, spread across 512 experts in a Qwen3.5-derived MoE architecture.

The model runs on a 262 thousand-token context window with a shared-weight MTP head for speculative decoding and multimodal support for images and video. The official FP8 checkpoint (internlm/Intern-S2-397B-FP8) halves the memory footprint and is the configuration the vendor validates. Running FP8 takes eight H100 or H200 GPUs; BF16 requires eight 141 GB H200s — not exactly gear every lab has lying around.

The official deployment guide disables DeepGEMM (VLLM_USE_DEEP_GEMM=0) and auto-selects the FlashInfer TRTLLM MoE backend on NVIDIA. Trust-remote-code is required because the repo ships a custom tokenizer (InternS1Tokenizer). The recommended launch command sets tensor-parallel-size to 8, the qwen3 reasoning parser, and the qwen3_coder tool-call parser — settings aimed squarely at agent and coding workloads.

The base config fixes max_position_embeddings at 262,144. A YaRN RoPE override with a factor of 4.0 can stretch --max-model-len to 1,010,000 tokens. The guide warns that static YaRN holds the scaling factor constant regardless of input length, so don't enable it unless you're actually serving long prompts; for workloads around 524 thousand tokens you can drop the factor to 2.0.

Thinking mode is on by default and the model card explicitly advises against turning it off for agentic tasks. Recommended sampling parameters: temperature 0.8, top_p 0.95, top_k 50, min_p 0.0, with max_tokens up to 32,768. One notable gap: time-series inference exists only in LMDeploy right now, not in vLLM. If you hit OOM at 262 thousand tokens, the guidance is to lower max-model-len to 65,536 or 131,072, or reduce gpu-memory-utilization.