Mistral Large 4 open weights: can you run it on your own servers?

说明与用途

组织可以在自己的服务器上托管 Mistral Large 4，但需要 4–8 台现代 GPU、遵守许可证并配置离线组件。

问题

Can an organization host Mistral Large 4 on its own servers without sending data to an API, and what are the hardware, licensing, and operational requirements?

结果

Self‑hosting Mistral Large 4 is feasible but requires substantial GPU resources, careful offline configuration, and compliance with a commercial license for third‑party service. Organizations can achieve sovereign AI while managing cost and operational complexity.

工作流

• 1. Verify model availability and license terms on Hugging Face.
• 2. Assess hardware needs: 4–8 latest‑generation GPUs (e.g., Nvidia B200/B300 Blackwell) for NVFP4 or 8 GPUs for bf16 with tensor parallelism.
• 3. Install vLLM and required dependencies, ensuring local tokenizer files are accessible (fix for mistral‑common bug).
• 4. Configure KV cache, autoscaling, and monitoring per vLLM best practices.
• 5. Evaluate the model against internal use cases and compare with smaller models and the API.
• 6. Add custom guardrails and compliance checks, since API protections do not apply.
• 7. Obtain a commercial license from Mistral if the model will be offered as a service to third parties.

证据

The weights are about 1.06 TB in bf16 and 610 GB in NVFP4. The published vLLM recipe uses eight GPUs with tensor parallelism. The NVFP4 version runs, according to Mistral, on four Nvidia B200 or B300 GPUs (Blackwell).

来源

The Mistral Ecosystem Open-Weight License is a modified MIT licence. Internal use, commercial use and embedding the model in a product are free, with no revenue threshold. Selling access to the model as a service requires a separate commercial licence with Mistral.

来源

I fixed a bug in mistral-common that prevented loading a tokenizer already on disk.

来源