Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Mistral Large 4 open weights: can you run it on your own servers?

Описание и применение

Организации могут развернуть модель Mistral Large 4 на собственных серверах, но для этого необходимы 4–8 современных GPU, соблюдение лицензии и настройка офлайн‑компонентов.

Задача

Can an organization host Mistral Large 4 on its own servers without sending data to an API, and what are the hardware, licensing, and operational requirements?

Результат

Self‑hosting Mistral Large 4 is feasible but requires substantial GPU resources, careful offline configuration, and compliance with a commercial license for third‑party service. Organizations can achieve sovereign AI while managing cost and operational complexity.

Рабочий процесс

  • 1. Verify model availability and license terms on Hugging Face.
  • 2. Assess hardware needs: 4–8 latest‑generation GPUs (e.g., Nvidia B200/B300 Blackwell) for NVFP4 or 8 GPUs for bf16 with tensor parallelism.
  • 3. Install vLLM and required dependencies, ensuring local tokenizer files are accessible (fix for mistral‑common bug).
  • 4. Configure KV cache, autoscaling, and monitoring per vLLM best practices.
  • 5. Evaluate the model against internal use cases and compare with smaller models and the API.
  • 6. Add custom guardrails and compliance checks, since API protections do not apply.
  • 7. Obtain a commercial license from Mistral if the model will be offered as a service to third parties.

Доказательства

The weights are about 1.06 TB in bf16 and 610 GB in NVFP4. The published vLLM recipe uses eight GPUs with tensor parallelism. The NVFP4 version runs, according to Mistral, on four Nvidia B200 or B300 GPUs (Blackwell).

Источник

The Mistral Ecosystem Open-Weight License is a modified MIT licence. Internal use, commercial use and embedding the model in a product are free, with no revenue threshold. Selling access to the model as a service requires a separate commercial licence with Mistral.

Источник

I fixed a bug in mistral-common that prevented loading a tokenizer already on disk.

Источник

  • Дата записи: 2026-11-01T18:14:01.910Z