Serving LFM2.5-24B-A2B on one 48 GB GPU: the flags, the traps and what to measure

Description and use

Deploying LFM2.5‑24B‑A2B on a single 48 GB GPU with vLLM 0.23.0, correct parsers, and no max_tokens limit to measure latency and throughput.

Problem

Deploying the 24B‑parameter LFM2.5‑24B‑A2B model on a single 48 GB GPU while managing memory, parsing, and concurrency constraints.

Outcome

The workflow enables LFM2.5‑24B‑A2B to run on a single 48 GB GPU with correct parsing and concurrency handling, preparing for empirical measurement of latency and throughput.

Workflow

• 1. Pin vLLM to 0.23.0 or newer to support LFM2.5.
• 2. Launch vLLM with command: `vllm serve LiquidAI/LFM2.5-24B-A2B-FP8 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser lfm2`
• 3. Ensure max_tokens is unset unless a hard ceiling is needed.
• 4. Use default sampling config (temperature 0.1, top_k 50, repetition_penalty 1.05) or override via extra_body.
• 5. Run the provided bench.py script to measure first‑visible‑token latency, tokens per second, and concurrency effects.
• 6. Monitor VRAM usage: ~23 GB free after weights, plus vLLM overhead and KV cache (~2.7 GB per 131,072‑token conversation).

Evidence

Liquid AI built LFM2.5-24B-A2B for agents on your own hardware. It has 24.0B parameters in total, the learned numbers that make up the model.

Source

vLLM runs it natively from 0.23.0, but needs telling how to split that output: `vllm serve LiquidAI/LFM2.5-24B-A2B-FP8 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser lfm2`

Source