Inside BitRun v0.2: How I Engineered a Terminal Cockpit for 1-Bit LLMs with Live Power Telemetry

Description and use

BitRun v0.2.0‑beta offers a terminal cockpit with live power telemetry and supports 8‑B models running entirely in CPU memory.

Problem

Developers running local language models on laptops lack real-time power consumption metrics and an interactive command‑center interface.

Outcome

BitRun v0.2.0‑beta runs 8‑B ternary models entirely on CPU memory (~2 GB RAM), achieves 21 tokens/sec on Apple Silicon and 4 tokens/sec on Windows for bonsai‑8b, while maintaining active power at ~11–22 Watts and energy per token between 0.316–5.650 Joules.

Workflow

• 1. Rebuild inference runtime on llama.cpp for SIMD control
• 2. Implement dynamic legacy type remapping to load 8‑B ternary checkpoints
• 3. Standardize row stride calculations to prevent memory corruption
• 4. Add ARM NEON and AVX2 kernels for platform‑specific acceleration
• 5. Build dual‑panel cockpit with left chat panel and right telemetry dashboard
• 6. Pipe runtime logs to ~/.bitrun/engine.log
• 7. Publish pre‑compiled binaries for macOS, Windows, Linux
• 8. Provide install scripts and cockpit launch commands

Evidence

Running bitrun cockpit bonsai-1.7b launches a dual-panel terminal interface designed specifically for low-power inference

Source

In this release, BitRun runs PrismML's 8-billion parameter bonsai-8b entirely in CPU memory using around 2.08 GB of RAM

Source

Apple Silicon benefits from dedicated ARM NEON dot-product intrinsics in arch/arm/quants.c

Source