Inside BitRun v0.2: How I Engineered a Terminal Cockpit for 1-Bit LLMs with Live Power Telemetry

说明与用途

BitRun v0.2.0‑beta 提供带实时功耗遥测的终端控制台，并支持完全在 CPU 内存中运行的 8‑B 模型。

问题

Developers running local language models on laptops lack real-time power consumption metrics and an interactive command‑center interface.

结果

BitRun v0.2.0‑beta runs 8‑B ternary models entirely on CPU memory (~2 GB RAM), achieves 21 tokens/sec on Apple Silicon and 4 tokens/sec on Windows for bonsai‑8b, while maintaining active power at ~11–22 Watts and energy per token between 0.316–5.650 Joules.

工作流

• 1. Rebuild inference runtime on llama.cpp for SIMD control
• 2. Implement dynamic legacy type remapping to load 8‑B ternary checkpoints
• 3. Standardize row stride calculations to prevent memory corruption
• 4. Add ARM NEON and AVX2 kernels for platform‑specific acceleration
• 5. Build dual‑panel cockpit with left chat panel and right telemetry dashboard
• 6. Pipe runtime logs to ~/.bitrun/engine.log
• 7. Publish pre‑compiled binaries for macOS, Windows, Linux
• 8. Provide install scripts and cockpit launch commands

证据

Running bitrun cockpit bonsai-1.7b launches a dual-panel terminal interface designed specifically for low-power inference

来源

In this release, BitRun runs PrismML's 8-billion parameter bonsai-8b entirely in CPU memory using around 2.08 GB of RAM

来源

Apple Silicon benefits from dedicated ARM NEON dot-product intrinsics in arch/arm/quants.c

来源