I built an offline P2P system to run 70B LLMs by pooling mismatched consumer laptops

Description and use

An open-source, zero-dependency, offline P2P inference broker enables heterogeneous consumer laptops to pool their combined RAM, VRAM, and GPU compute to run large models without cloud dependencies, with automatic layer splitting, dynamic peer discovery, and fault-tolerant local fallback to smaller models when peers disconnect.

Problem

Running massive, smart open-source AI models (e.g., 70B LLMs) natively on a standard laptop causes Out of Memory crashes, stuttering, or system unresponsiveness due to insufficient RAM and VRAM.

Outcome

An open-source, zero-dependency, offline P2P inference broker enables heterogeneous consumer laptops to pool their combined RAM, VRAM, and GPU compute to run large models without cloud dependencies, with automatic layer splitting, dynamic peer discovery, and fault-tolerant local fallback to smaller models when peers disconnect.

Workflow

• Start background daemon on a hotspot LAN using UDP Multicast Broadcasting to detect peers and map cluster topology dynamically
• Run network performance profile loop (blasting a 10MB test payload) before sessions begin to balance workload based on hardware capacity
• Node 0 loads and computes first set of layers, serializes intermediate activation tensors, and streams them over TCP sockets to Node 1
• If a peer drops off or battery dies, hotswap execution target down to a highly optimized local standalone model (e.g., Qwen 2.5 Coder 1.5B) within host machine's internal memory
• Launch cluster locally via npm: npm install -g airdrop-llm airdrop start

Evidence

Running massive, smart open-source AI models natively on a standard laptop is a painful experience. If you try to load a heavy coding model on a base 16GB MacBook Air or a standard gaming laptop, the system immediately runs out of RAM, stutters, or throws a brutal Out of Memory crash.

Source

It is a zero-dependency, offline, multi-node pipeline-parallel inference broker written in Java. It allows resource-constrained edge devices to pool their hardware resources (RAM, VRAM, and GPU compute) with other heterogeneous consumer machines over a local smartphone hotspot or raw Wi-Fi Direct network—100% offline, with no cloud dependencies.

Source

Automated P2P Discovery: When the background daemon starts up on a hotspot LAN, it uses UDP Multicast Broadcasting to instantly detect other peers, exchange local hardware specs, and map the cluster topology dynamically.

Source

Low-Overhead Binary Streaming: Communication is handled via raw TCP sockets optimized for low-latency binary streams. Node 0 loads and computes the first set of layers, serializes the intermediate activation tensors, and blasts them directly over the wireless pipe to Node 1 to finish the generation loop.

Source

Asymmetric Layer Allocation: The scheduling engine automatically runs a network performance profile loop (blasting a 10MB test payload) before sessions begin. It balances the workload based on hardware capacity—routing lighter execution chunks to standard system RAM and heavier allocations to dedicated high-throughput VRAM (like an Nvidia CUDA card) to prevent pipeline stalls.

Source

Fault-Tolerant Local Fallback: If a friend's laptop drops off the hotspot network or a battery dies mid-sentence, the daemon catches the socket drop instantly.