NodeShare AI: Building a Distributed AI Compute Network from Existing PCs

Description and use

NodeShare AI enables developers to harness their own PCs for distributed inference of open-weight AI models, reducing reliance on paid cloud services.

Problem

Developers often rely on paid hosted AI APIs or dedicated GPU servers to experiment with AI, limiting flexibility and cost.

Outcome

NodeShare AI demonstrates a proof‑of‑concept for distributed AI inference using existing PCs, exposing a REST API for job dispatch and local inference via llama.cpp.

Workflow

• A client sends an inference request to the cluster controller API.
• The controller evaluates registered node information and selects a suitable compute node.
• The controller creates and dispatches a job to the chosen node.
• The node agent runs the local inference runtime (llama.cpp) on the node.
• The inference result is returned to the client via the controller API.

Evidence

NodeShare AI is an experimental distributed AI inference platform designed to help developers explore running open-weight AI models using compute resources across their own computers.

Source

The project is intended for independent developers, students, freelancers, and small teams who want to experiment with self-hosted AI.

Source

The intended execution flow is: 1) A client sends an inference request to the controller. 2) The controller evaluates available node information and attempts to select a suitable compute node. 3) The controller creates and dispatches a job. 4) The node agent and local inference runtime are intended to execute the workload. 5) The result is returned through the API.

Source

The project contains the implementation of the Python controller and API layer.

Source