A new open-source framework called Mesh LLM is enabling developers to run large language models across distributed, peer-to-peer GPU networks. By combining the iroh networking library with a decentralized inference architecture, the project allows teams to pool existing hardware, eliminate recurring cloud costs, and maintain full control over data and compute environments. Traditional AI deployment relies on centralized data centers and metered APIs, forcing users to cede control over model updates, hardware selection, and data privacy. Mesh LLM addresses this by aggregating idle or dedicated GPUs across multiple machines into a single, unified compute cluster. The system exposes a standard OpenAI-compatible API endpoint at localhost:9337/v1, allowing any compatible client to route prompts without awareness of the underlying distributed infrastructure. At its core, Mesh LLM utilizes a custom gossip protocol built atop iroh’s authenticated QUIC transport to manage peer discovery, routing, and secure tunneling. Because iroh handles NAT traversal and relay fallbacks, nodes can establish direct, low-latency connections regardless of network topology. The framework supports two distinct networking modes: a primary mesh channel for routing and plugin communication, and a secondary channel optimized for latency-sensitive activation transport. Inside the main channel, a single byte demultiplexes bidirectional QUIC streams, separating gossip announcements, HTTP inference tunnels, routing requests, and peer lifecycle events. To run models that exceed individual hardware capabilities, Mesh LLM employs a layer-partitioning feature internally referred to as Skippy. This mechanism splits large models across multiple nodes in a processing pipeline, with each node handling a specific layer range. Activations flow sequentially between stages, enabling modest machines to jointly execute models with hundreds of billions of parameters. The architecture remains highly modular, with a catalog of over forty preconfigured models ranging from lightweight laptop-compatible variants to large mixture-of-experts architectures. Plugin support allows developers to extend capabilities across MCP, HTTP, and inference interfaces. Deployment is designed for accessibility, with a lightweight client footprint of approximately eighteen megabytes. Users can join a public mesh or configure private, access-controlled deployments. Ownership attestation and version compatibility checks are enforced at the protocol level, ensuring that only trusted peers and compatible software versions participate in the network. An upcoming mobile application, built on iroh’s Swift SDK, will introduce support for the Agent Communication Protocol, expanding interoperability across decentralized AI clients. By replacing centralized API dependencies with a scalable, peer-to-peer inference layer, Mesh LLM offers a practical path toward cost-effective, sovereign AI deployment. The project remains fully open source, with documentation, source code, and community channels publicly available for developers seeking to decentralize large language model workloads.