The Node App

Evolve Node app is an application that allows peers to use their own GPU or connect to a global P2P network for LLM inference and agents framework.

It supports both standalone operation and networked LLM distribution for users, based on their compute capabilities and preferences.

Local Inference and Agents Framework:

• Inference Engine: The node app uses Ollama (based on llama.cpp) inference engine natively, allowing users to run powerful LLMs locally on their GPUs. This ensures high-performance inference with minimal latency, making it ideal for real-time AI agent applications.

• Agents Framework: Users can leverage the agents framework to build and manage sophisticated workflows. This framework supports various AI-driven tasks, facilitating seamless integration and execution of agent-based models.

P2P Network Integration:

• Connect to the Network: Users have the option to connect to the Evolve network by linking their Web3 wallets to access compute hosted by peers across the globe, providing additional computational power when local resources are insufficient using libraries like hivemind and petals.

• Geo-Clustered Compute Power: The network is designed to optimize latency by clustering compute nodes geographically. This ensures that users can access fast and efficient compute resources from peers located nearby, significantly reducing network latency and improving performance.

Incentives and Compute Sharing:

• Earn Tokens: By becoming a compute provider, users can share their GPU resources with the network. The software manages the distribution of LLM model blocks, ensuring that providers contribute to inference tasks as needed. In return, they earn $EVOLVE tokens (launching soon), which can be withdrawn as USDT to their wallets.

• Decentralized Inference: The network orchestrates GPU compute power, dynamically routing inference requests to available nodes. This distributed approach ensures efficient utilization of global resources, supporting scalable and resilient AI operations.

Controlled Participation:

• Local and Networked Operation: Users can choose to operate entirely locally, utilizing their own GPU for all computations, or connect to the network to benefit from shared resources. This flexibility allows users to tailor their setup according to their compute capabilities and needs.

• Optimized Compute Routes: The software leverages a distributed hash table (DHT) to manage and optimize compute routes. This ensures that inference tasks are allocated to the most suitable nodes, balancing load and minimizing latency across the network.