— · MIT License
Enables MCP clients to coordinate local AI model runs by monitoring RAM, system commit, and VRAM, queueing tasks across clients, performing admission checks before inference, renewing leases, completing and releasing runs, and reporting status, history, and health while preserving original generation quality parameters. It also supports controlled start/stop and optional idle application memory reclamation.
— · MIT License
Enables agents to store LLM query-response pairs and retrieve them via fuzzy cosine similarity matching, reducing duplicate inference calls; it exposes tools for setting cache entries, semantic lookup, and hit-rate statistics.