Ollama-shape inference manager using Tom's TurboQuant llama.cpp. FIFO queue + grace + IDLE_HOT hot-hold + model swap on Blackwell.
Python Updated 6 months ago 1 commits 4 stars
prism-dev|Mirror: MrTrenchTrucker/turbohaul-manager (⭐4)
6 months agoE2B SandboxFirecracker microVMpython
No executable code files found in this repository.
About
Ollama-shape inference manager using Tom's TurboQuant llama.cpp. FIFO queue + grace + IDLE_HOT hot-hold + model swap on Blackwell.
Readme
4 stars