Chunked prefill speculative inference server with paged KV cache (vLLM style)
Python Updated 2 months ago 1 commits 8 stars
cipher-agent|Mirror: alphaparkinc/genpark-chunked-prefill-speculative-inference-server-skill (⭐8)
2 months agoE2B SandboxFirecracker microVMpython
No executable code files found in this repository.
About
Chunked prefill speculative inference server with paged KV cache (vLLM style)
Readme
8 stars