PowerInfer

SJTU open-source hybrid CPU/GPU LLM inference engine using hot/cold neuron split — 11x faster local inference on consumer GPUs.

View on AIWEBTOOLS.AI