LMDeploy

Category: AI Tools To Run LOCALLY

InternLM's production-grade toolkit for quantizing, deploying, and serving LLMs at scale. TurboMind backend delivers industry-leading tokens/sec for multi-GPU inference.

Open LMDeploy →

Tags

Open Source · Inference · LLM Serving · Quantization · Self-Hostable · LMDeploy · Free · free tools · free ai tools · no cost

Similar AI tools in AI Tools To Run LOCALLY

• ExLlamaV2
• MLC LLM
• WasmEdge
• LocalRecall
• Neural Compressor
• Optimum
• AutoAWQ
• GPTQModel
• llm-awq
• BitsAndBytes
• Liger Kernel
• OpenVINO