LightLLM

Category: AI Development & Platforms

Open-source pure-Python LLM inference framework with token-level scheduling and async I/O — designed for lightweight high-throughput serving on commodity GPUs.

Open LightLLM →

Tags

Open Source · Inference · LightLLM · Python · High Throughput · Self-Hosted · Simulation · Free · free tools · free ai tools · no cost

Similar AI tools in AI Development & Platforms

• Qwen3
• DeepSeek V3
• Mistral Large 2
• Phi-4
• Gemma 3
• Command R+
• Falcon 3
• Yi-Large
• OLMo 2
• Jan
• GPT4ALL
• Cherry Studio