NVIDIA TensorRT-LLM
NVIDIA's open-source SDK for ultra-optimized LLM inference on H100/A100 GPUs. State-of-the-art throughput with FP8, in-flight batching, and KV cache management.
Tags
Open Source · Inference · TensorRT-LLM · NVIDIA · GPU · FP8 · Production · Simulation · NVIDIA TensorRT-LLM · Free · free tools · free ai tools