vLLM

vLLM is a high-throughput inference and serving engine for LLMs with an OpenAI-compatible server. Use Phoenix to trace vLLM server spans and client calls.

vLLM

Website:

vLLM Tracing