Groq Tracing

Instrument LLM applications built with Groq

Groq provides low latency and lightning-fast inference for AI models. Arize supports instrumenting Groq API calls, including role types such as system, user, and assistant messages, as well as tool use. You can create a free GroqCloud account and generate a Groq API Key here to get started.

Install

pip install openinference-instrumentation-groq groq

Setup

Connect your application to Phoenix with the register function:

Run Groq

A simple Groq application that is now instrumented

import os
from groq import Groq

client = Groq(
    # This is the default and can be omitted
    api_key=os.environ.get("GROQ_API_KEY"),
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="mixtral-8x7b-32768",
)
print(chat_completion.choices[0].message.content)

Observe

Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.

Resources:

• Example Chat Completions

• Example Async Chat Completions

• Tutorial

• OpenInference package