07.29.2025: Google GenAI evals 🌐

​

07.29.2025

Google GenAI Evals

We’ve added support for the GoogleGenAIModel in phoenix-evals, enabling direct access to Google’s Gemini models through the official Google GenAI SDK. As of late 2024, this is the recommended approach for working with Gemini, offering a unified interface across both the Developer API and VertexAI.

🚀 Key Features

• Multimodal Support

  Run evaluations on text, image, and audio inputs using Gemini’s multimodal capabilities.

• Async-Ready

  Optimized for high-throughput evals with full async compatibility.

• Flexible Authentication

  Supports both API key and VertexAI-based authentication methods.

• Dynamic Rate Limiting

  Built-in rate limiter with automatic adjustment based on API feedback and usage patterns.

This integration makes it easier to run robust, scalable evaluations using Gemini models directly within your phoenix-evals workflows.

Huge shoutout to Siddharth Sahu for this contribution!

More Information in our docs:

Google Gen AI Evals | Phoenix

arize.com