> ## Documentation Index
> Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Client-Side Evals (SDK)

The Phoenix Evals SDK provides composable building blocks for writing and running evaluations in Python or TypeScript. This page covers the core mental model: what an evaluator is, the two evaluator types, and how input mapping works.

## What Is an Evaluator?

An evaluator is anything that takes inputs and returns a **Score**. The `Score` object is the universal output of all evaluators:

| Property      | Required | Description                                              |
| ------------- | -------- | -------------------------------------------------------- |
| `name`        | ✓        | Human-readable name of the evaluator                     |
| `kind`        | ✓        | Origin of the signal: `llm`, `code`, or `human`          |
| `direction`   | ✓        | Whether a higher score is better or worse                |
| `score`       | optional | Numeric result                                           |
| `label`       | optional | Categorical outcome (e.g. `"correct"`, `"hallucinated"`) |
| `explanation` | optional | Reasoning behind the result                              |
| `metadata`    | optional | Arbitrary extra context                                  |

Every evaluator exposes `evaluate` and `async_evaluate` methods for running on a single record, and an `input_schema` that describes what fields it needs.

## Evaluator Types

**LLM-based evaluators** use a judge model to assess qualitative criteria — things like faithfulness, toxicity, or relevance — where "correct" is subjective. The judge reads a prompt template and produces a labeled score with an explanation. See [Custom LLM Evaluators](/docs/phoenix/evaluation/how-to-evals/custom-llm-evaluators) and [Configuring the LLM](/docs/phoenix/evaluation/how-to-evals/configuring-the-llm).

**Code evaluators** use deterministic logic or heuristics — exact match, regex, Levenshtein distance — where "correct" is objective. They run without any LLM call and are fast and cheap. See [Code Evaluators](/docs/phoenix/evaluation/how-to-evals/code-evaluators).

## Input Mapping

Your data rarely matches an evaluator's expected field names exactly. Instead of reshaping your data to fit each evaluator, **input mapping** makes the evaluator fit your data.

Each evaluator has a discoverable `input_schema` that lists the fields it needs. You pass an `input_mapping` alongside your data to tell the evaluator how to extract those fields. Mapping values can be one of three types:

* **Key mapping** — a plain string that maps directly to a top-level key in your input: `"response"`
* **Path mapping** — a dot-path string that traverses nested structures and arrays using [JSONPath](https://www.rfc-editor.org/rfc/rfc9535.html) syntax: `"output.response"`, `"messages[0].content"`
* **Callable** — a function that receives the full input and returns the value, for transforms that can't be expressed as a path: `lambda x: " ".join(x["documents"])`

<Tabs>
  <Tab title="Python" icon="python">
    ```python theme={null}
    eval_input = {
        "input": {"query": "What is photosynthesis?", "documents": ["doc A", "doc B"]},
        "output": {"response": "Photosynthesis converts sunlight to energy."},
    }

    # Map evaluator field names → paths into your data
    input_mapping = {
        "input": "input.query",        # dot notation for nested keys
        "context": lambda x: " ".join(x["input"]["documents"]),  # callable for transforms
        "output": "output.response",
    }

    scores = faithfulness_evaluator.evaluate(eval_input, input_mapping)
    ```
  </Tab>

  <Tab title="TypeScript" icon="js">
    ```typescript theme={null}
    import { bindEvaluator, createFaithfulnessEvaluator } from "@arizeai/phoenix-evals";
    import { openai } from "@ai-sdk/openai";

    const evaluator = bindEvaluator(
      createFaithfulnessEvaluator({ model: openai("gpt-4o") }),
      {
        inputMapping: {
          input: "input.query",
          context: (data) => data.input.documents.join(" "),
          output: "output.response",
        },
      }
    );

    const scores = await evaluator.evaluate(evalInput);
    ```
  </Tab>
</Tabs>

### The Bind Pattern

When you want to reuse the same evaluator with the same mapping across many records (for example, batch eval runs or inside an experiment), **bind** the mapping once:

<Tabs>
  <Tab title="Python" icon="python">
    ```python theme={null}
    from phoenix.evals import bind_evaluator

    bound = bind_evaluator(faithfulness_evaluator, {
        "input": "input.query",
        "context": lambda x: " ".join(x["input"]["documents"]),
        "output": "output.response",
    })

    # Now call it with just the data — mapping is baked in
    scores = bound(eval_input)
    ```
  </Tab>

  <Tab title="TypeScript" icon="js">
    ```typescript theme={null}
    // bindEvaluator returns a pre-configured evaluator
    const bound = bindEvaluator(faithfulness_evaluator, {
      inputMapping: { input: "input.query", output: "output.response" },
    });

    const scores = await bound.evaluate(evalInput);
    ```
  </Tab>
</Tabs>

See [Custom LLM Evaluators](/docs/phoenix/evaluation/how-to-evals/custom-llm-evaluators) for deeper examples.

## Sync vs Async

Use `evaluate` for simple scripts or notebooks. Use `async_evaluate` when you're running many evaluations concurrently — the executor underneath handles rate limits, retries, and dynamic concurrency automatically.

For running evaluations over a full dataframe, use `async_evaluate_dataframe`. See [Batch Evaluations](/docs/phoenix/evaluation/how-to-evals/batch-evaluations) for the full workflow.

## Next Steps

<CardGroup cols={2}>
  <Card title="Custom LLM Evaluators" icon="wand-magic-sparkles" href="/docs/phoenix/evaluation/how-to-evals/custom-llm-evaluators">
    Build classification and scoring evaluators with prompt templates
  </Card>

  <Card title="Code Evaluators" icon="code" href="/docs/phoenix/evaluation/how-to-evals/code-evaluators">
    Create deterministic evaluators using functions
  </Card>

  <Card title="Batch Evaluations" icon="table" href="/docs/phoenix/evaluation/how-to-evals/batch-evaluations">
    Run evaluations efficiently over dataframes
  </Card>

  <Card title="Pre-Built Metrics" icon="box" href="/docs/phoenix/evaluation/pre-built-metrics">
    Use pre-tested evaluators for faithfulness, relevance, and more
  </Card>
</CardGroup>