> ## Documentation Index
> Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Precision / Recall / F-Score

> Compute precision, recall, and F-beta scores for classification tasks

## Overview

The **PrecisionRecallFScore** evaluator computes precision, recall, and F-beta scores for comparing predicted labels against expected labels. It supports both binary and multi-class classification with various averaging strategies, and is available in Python and TypeScript.

### When to Use

Use the PrecisionRecallFScore evaluator when you need to:

* **Evaluate classification performance** - Measure how well your model predicts correct labels
* **Compare label sequences** - Assess predicted vs expected labels for multi-item outputs
* **Binary classification metrics** - Compute metrics for spam/ham, positive/negative, etc.
* **Multi-class evaluation** - Evaluate across multiple categories with different averaging strategies

<Info>
  This is a code-based evaluator that computes standard classification metrics. Both `expected` and `output` should be sequences of labels (strings or integers) — the full sequence across a dataset, not a single row's label.
</Info>

## How These Metrics Work

Every prediction for a given class falls into one of four buckets: **true positive** (TP, correctly predicted that class), **false positive** (FP, predicted that class but it wasn't), **false negative** (FN, was that class but predicted something else), and true negative (not that class and not predicted as it). Precision, recall, and F-beta are all built from TP, FP, and FN:

| Metric        | Formula                                                             | Answers                                                                      |
| ------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| **Precision** | `TP / (TP + FP)`                                                    | Of everything predicted as this class, what fraction actually was?           |
| **Recall**    | `TP / (TP + FN)`                                                    | Of everything that actually is this class, what fraction did the model find? |
| **F-beta**    | `(1 + beta^2) * precision * recall / (beta^2 * precision + recall)` | The weighted harmonic mean of precision and recall.                          |

Precision and recall trade off against each other: a model that predicts the positive class more aggressively tends to raise recall (catches more true positives) at the cost of precision (more false alarms), and vice versa. Which one matters more is a property of the task, not the model:

* **Spam filtering** — a false positive (a real email marked as spam) is usually worse than a false negative (spam that slips through), so precision matters more.
* **Medical screening / fraud detection** — a false negative (a missed disease or fraudulent transaction) is usually worse than a false positive (an unnecessary follow-up), so recall matters more.

**F-beta** (the F-measure introduced by Van Rijsbergen, see [References](#references)) combines them into a single number, where `beta` sets how much more recall is weighted than precision: `beta = 1` (F1, the default) weights them equally, `beta > 1` (e.g. F2) favors recall, and `beta < 1` (e.g. F0.5) favors precision. Because it's a harmonic mean, F-beta stays low if *either* precision or recall is low.

### Averaging Strategies

Precision, recall, and F-beta are inherently per-class metrics. For multi-class classification, `average` controls how the per-class scores combine into a single number:

| Strategy   | Description                                                                                        | Good for                                                                                                                                                                                        |
| ---------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `macro`    | Compute each class's metric independently, then take the unweighted mean.                          | Surfacing whether a rare class is being ignored — every class counts equally regardless of how often it occurs.                                                                                 |
| `weighted` | Like `macro`, but each class's metric is weighted by its support (how often it actually occurs).   | Reflecting overall performance when class frequencies are meaningful and imbalance is expected.                                                                                                 |
| `micro`    | Pool every class's TP, FP, and FN first, then compute one precision/recall/F-beta from the totals. | A single aggregate number. For single-label multi-class problems (exactly one predicted and one expected label per example), micro precision, recall, and F1 are all equal to overall accuracy. |

`macro` and `weighted` compute F-beta per class first and then average the per-class F-beta values — they don't derive F-beta from the averaged precision and recall. This matches [scikit-learn's `precision_recall_fscore_support`](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html) semantics (see [References](#references)), so results are directly comparable to the equivalent `sklearn.metrics` call.

<Note>
  For numeric labels `{0, 1}` with the default `"macro"` average and no `positive_label`/`positiveLabel` configured, the evaluator automatically treats `1` as the positive class and computes binary (one-vs-rest) metrics instead of multi-class averaging. Configuring a non-default `average` skips this auto-detection, so an explicitly requested averaging strategy is never silently overridden by the shape of the data.
</Note>

## Supported Levels

This evaluator is not tied to specific tracing levels. It operates on lists of predicted and expected labels, making it useful for:

* Comparing model predictions against ground truth labels
* Evaluating classification outputs at any level where you have paired label sequences
* Batch evaluation of classification tasks in experiments

## Input Requirements

The evaluator requires two inputs:

| Field      | Type               | Description                  |
| ---------- | ------------------ | ---------------------------- |
| `expected` | `List[str \| int]` | List of expected/true labels |
| `output`   | `List[str \| int]` | List of predicted labels     |

Both sequences must have the same length and contain at least one element.

### Constructor Arguments

<Tabs>
  <Tab title="Python" icon="python">
    | Argument         | Type         | Default   | Description                                               |
    | ---------------- | ------------ | --------- | --------------------------------------------------------- |
    | `beta`           | `float`      | `1.0`     | Weight of recall relative to precision (F1 by default)    |
    | `average`        | `str`        | `"macro"` | Averaging strategy: `"macro"`, `"micro"`, or `"weighted"` |
    | `positive_label` | `str \| int` | `None`    | For binary classification, specify the positive class     |
    | `zero_division`  | `float`      | `0.0`     | Value to use when a metric is undefined (0/0)             |
  </Tab>

  <Tab title="TypeScript" icon="js">
    | Option          | Type                               | Default     | Description                                                                                                                  |
    | --------------- | ---------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------- |
    | `beta`          | `number`                           | `1`         | Weight of recall relative to precision. Only on `createFBetaEvaluator` (F1 is `createF1Evaluator`, equivalent to `beta: 1`). |
    | `average`       | `"macro" \| "micro" \| "weighted"` | `"macro"`   | Averaging strategy                                                                                                           |
    | `positiveLabel` | `string \| number`                 | `undefined` | For binary classification, specify the positive class                                                                        |
    | `zeroDivision`  | `number`                           | `0`         | Value to use when a metric is undefined (0/0)                                                                                |
  </Tab>
</Tabs>

## Output Interpretation

<Tabs>
  <Tab title="Python" icon="python">
    The evaluator returns three `Score` objects:

    | Score Name          | Description                                    |
    | ------------------- | ---------------------------------------------- |
    | `precision`         | Ratio of true positives to predicted positives |
    | `recall`            | Ratio of true positives to actual positives    |
    | `f1` (or `f{beta}`) | Harmonic mean of precision and recall          |

    All scores have `direction = "maximize"` (higher is better) and `kind = "code"` (code-based evaluator).
  </Tab>

  <Tab title="TypeScript" icon="js">
    Each factory returns an evaluator whose `.evaluate({ expected, output })` resolves to a single `{ score }`:

    | Evaluator                                                | `score` is  |
    | -------------------------------------------------------- | ----------- |
    | `createPrecisionEvaluator()`                             | Precision   |
    | `createRecallEvaluator()`                                | Recall      |
    | `createF1Evaluator()` / `createFBetaEvaluator({ beta })` | F1 / F-beta |

    `createPrecisionRecallFScoreEvaluators()` bundles matching `{ precision, recall, fScore }` evaluators from one options object. Every evaluator has `kind: "CODE"` and `optimizationDirection: "MAXIMIZE"`, and its `name` reflects the configuration, e.g. `precision`, `recall_weighted`, `f1`, `f0_5_micro`.
  </Tab>
</Tabs>

## Usage Examples

<Tabs>
  <Tab title="Python" icon="python">
    ```python theme={null}
    from phoenix.evals.metrics import PrecisionRecallFScore

    # Create evaluator with default settings (F1, macro averaging)
    evaluator = PrecisionRecallFScore()

    # Inspect the evaluator's requirements
    print(evaluator.describe())

    # Multi-class evaluation
    eval_input = {
        "expected": ["cat", "dog", "cat", "bird", "dog"],
        "output": ["cat", "cat", "cat", "bird", "dog"]
    }

    scores = evaluator.evaluate(eval_input)
    for score in scores:
        print(f"{score.name}: {score.score:.3f}")
    # precision: 0.889
    # recall: 0.833
    # f1: 0.822
    ```
  </Tab>

  <Tab title="TypeScript" icon="js">
    ```typescript theme={null}
    import {
      createPrecisionRecallFScoreEvaluators,
    } from "@arizeai/phoenix-evals/code";

    // Matching precision/recall/F1 evaluators sharing one options object
    // (F1, macro averaging by default)
    const { precision, recall, fScore } = createPrecisionRecallFScoreEvaluators();

    // Multi-class evaluation
    const example = {
      expected: ["cat", "dog", "cat", "bird", "dog"],
      output: ["cat", "cat", "cat", "bird", "dog"],
    };

    console.log("precision:", (await precision.evaluate(example)).score?.toFixed(3));
    console.log("recall:", (await recall.evaluate(example)).score?.toFixed(3));
    console.log("f1:", (await fScore.evaluate(example)).score?.toFixed(3));
    // precision: 0.889
    // recall: 0.833
    // f1: 0.822
    ```
  </Tab>
</Tabs>

### Binary Classification

For binary classification, specify the positive label:

<Tabs>
  <Tab title="Python" icon="python">
    ```python theme={null}
    from phoenix.evals.metrics import PrecisionRecallFScore

    # Binary classification for spam detection
    evaluator = PrecisionRecallFScore(positive_label="spam")

    eval_input = {
        "expected": ["spam", "ham", "spam", "ham", "spam"],
        "output": ["spam", "spam", "ham", "ham", "spam"]
    }

    scores = evaluator.evaluate(eval_input)
    for score in scores:
        print(f"{score.name}: {score.score:.3f}")
    # precision: 0.667  (2 TP / 3 predicted spam)
    # recall: 0.667     (2 TP / 3 actual spam)
    # f1: 0.667
    ```
  </Tab>

  <Tab title="TypeScript" icon="js">
    ```typescript theme={null}
    import {
      createPrecisionEvaluator,
      createRecallEvaluator,
      createF1Evaluator,
    } from "@arizeai/phoenix-evals/code";

    // Binary classification for spam detection
    const precision = createPrecisionEvaluator({ positiveLabel: "spam" });
    const recall = createRecallEvaluator({ positiveLabel: "spam" });
    const f1 = createF1Evaluator({ positiveLabel: "spam" });

    const example = {
      expected: ["spam", "ham", "spam", "ham", "spam"],
      output: ["spam", "spam", "ham", "ham", "spam"],
    };

    console.log("precision:", (await precision.evaluate(example)).score?.toFixed(3));
    console.log("recall:", (await recall.evaluate(example)).score?.toFixed(3));
    console.log("f1:", (await f1.evaluate(example)).score?.toFixed(3));
    // precision: 0.667  (2 TP / 3 predicted spam)
    // recall: 0.667     (2 TP / 3 actual spam)
    // f1: 0.667
    ```

    A runnable version of this example, plus F-beta and all three averaging strategies, lives in [`examples/classification_metrics_example.ts`](https://github.com/Arize-ai/phoenix/blob/main/js/packages/phoenix-evals/examples/classification_metrics_example.ts).
  </Tab>
</Tabs>

## Using with Phoenix

### Evaluating Traces

Run evaluations on traces collected in Phoenix and log results as annotations:

* [Evaluating Phoenix Traces](/docs/phoenix/tracing/how-to-tracing/feedback-and-annotations/evaluating-phoenix-traces)

### Running Experiments

Use the PrecisionRecallFScore evaluator in Phoenix experiments:

* [Using Evaluators in Experiments](/docs/phoenix/datasets-and-experiments/how-to-experiments/using-evaluators)

<Note>
  The PrecisionRecallFScore evaluator is dataset-level (batch), not per-row: `expected`/`output` are the full sequence of labels across every example you want to score together. Collect every row's expected and predicted label first, then call the evaluator once over the full arrays — don't wire it in as a per-row experiment evaluator.
</Note>

## References

* Van Rijsbergen, C.J. (1979). *Information Retrieval* (2nd ed.). Butterworth-Heinemann. — origin of the F-measure and its `beta` parameter.
* [Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) — Wikipedia overview of the underlying concepts and terminology.
* [`sklearn.metrics.precision_recall_fscore_support`](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html) — scikit-learn's reference implementation; the `average` strategies here follow the same semantics.

## API Reference

* **Python**: [PrecisionRecallFScore](https://arize-phoenix.readthedocs.io/projects/evals/api/evals.html#module-phoenix.evals.metrics)
* **TypeScript**: [Classification Metrics](/docs/phoenix/sdk-api-reference/typescript/packages/phoenix-evals/classification-metrics)

## Related

* [Exact Match Evaluator](/docs/phoenix/evaluation/pre-built-metrics/exact-match) - For exact string comparison
* [Correctness Evaluator](/docs/phoenix/evaluation/pre-built-metrics/correctness) - For semantic correctness evaluation