Skip to main content
Use createEvaluator when your evaluation logic is plain TypeScript and you want a reusable evaluator object with consistent metadata and telemetry.

​
Relevant Source Files

  • src/helpers/createEvaluator.ts

​
Example

import { createEvaluator } from "@arizeai/phoenix-evals";

const exactMatch = createEvaluator(
  ({ output, expected }) => ({
    score: output === expected ? 1 : 0,
    label: output === expected ? "match" : "mismatch",
  }),
  {
    name: "exact-match",
    kind: "CODE",
  }
);

const result = await exactMatch.evaluate({
  output: "Paris",
  expected: "Paris",
});

​
What You Get

  • an evaluator name
  • evaluator kind such as CODE or LLM
  • optimization direction metadata
  • optional OpenTelemetry spans around execution

​
When To Use Code Evaluators

Code evaluators are the right fit when the scoring logic should stay deterministic, cheap, and fully under your control:
  • regex or exact-match checks
  • JSON structure validation
  • latency and cost thresholds
  • post-processing checks on existing model output
import { createEvaluator } from "@arizeai/phoenix-evals";

const lengthCheck = createEvaluator(
  ({ output }) => {
    const score = typeof output === "string" && output.length < 280 ? 1 : 0;
    return {
      score,
      label: score ? "fits-limit" : "too-long",
    };
  },
  {
    name: "response-length",
    kind: "CODE",
  }
);
  • asEvaluatorFn
  • toEvaluationResult

​
Source Map

  • src/helpers/createEvaluator.ts
  • src/helpers/asEvaluatorFn.ts
  • src/helpers/toEvaluationResult.ts
  • src/core/FunctionEvaluator.ts
  • src/core/EvaluatorBase.ts
  • src/types/evals.ts