Prompts Concepts

What a Phoenix prompt contains, and how templates, versions, tags, formats, tools, and response formats fit together.

​

Prompt

In general use, a prompt is the content you send to an LLM, such as the text you send to OpenAI’s gpt-4. Phoenix expands this definition. In Phoenix, a prompt is everything that you need to invoke an LLM:

• The prompt template of the messages to send to a completion endpoint
• The invocation parameters (temperature, frequency penalty, etc.)
• The tools that the LLM can use (for example, a weather API)
• The response format (also called the output schema) that you use when JSON mode is on

Phoenix saves a snapshot of all of these parts together. Because of this, your application invokes the LLM with the same configuration each time, and the result is more deterministic.

A phoenix prompt captures everything needed to invoke an LLM

​

Prompt Templates

People often use the terms “prompt” and “prompt template” as if they mean the same thing. They are different.

A prompt is the message or messages that go to the language model.

A prompt template is a format that puts the information you want (such as context and examples) into the prompt. A prompt template can contain placeholders (variables). At runtime, the variables are filled with data such as few-shot examples, outside context (RAG), or other external data.

Prompt templates have placeholders for variables that are dynamically filled at runtime

​

Prompt Version

Each time you save a prompt in Phoenix, Phoenix saves a snapshot of it as a prompt version. Versions have two purposes:

• You can see how a prompt changed over time.
• You can test a specific prompt version and become confident in it before your application uses it.

For each prompt version, Phoenix records the author and the date when the version was saved.

Git tracks changes to code with commit SHAs. In the same way, Phoenix tracks each change to a prompt with a prompt_id.

Prompt version tags

​

Prompt Version Tag

A prompt version tag is a label on a specific prompt version. Tags let you control when and where a prompt version is deployed. A tag marks a version as important, and the tagged version does not change. Because of this, you can refer to it later with confidence.

Applications often move through different environments before they go live, for example:

1. Development: where you build new features.
2. Staging: where you test.
3. Production: the live system that users interact with.

If you tag prompt versions with environment tags, you can build, test, and deploy prompts in the same way as the rest of the application. Each prompt change then goes through the same systematic testing and deployment.

You can also use custom tags, as teams do with custom Git tags on code (for example, v0.0.1). Custom tags can show a deployment stage, feature readiness, or any other status that fits your workflow.

Prompt version tags work in the same way as git tags. To create and use tags, see Tag a prompt.

​

Prompt Format

A prompt template can include any attribute from spans or datasets. You add attributes as variables in one of two formats: F-String or Mustache.

F-String variables use single {s:

{question}

To escape a { in F-String format, add a second { in front of it, e.g., {{escaped}} {not-escaped}. Escaped variables are removed from the inputs in the Playground.

Mustache variables use double {{s:

{{question}}

We recommend Mustache where possible, because it has better support for nested attributes, e.g. attributes.input.value.

Phoenix supports the full Mustache template syntax, including sections and inverted sections. Because of this, you can write more complex prompt templates when necessary.

​

Tools

Tools let an LLM interact with the external environment. With tools, the LLM can interface with your application in more controlled ways. When an LLM gets a prompt and a set of tools, it can use one tool, many tools, or no tools. Many LLM APIs also have a tool choice parameter. This parameter controls how the LLM selects tools and which tools it can select.

This is an example of a tool for a weather API, in OpenAI format:

{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA",
                },
            },
            "required": ["location"],
        }
    }
}

Phoenix also supports provider tools. These are built-in tools that are specific to one vendor, such as Anthropic web search, OpenAI Responses file_search and code_interpreter, Gemini google_search grounding, and Amazon Nova nova_grounding. Phoenix sends provider tools to the provider as opaque JSON. For the JSON shape for each provider, see Use provider tools.

​

Response Format

Some LLMs support structured responses. This feature is called response format or output schema. It lets you specify an exact schema for the output of the model.

With Structured Outputs, the model always generates responses that match a defined JSON Schema. This prevents problems such as missing keys or values that are not valid.

​

Benefits of Structured Outputs:

• Reliable type-safety: You do not have to validate responses or retry responses that have an incorrect format.
• Explicit refusals: Your code can detect refusals that the model makes for safety reasons.
• Simpler prompting: You do not need strongly worded prompts to get a consistent format.

For more details, see this OpenAI guide.