GitHub Copilot

Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Phoenix.

Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Phoenix for full observability.

Trace your GitHub Copilot sessions — in VS Code or the CLI — in Phoenix with the coding-harness-tracing toolkit. Every turn shows up as a trace, with each tool call and any subagents captured as nested spans and grouped into its session. No application code changes required: the toolkit hooks into Copilot’s own events and streams OpenInference spans to Phoenix. VS Code uses per-event hook files; the CLI uses a single hooks.json.

Launch Phoenix

The fastest way to get started with Phoenix is by signing up for a free Phoenix Cloud account. If you prefer, you can also run Phoenix in a notebook, self-host it, or use it directly from your terminal.

Go to the settings page in your Phoenix instance to find your endpoint and API key. A self-hosted Phoenix defaults to http://localhost:6006; the API key is only required when auth is enabled.

Install

Copilot hooks are installed at the project level under .github/hooks/. Run the installer from the root of each repository where you want tracing.

The curl installer is the simplest — it runs a short wizard that saves your Phoenix credentials for you. Use a local clone if you’d rather run the installer from a checkout of the source.

Curl installer (recommended)

macOS / Linux:

cd /path/to/your/project
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- copilot

Windows (PowerShell):

cd C:\path\to\your\project
iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat copilot

Local clone

git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd /path/to/your/project
/path/to/coding-harness-tracing/install.sh copilot         # macOS / Linux
\path\to\coding-harness-tracing\install.bat copilot        # Windows

The installer prompts for your backend — select Phoenix, then enter your endpoint and optional API key — and your project name, writes them to ~/.arize/harness/config.json, and registers VS Code (.github/hooks/*.json) and Copilot CLI (.github/hooks/hooks.json) hooks in the current project.

Configuration

Credentials live in ~/.arize/harness/config.json and apply across all projects. Per-project environment variables override config.json and can be set in your shell profile.

export PHOENIX_ENDPOINT="http://localhost:6006"
export PHOENIX_API_KEY="<your-api-key>"   # optional, only if auth is enabled
export PHOENIX_PROJECT="copilot"
export ARIZE_TRACE_ENABLED="true"

On the Phoenix backend, set the project name with PHOENIX_PROJECT (or PHOENIX_PROJECT_NAME); ARIZE_PROJECT_NAME is Arize-only and ignored here. ARIZE_TRACE_ENABLED is a backend-agnostic harness setting and keeps the ARIZE_ prefix regardless of destination.

Redaction controls

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category:

export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"

Flag	Redacts
ARIZE_LOG_PROMPTS	User prompt and assistant response text
ARIZE_LOG_TOOL_DETAILS	Tool names and arguments
ARIZE_LOG_TOOL_CONTENT	Tool call output content

Observe

Once tracing is enabled, Copilot activity from both VS Code and the CLI streams to Phoenix. You’ll see:

• Turn traces — each prompt → response cycle as a parent span
• Tool spans — nested spans for each tool call with inputs, outputs, and duration
• Subagent spans — activity from any subagents Copilot spawns (VS Code)
• Error spans — errorOccurred events from the CLI
• Session grouping — all turns from the same session grouped by session_id

Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the Copilot tracing README.

Uninstall

Run from the project root where you installed the hooks:

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall copilot

Resources

Arize Coding Harness Tracing

OpenInference

GitHub Copilot