Product 05 · Monitor / API / MCP / Dashboard · Local Slice

Goal Drift Monitor

Long-running agents drift. A task that starts focused on one goal slowly shifts across tool calls, subtasks, and multi-step work — without any system noticing. Goal Drift Monitor scores alignment turn by turn and fires webhook alerts when the drift threshold is breached.

Local Install → How It Works

Local product slice — Hosted monitoring not yet live

← Product 04: Agent Cost Router Next: Product 06 Cognitive Memory MCP Server →

The Problem

Agents drift off-goal without warning.

Long-running agents don't fail obviously. They drift. A task that starts with a clear goal — "summarize customer refund policy questions" — gradually shifts into tangential directions across dozens of tool calls, spawned subtasks, and multi-step reasoning chains. No error is thrown. The agent keeps working.

By the time a team reviews the output, the agent has been off-track for many turns. There is no record of when alignment started breaking down, no signal that fired when the drift was still recoverable, and no way to understand which turns caused the divergence.

Goal Drift Monitor adds a scoring layer that evaluates each agent turn against the original goal, records the drift score, fires a webhook alert when the threshold is breached, and keeps a full per-session history so you can see exactly where alignment started to slip.

How It Works

Session → Score → Alert

Create a goal session

Register the agent's original goal text and a drift threshold (0–1). An optional webhook URL can be supplied to receive alert payloads. The session is persisted locally and returns a session ID for all subsequent calls.

Record turns and score drift

After each agent action or message, add a turn with the turn content. The scorer compares the turn against the goal text and returns an alignmentScore (0–1) and a driftScore (1 minus alignment). Every turn result is persisted to the local session history.

Receive alerts and review history

When a turn's drift score meets or exceeds the threshold, an alert fires and an alertReceipt is stored recording delivery status, HTTP status code, and any error. Use the CLI, MCP tools, or local dashboard to review per-turn drift history, summaries, and alert receipts.

The MCP and CLI interfaces currently expose lexical scoring (TF-weighted Jaccard overlap) for predictable, API-key-free local operation. The SDK also exposes a semantic scorer backed by an LLM, which is more accurate but requires a provider API key and incurs token cost. Webhook delivery is distinct from alert threshold: alertFired means the drift threshold was breached; alertDeliveryStatus records whether the webhook was configured, delivered, or failed.

Drift Signals

What the scorer measures

Every turn scored by Goal Drift Monitor returns a structured result. The fields below are pulled directly from the scorer and monitor interfaces in the source.

Field	Type	Range / Values	Description
alignmentScore	number	0 – 1	How well the turn text aligns with the original goal. 1 = perfectly aligned; 0 = completely off-topic.
driftScore	number	0 – 1	1 minus alignmentScore. Higher value means more divergence from the goal.
method	string	`lexical` / `semantic`	Scoring method used. MCP and CLI default to lexical. Semantic requires a provider API key.
alertFired	boolean	true / false	Whether the drift score met or exceeded the session's configured drift threshold.
alertDeliveryStatus	string	not_configured / `delivered` / `failed`	Webhook delivery result. Separate from alertFired — an alert can fire even if no webhook is configured.
driftTrend	string	`stable` / `rising` / `falling`	Computed from the last three turns. Rising means drift is increasing; falling means the agent is re-aligning.
averageDrift	number	0 – 1	Mean drift score across all turns in the session. Available in the session summary.
maxDrift	number	0 – 1	Highest drift score recorded across all turns. Available in the session summary.
alertCount	number	integer ≥ 0	Number of turns in which alertFired was true for the session. Available in the session summary.

The lexical scorer uses TF-weighted Jaccard overlap after stopword removal. The default drift threshold is 0.6. Override it per session with --threshold (CLI) or drift_threshold (MCP / SDK).

Local Install

CLI, MCP server, and local dashboard

Goal Drift Monitor ships as a local CLI and MCP server. Use the CLI to create sessions, add turns, review summaries, and serve the local dashboard. Wire the MCP tools into any MCP-compatible agent runtime for automated in-process monitoring.

The local dashboard starts on http://127.0.0.1:4321/dashboard after running goal-drift serve.

View on npm →

npm package

# Install locally for MCP config
npm install @certaworks/goal-drift-monitor

# Optional: install globally for direct CLI commands
npm install -g @certaworks/goal-drift-monitor

CLI usage

# Create a goal session with a custom threshold
goal-drift session create \
  --goal "Keep the agent focused on refund policy questions" \
  --threshold 0.35

# Record an agent turn and score drift
goal-drift turn add \
  --session <id> \
  --content "The agent is discussing vacation planning"

# View session summary (avg drift, max drift, alert count)
goal-drift summary --session <id>

# View full per-turn drift history
goal-drift history --session <id>

# View webhook alert delivery receipts
goal-drift alerts --session <id>

# Start local HTTP API + dashboard
goal-drift serve --port 4321

Project-local Claude Desktop config

{
  "mcpServers": {
    "goal-drift-monitor": {
      "command": "node",
      "args": ["./node_modules/@certaworks/goal-drift-monitor/dist/mcp/server.js"]
    }
  }
}

MCP Interface

Available MCP tools

Goal Drift Monitor exposes the following tools to any MCP-compatible agent runtime. Tool names match the TOOLS array in src/mcp/server.ts exactly.

Session Management

create_session

Register a goal and create a drift monitoring session. Returns a session ID to use in subsequent calls. Accepts optional drift threshold and alert webhook URL.

list_sessions

List active monitoring sessions, sorted by most recently updated. Returns session ID, goal text, status, and turn count for each.

delete_session

Delete a monitoring session from the local store. Note: compliance-grade immutable audit retention is roadmap scope only.

Turn Scoring

add_turn

Score an agent turn against the registered goal. Returns drift score, alignment score, alert fired status, alert delivery status, drift trend, and scoring method.

History & Alerts

get_summary

Get drift summary for a session: average drift, max drift, alert count, latest agent intent, last alert delivery status, and trend direction.

get_history

Get the per-turn drift history for a session. Returns turn number, drift score, alignment score, timestamp, alert fired flag, and delivery status for every recorded turn.

get_alert_receipts

Get webhook alert delivery receipts for a session. Each receipt records attempted timestamp, webhook URL, delivery status, HTTP status code, and any error.

Dashboard & History

Local dashboard — prototype status

Goal Drift Monitor includes a local HTTP API and HTML dashboard for reviewing sessions, per-turn drift scores, summaries, and alert receipts. The dashboard starts alongside the local server and requires no cloud connection.

Sessions are persisted to .goal-drift-monitor/sessions.json in the working directory by default. Override the path with the GOAL_DRIFT_MONITOR_STORE_PATH environment variable. The store records goal text, drift threshold, all turn scores, and alert receipt records.

Roadmap — not currently built

Hosted SaaS monitoring dashboard, team accounts, persistent cloud session history, managed alerting beyond webhook receipts, and compliance-grade immutable audit retention are roadmap items. They are not currently built or available. What is available today is the local CLI, MCP server, local HTTP API at http://127.0.0.1:4321, and local prototype dashboard.

View local dashboard prototype →

Scope

What Goal Drift Monitor does and does not do

Does

Store durable local goal sessions and turn history in a JSON file
Score each turn against the original goal using deterministic lexical drift scoring
Fire a drift threshold alert and record a webhook delivery receipt when the threshold is breached
Track drift trend (stable, rising, falling) across the last three turns
Expose CLI, MCP tools, and local HTTP API for integration into agent pipelines
Provide session summaries with average drift, max drift, alert count, and latest agent intent

Does Not

Provide hosted SaaS monitoring, team accounts, or managed cloud history (roadmap)
Send email alerts — webhook receipts are implemented; email delivery is not
Provide compliance-grade immutable audit retention (roadmap)
Support team authentication or multi-user accounts (roadmap)
Intercept agent messages automatically — turns must be explicitly recorded via CLI, MCP, or SDK
Expose semantic scoring through the MCP interface — the MCP server supports lexical scoring only

Hosted Beta

Get early access to hosted Goal Drift Monitor

The local CLI and MCP server are available now for private testing. Hosted beta with cloud session history, managed alerting, shared team dashboards, and longer retention is in development. Leave your details and we will reach out when hosted access opens.