Usage tracking for token counts, tool call metrics, session analytics, and cost analysis.

Telemetry

Supyagent tracks usage data locally for each agent, giving you visibility into token consumption, tool call patterns, and error rates. All telemetry data stays on your machine in .supyagent/telemetry/.

What Is Tracked

Telemetry records three types of events:

Event Type	Fields	Description
`llm_call`	`input_tokens`, `output_tokens`, `model`, `duration_ms`	Each LLM API call
`tool_call`	`tool`, `ok`, `duration_ms`, `is_service`	Each tool execution
`turn`	`messages_count`, `tools_used`	Each user message/response cycle
`error`	`type`, `message`	Errors during execution

Events are stored as JSONL files at .supyagent/telemetry/{agent_name}/{session_id}.jsonl.

Viewing Telemetry

CLI Command

# View telemetry for an agent (all sessions)
supyagent telemetry myagent

# View telemetry for a specific session
supyagent telemetry myagent --session abc123

# Export as JSON for analysis
supyagent telemetry myagent --json

The default output shows a summary:

Telemetry for myagent
  Sessions: 5
  Turns: 42
  LLM calls: 87
  Total tokens: 245,000
  Errors: 2

Tool Usage
  Tool                   Calls   OK  Errors  Avg (ms)  Type
  shell__run_command        35   33       2      1250  local
  files__read_file          28   28       0       120  local
  edit__edit_replace        15   15       0       350  local
  gmail__send_message        5    5       0       890  service
  search__search_files       8    8       0       210  local

JSON Export

The --json flag outputs structured data for programmatic analysis:

{
  "agent": "myagent",
  "turns": 42,
  "llm_calls": 87,
  "total_tokens": 245000,
  "errors": 2,
  "tools": {
    "shell__run_command": {
      "calls": 35,
      "ok": 33,
      "errors": 2,
      "total_ms": 43750,
      "is_service": false
    },
    "gmail__send_message": {
      "calls": 5,
      "ok": 5,
      "errors": 0,
      "total_ms": 4450,
      "is_service": true
    }
  }
}

In-Chat Monitoring

`/tokens` Command

Toggle real-time token usage display after each turn:

/tokens
Token display on

You: What files are in this directory?

myagent> [tool: files__list_directory] ...
Here are the files in the current directory: ...

  tokens: 12,450 msgs + 8,200 tools | context: 20,650 / 200,000 (10%)

The display shows:

Message tokens: Total tokens in the conversation messages
Tool tokens: Tokens consumed by tool definitions
Context usage: Total context consumption as a fraction of the limit

`/context` Command

View the full context window status including summarization trigger progress:

/context

Context Status
  Context limit: 200,000 tokens
  Tool definitions: 8,200 tokens (35 tools)
  Last summary: 42 messages -> 580 tokens
  Created: 2025-01-15 14:30

Summarization Triggers (N messages OR K tokens)
  Messages: 18 / 30 (60%)
           [████████████░░░░░░░░]
  Tokens:   45,000 / 128,000 (35%)
           [███████░░░░░░░░░░░░░]

Analyzing Usage Patterns

Token Cost Estimation

Use the JSON export to estimate costs:

supyagent telemetry myagent --json | python3 -c "
import json, sys
data = json.load(sys.stdin)
tokens = data['total_tokens']
# Rough estimate at $3/M input tokens
cost = (tokens / 1_000_000) * 3
print(f'Total tokens: {tokens:,}')
print(f'Estimated cost: \${cost:.2f}')
"

Tool Error Analysis

Identify tools with high error rates:

supyagent telemetry myagent --json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for name, stats in sorted(data['tools'].items(), key=lambda x: x[1]['errors'], reverse=True):
    if stats['errors'] > 0:
        rate = stats['errors'] / stats['calls'] * 100
        print(f'{name}: {stats[\"errors\"]}/{stats[\"calls\"]} errors ({rate:.0f}%)')
"

Performance Bottlenecks

Find the slowest tools by average execution time:

supyagent telemetry myagent --json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for name, stats in sorted(data['tools'].items(), key=lambda x: x[1]['total_ms']/max(x[1]['calls'],1), reverse=True):
    avg = stats['total_ms'] / max(stats['calls'], 1)
    print(f'{name}: {avg:.0f}ms avg ({stats[\"calls\"]} calls)')
"

Local vs Service Tools

Telemetry distinguishes between local tools (supypowers scripts) and service tools (cloud integrations). The is_service flag in tool call events lets you analyze usage separately:

Local tools: Execute via supypowers run on your machine
Service tools: Execute via the supyagent service API (Gmail, Slack, GitHub, etc.)

Data Retention

Telemetry data accumulates over time. It is stored per-session, so you can delete individual session telemetry files or clear all telemetry for an agent:

# Remove all telemetry for an agent
rm -rf .supyagent/telemetry/myagent/

# Remove telemetry for a specific session
rm .supyagent/telemetry/myagent/SESSION_ID.jsonl

Use supyagent cleanup to remove stale data:

supyagent cleanup --sessions   # Also cleans empty sessions

Context Management -- Token budgets and summarization
Prompt Caching -- Reduce costs with caching
CLI Reference -- Telemetry command options

Telemetry

On this page