Test your supypowers tools before handing them to agents, using fixtures or auto-generated inputs.

Testing Tools

Before giving a tool to an agent, verify it works correctly. The supypowers test command makes this easy -- it can auto-generate example inputs from the schema or use fixture files.

Quick Test with Auto-Generated Input

The simplest way to test is to let supypowers generate example input from the function's schema:

supypowers test weather:get_weather

Supypowers reads the input model's schema, generates example values for required fields, and runs the function:

Testing weather:get_weather with input: {"city": "..."}
{"ok": true, "data": {"ok": true, "temperature": null, "condition": null, "error": "WEATHER_API_KEY not set"}}

The auto-generated values use sensible defaults based on field type:

Field Type	Generated Value
`str`	`"..."`
`int`	`0`
`float`	`0.0`
`bool`	`true`
`list`	`[]`
`dict` / `object`	`{}`

Only required fields are included in the auto-generated input. Optional fields with defaults are omitted.

Testing with Fixture Files

For realistic testing, create a JSON fixture file with the exact input you want:

fixtures/weather_london.json

{
  "city": "London",
  "units": "metric"
}

Run the test with the --fixture flag:

supypowers test weather:get_weather --fixture fixtures/weather_london.json

Output:

Testing weather:get_weather with input: {"city": "London", "units": "metric"}
{"ok": true, "data": {"ok": true, "temperature": 18.5, "condition": "Partly cloudy", "error": null}}

Testing with Secrets

If your tool needs API keys or other secrets, pass them with --secrets:

# Inline secret
supypowers test weather:get_weather --fixture fixtures/weather_london.json \
  --secrets WEATHER_API_KEY=abc123

# From a .env file
supypowers test weather:get_weather --fixture fixtures/weather_london.json \
  --secrets .env

Testing from Bundled Examples

To test the built-in example tools (useful for verifying your supypowers installation):

supypowers test exponents:compute_sqrt --examples

This runs the example scripts bundled with supypowers instead of looking in the local powers/ directory.

Manual Testing with `supypowers run`

For full control over the input, use supypowers run directly:

supypowers run weather:get_weather '{"city": "Tokyo", "units": "imperial"}' \
  --secrets WEATHER_API_KEY=abc123

This is identical to what the agent does when it calls the tool. The output format is:

{"ok": true, "data": {"ok": true, "temperature": 75.2, "condition": "Clear", "error": null}}

Or on error:

{"ok": false, "error": "function not found: get_weatherr"}

Organizing Fixture Files

A recommended structure for test fixtures:

my-project/
  powers/
    weather.py
    email.py
  fixtures/
    weather/
      london.json
      invalid_city.json
      missing_key.json
    email/
      send_basic.json
      send_with_attachment.json

Example Fixtures for Error Cases

Test that your tool handles errors gracefully:

fixtures/weather/invalid_city.json

{
  "city": "XXXXXXX_NOT_A_CITY",
  "units": "metric"
}

fixtures/weather/missing_key.json

{
  "city": "London"
}

Run them and verify the error messages are clear:

supypowers test weather:get_weather --fixture fixtures/weather/invalid_city.json
supypowers test weather:get_weather --fixture fixtures/weather/missing_key.json

Viewing the Tool Schema

Before testing, inspect the tool's schema with supypowers docs:

# JSON schema output
supypowers docs

# Human-readable markdown
supypowers docs --format md

This shows you all discovered functions, their input schemas, and descriptions -- exactly what the agent sees.

Checking for Common Issues

Symptom	Cause	Fix
`function not found: X`	Typo in function name (case-sensitive)	Check spelling with `supypowers docs`
`input must be a Pydantic BaseModel`	Missing type annotation on `input`	Add `def func(input: MyModel)`
`function must accept exactly one parameter`	Extra parameters on function	Only one param named `input` allowed
`runner did not emit valid JSON`	`print()` in your script	Remove all print statements
Import error	Missing dependency	Add to `# /// script` block
Empty/null auto-generated input	No required fields	Add required fields with `Field(...)`

What's Next

Creating Tools -- Writing tools from scratch
Dependencies -- Managing per-script dependencies
Publishing -- Sharing tools across projects

Testing Tools

On this page