CLI
The promptic CLI covers common interactive and automation workflows with human-readable output.
Use the Python SDK for lower-level streaming, callbacks, and asynchronous orchestration.
Authentication
# Interactive login (opens browser)
promptic login
# Save API key directly (CI/CD)
promptic configure --api-key "ptc_..."
# Log out
promptic logoutAI Application
promptic ai-application info # Show current AI application
promptic ai-application list # List all AI applications
promptic ai-application select # Switch AI application (interactive)Components
promptic components list
promptic components create <name> [--description "..."]
promptic components get <component-id>
promptic components delete <component-id>Experiments
promptic experiments list [--component <id>] [--status pending|scheduled|running|canceling|canceled|completed|failed]
promptic experiments create <component-id> <target-model> [options]
promptic experiments get <experiment-id>
promptic experiments update <experiment-id> [--name "..." --description "..."]
promptic experiments delete <experiment-id>
promptic experiments start <experiment-id>
promptic experiments cancel <experiment-id> [--yes] [--json]
# Clone an experiment (dataset cases + evaluators) under the same component.
# By default starts from the source's initial prompt; use --initial-prompt to override.
promptic experiments duplicate <experiment-id> [--initial-prompt "..."] [--start]
# Clone an experiment and seed the new initial prompt from the source's
# best optimized iteration — useful to chain optimization runs.
promptic experiments continue <experiment-id> [--start]Cancellation
Requires a CLI and server release with experiment-cancellation support. The
command asks for confirmation; use --yes for scripts. --json returns
{"status": "canceling"} or {"status": "canceled"}. Successful exit means the
request was accepted, not necessarily that the worker stopped. Use
promptic experiments get <experiment-id> --json to check the final status.
In-flight work can still finish and incur usage charges. A client timeout or Ctrl-C does not cancel the remote run; deletion is not a cancellation mechanism.
Create options
| Flag | Description | Default |
|---|---|---|
--task-type | classification, textGeneration, structuredOutput | classification |
--provider | openai, openrouter, custom, google | openai |
--optimizer | prompticV2, miproV2, bootstrapFewShot, gepa | prompticV2 |
--initial-prompt | Starting prompt text | — |
Evaluators
promptic evaluators list <experiment-id>
promptic evaluators add <experiment-id> --name "..." --type f1|referenceJudge|comparisonJudge|generalJudge|similarity|structuredOutput [--weight 1.0]
promptic evaluators delete <experiment-id> <evaluator-id>Iterations
promptic iterations list <experiment-id>
promptic iterations get <experiment-id> <iteration-id>
promptic iterations best <experiment-id>Deployments
promptic deployments status <component-id>
promptic deployments deploy <component-id> --experiment <experiment-id>
promptic deployments prompt <component-id>
promptic deployments undeploy <component-id>Traces
promptic traces list [--status ok|error] [--limit 50] [--offset 0]
promptic traces get <trace-id>
promptic traces artifacts <trace-id>
promptic traces stats [--days-back 30]Artifacts
promptic artifacts get <artifact-id> # Show artifact metadata
promptic artifacts get <artifact-id> --output artifact.bin # Download the artifact bytes to a fileDatasets
promptic datasets create --component <component-id> --name <name> [--description "..."]
promptic datasets list --component <component-id>
promptic datasets get <dataset-id> --component <component-id>
promptic datasets delete <dataset-id> --component <component-id>datasets get displays the canonical cases currently in the dataset. To
create, update, or delete individual cases, use the
Python client or the dataset-case REST
endpoints in the API reference.
Agent Optimization
Create an Agent from a complete JSON definition, publish revisions, execute a trusted callback, and inspect submitted results:
promptic agent-gym create agent.json
promptic agent-gym status <benchmark-id>
promptic agent-gym revisions <benchmark-id>
promptic agent-gym publish-draft <benchmark-id>
promptic agent-gym dataset-pull <benchmark-id> --output ./cases
promptic agent-gym run <benchmark-id> my_agent:run \
--name invoice-agent \
--version 1.0.0 \
--architecture architecture.md
promptic agent-gym results <benchmark-id> <run-id>
promptic agent-gym case-results <benchmark-id> <run-id> --sort score --limit 5
promptic agent-gym case-result <benchmark-id> <run-id> <case-id>
promptic agent-gym artifact-download \
<benchmark-id> <run-id> <case-id> 0 \
--output review/report.pdf
promptic agent-gym compare-runs <benchmark-id> <baseline-run-id> <candidate-run-id>Monitor and recover lower-level external submission sessions:
promptic agent-gym submission-status <benchmark-id> <submission-id>
promptic agent-gym submission-wait <benchmark-id> <submission-id> --max-wait 600
promptic agent-gym retry-scoring <benchmark-id> <run-id>
promptic agent-gym submission-cancel <benchmark-id> <submission-id> --yesretry-scoring recovers original scoring using the external run's pinned evaluator revision.
It reuses saved predictions and is unavailable once official scoring has succeeded. To apply
the active evaluator revision to saved predictions, use the SDK's reevaluate_run() method or
the API's /reevaluate endpoint. See
Re-evaluate or retry scoring for the full
endpoint paths and guidance on choosing an action.
create always creates a new Agent. The old apply alias is deprecated because it never updated
an existing Agent. See External Agent Submissions
for the JSON contract, execution callback, result inspection, and recovery workflow.
Global options
All list commands support --json for JSON output:
promptic components list --json
promptic traces list --json | jq '.[] | .traceId'