Troubleshooting

Start with the narrowest failing boundary: authentication, data creation, job start, model call, evaluation, or result rendering. Preserve the error message and relevant IDs without copying credentials or raw customer payloads.

No traces appear

Check that:

1. promptic_sdk.init() runs before the instrumented provider or framework is used;
2. the correct SDK extra is installed for the library;
3. PROMPTIC_API_KEY belongs to the intended AI Application;
4. PROMPTIC_ENDPOINT is correct for self-hosted or demo environments;
5. the process can reach the endpoint;
6. service_name and deployment.environment.name match the active Tracing filters;
7. the application exits gracefully enough to flush telemetry.

AI Component attribution is optional. If a trace appears globally but not under a component filter, check that the ai_component() context wraps the spans you expect.

Authentication fails

• 401 usually means the bearer token is missing, malformed, revoked, or not a Promptic API key.
• 403 means the authenticated identity lacks permission for the resource.
• 404 can intentionally hide a resource outside the active AI Application.

Do not log the key to debug it. Create a replacement key and compare only its scope and environment placement.

An experiment will not start

Verify:

• the experiment is still pending;
• dataset cases exist and contain every variable used by the prompt;
• at least one evaluator is configured correctly;
• target, optimizer, and judge models are available;
• provider credentials pass their connection test;
• the Organization has an active plan or allowed free-tier capacity;
• managed-model spending controls have not blocked new work.

An experiment can consume model usage before later work fails. Inspect the experiment status and usage rather than repeatedly clicking Start.

Scores are empty or surprising

• Look for skipped or failed evaluator executions.
• Confirm expected values use the correct labels or schema.
• Check that a judge rubric references available variables.
• Inspect predictions case by case before trusting the aggregate.
• For train/eval split, confirm which set the selected metric represents.
• A high similarity score does not necessarily mean factual correctness.

Tool Optimization fails

• Test MCP connectivity and authentication separately.
• Ensure tool names are unique after normalization.
• Use exact tool names in expected results.
• Use none, not a blank cell, for uploaded no-tool cases.
• Include at least one case for each tool you expect to improve.

Collecting support information

Include:

• page or endpoint;
• AI Application, component, dataset, experiment, iteration, or trace ID;
• timestamp and timezone;
• status code and sanitized error message;
• whether the problem is reproducible;
• SDK and provider-library versions.

Never include API keys, provider credentials, session cookies, private prompts, or unredacted customer data. For structured API errors, see Error handling.