# Docs - Documentation: Guides, SDK reference & concepts - [Introduction](/docs/guides/introduction): Learn how Promptic improves GenAI quality and cost, then choose your first optimization workflow. - [Quickstart](/docs/guides/dashboard-quickstart): Create your first AI Application and choose the right component workflow. - **Concepts** - [How Promptic is organized](/docs/guides/platform-model): Understand Organizations, AI Applications, AI Components, and their boundaries. - [AI Components](/docs/guides/ai-components): Choose the correct component type and understand what data each workflow creates. - [Models and providers](/docs/guides/models-and-providers): Choose managed models, BYOK credentials, Google Vertex AI, or custom providers. - **Optimizers** - [Prompt Optimization](/docs/guides/prompt-optimization): Automatically find the best prompt for your task using experiments. - [Agent Optimization](/docs/guides/agent-optimization): Benchmark complete agents, compare variants, and improve them against versioned evidence. - [MCP Optimization](/docs/guides/mcp-tool-optimization): Connect an MCP server and improve tool descriptions for more reliable selection. - [Tool Selection Optimization](/docs/guides/agent-tool-optimization): Improve pasted agent tool definitions without connecting an MCP server. - **Workflows** - [Tracing](/docs/guides/tracing): Instrument your LLM calls with automatic observability. - [External Agent Submissions](/docs/guides/external-agent-gym-submissions): Run an agent in your own environment and submit predictions, artifacts, and traces for evaluation. - [Evaluators and judges](/docs/guides/evaluators): Choose scoring methods for prompts, structured outputs, and tool selection. - [Model comparison](/docs/guides/model-comparison): Compare candidate models on quality, cost, and latency using the same experiment data. - [Deployment](/docs/guides/deployment): Deploy optimized prompts and fetch them at runtime. - **Manage** - [Teams and access](/docs/guides/teams-and-access): Manage Organization and AI Application membership with least-privilege access. - [API keys](/docs/guides/api-keys): Create, store, rotate, and use AI Application API keys safely. - [Billing and pricing](/docs/guides/billing-and-pricing): Understand AI Applications, Promptic Credits, model usage, retention, and add-ons. - [Security and data handling](/docs/guides/security-and-data): Understand captured data, credentials, artifacts, retention, and safe instrumentation. - [Troubleshooting](/docs/guides/troubleshooting): Diagnose missing traces, authentication problems, failed experiments, and empty results. - **Python SDK** - [Quickstart](/docs/guides/sdk/quickstart): Get Promptic tracing running in under 5 minutes. - [Installation](/docs/guides/sdk/installation): Install and configure the Promptic Python SDK. - [Tracing API](/docs/guides/sdk/tracing-api): Reference for init(), ai_component(), dataset(), and artifact() functions. - [Client](/docs/guides/sdk/client): Reference for the core and Agent Optimization Python clients. - [CLI](/docs/guides/sdk/cli): Reference for the promptic command-line tool. - **API** - [Authentication](/docs/guides/authentication): How to authenticate requests to the Promptic API. - [Pagination](/docs/guides/pagination): How to paginate through API results. - [Errors](/docs/guides/errors): Understanding Promptic API error responses. - API Reference: OpenAPI endpoints - [Overview](/docs/openapi): Explore the Promptic API endpoints. - **AI Application** - [Get current AI Application](/docs/openapi/ai-application/get) - [List AI Applications for current user](/docs/openapi/ai-applications/get): Requires session authentication (not API key) - **Models** - [List models available to the current AI Application](/docs/openapi/models/get): Returns model selection IDs after applying the AI Application's configured providers and disabled-model settings. `judgeEligible` identifies models selectable in the benchmark evaluator Judge model picker. An API key carries its AI Application scope; session requests require X-AI-Application-Id or aiApplicationId. - **AI Components** - [List all components](/docs/openapi/components/get) - [Create a component](/docs/openapi/components/post) - [Get a component by ID](/docs/openapi/components/componentid/get) - [Delete a component](/docs/openapi/components/componentid/delete) - **Experiments** - [List experiments](/docs/openapi/experiments/get) - [Create an experiment](/docs/openapi/experiments/post) - [Create a complete tool-selection experiment](/docs/openapi/experiments/tool-selection/post): Atomically creates the experiment, managed dataset, tools, canonical cases, system-prompt settings, and required evaluator. - [Get an experiment by ID](/docs/openapi/experiments/experimentid/get) - [Update a pending experiment](/docs/openapi/experiments/experimentid/patch) - [Delete an experiment](/docs/openapi/experiments/experimentid/delete) - [Start a pending experiment](/docs/openapi/experiments/experimentid/start/post) - [Duplicate an experiment (clones dataset cases + evaluators)](/docs/openapi/experiments/experimentid/duplicate/post): Creates a new experiment under the same AI component as the source, copying dataset cases and evaluators. The initial prompt comes from the source experiment by default; pass `continueFromOptimized: true` to seed from the source's best optimized prompt (continue flow), or `initialPromptOverride` to override with custom text. - **Evaluators** - [List evaluators for an experiment](/docs/openapi/experiments/experimentid/evaluators/get) - [Create evaluator(s)](/docs/openapi/experiments/experimentid/evaluators/post): Accepts a single evaluator object or an array - [Update an evaluator](/docs/openapi/experiments/experimentid/evaluators/evaluatorid/patch) - [Delete an evaluator](/docs/openapi/experiments/experimentid/evaluators/evaluatorid/delete) - **Iterations** - [List iterations for an experiment](/docs/openapi/experiments/experimentid/iterations/get) - [Get the best iteration for an experiment](/docs/openapi/experiments/experimentid/iterations/best/get) - [Get an iteration with scores](/docs/openapi/experiments/experimentid/iterations/iterationid/get) - **Deployment** - [Get current deployment for a component](/docs/openapi/components/componentid/deployment/get) - [Deploy an experiment to a component](/docs/openapi/components/componentid/deployment/post) - [Undeploy from a component](/docs/openapi/components/componentid/deployment/delete) - [Get the deployed prompt for a component](/docs/openapi/components/componentid/deployment/prompt/get) - **Traces** - [List traces](/docs/openapi/traces/get) - [Ingest OTLP traces](/docs/openapi/traces/post): Accepts OpenTelemetry trace data in JSON or protobuf format - [Get a trace by OpenTelemetry ID](/docs/openapi/traces/traceid/get) - [List artifacts referenced by a trace](/docs/openapi/traces/traceid/artifacts/get) - [Get tracing statistics](/docs/openapi/traces/stats/get) - **Storage** - [Create a presigned upload](/docs/openapi/storage-objects/presign/post): Reserves an AI Application-scoped storage object and returns a short-lived direct-upload descriptor. Upload the file bytes straight to `uploadUrl` using the returned `method`, `headers`, and `fields`; the file never passes through Promptic. After uploading, POST to /storage-objects/{id}/complete to validate the actual stored size and MIME type. Declared size is not recorded as actual usage. Defaults to private access. MIME types are checked before signing: artifacts and benchmarks folders accept PDF, PNG, JPEG, WebP, plain text, CSV, Markdown, JSON, DOCX, and XLSX. Other folders additionally accept binary (application/octet-stream), PPTX, ZIP, GIF, SVG, and HTML. Media type matching is case-insensitive and ignores MIME parameters. Unsupported types return 400. - [Complete a direct upload](/docs/openapi/storage-objects/id/complete/post): Checks the uploaded object against the size limit and MIME type authorized when the upload was created. Records actual size only after validation. Oversized or MIME-mismatched objects are deleted. No request body is required; client size or limit claims are ignored. Repeated completion requests recheck storage metadata. - [Get a presigned download URL](/docs/openapi/storage-objects/id/url/get): Returns a short-lived signed GET URL for a storage object in the caller's AI Application. Requests for objects outside the caller's AI Application return 404 with no existence leak. - [Redirect to storage object content](/docs/openapi/storage-objects/id/content/get): Convenience 302 redirect to a short-lived signed object URL for browser consumers. The `/url` sibling returns the same URL as JSON without redirecting. - **Artifacts** - [Upload an artifact](/docs/openapi/artifacts/post) - [Get artifact metadata](/docs/openapi/artifacts/artifactid/get) - [Download artifact content](/docs/openapi/artifacts/artifactid/content/get): Redirects to a short-lived provider URL for the private artifact object. - **Datasets** - [List datasets for a component](/docs/openapi/components/componentid/datasets/get) - [Create a dataset](/docs/openapi/components/componentid/datasets/post) - [Get a dataset with cases](/docs/openapi/components/componentid/datasets/datasetid/get) - [Delete a dataset](/docs/openapi/components/componentid/datasets/datasetid/delete) - [List cases in a dataset](/docs/openapi/components/componentid/datasets/datasetid/cases/get) - [Create dataset cases](/docs/openapi/components/componentid/datasets/datasetid/cases/post): Accepts one canonical dataset case or an array of up to 1000 cases. - [Get a dataset case](/docs/openapi/components/componentid/datasets/datasetid/cases/caseid/get) - [Update a dataset case](/docs/openapi/components/componentid/datasets/datasetid/cases/caseid/patch) - [Delete a dataset case](/docs/openapi/components/componentid/datasets/datasetid/cases/caseid/delete)