Tracing

Promptic uses OpenTelemetry to automatically capture every LLM call your application makes. Once set up, you get full visibility into requests, responses, token usage, cost, and latency — with zero manual instrumentation.

Setup

import promptic_sdk

promptic_sdk.init()

init() installs OpenTelemetry instrumentation for all detected LLM libraries and exports spans to the Promptic platform.

Supported providers and frameworks

Install the SDK with the extras for your stack:

Pydantic AI is also supported without any extra: it ships its own OpenTelemetry emitter. Enable it with Agent(..., instrument=True) and its spans will flow to Promptic through the same TracerProvider that init() configures.

Configuration

promptic_sdk.init(
    api_key="pk_live_...",      # default: PROMPTIC_API_KEY env var
    endpoint="https://...",      # default: https://promptic.eu
    auto_instrument=True,        # default: True — auto-detect and instrument LLM libs
    service_name="my-service",   # optional — sets the OTel service name
)

AI Components

AI Components are logical groupings for your LLM-powered features. Use them to separate traces for different parts of your application:

with promptic_sdk.ai_component("email-classifier"):
    # All LLM calls inside this block are tagged with "email-classifier"
    response = client.chat.completions.create(...)

with promptic_sdk.ai_component("support-agent"):
    # These traces are tagged with "support-agent"
    response = client.chat.completions.create(...)

Components are auto-created in Promptic when the first trace arrives. You can also create them explicitly via the SDK client or API.

Datasets and runs

Tag traces with a dataset and run to group them for evaluation:

with promptic_sdk.ai_component("support-agent", dataset="eval-set", run="v2"):
    for query in test_queries:
        agent.run(query)

This creates a dataset called "eval-set" with a run called "v2" under the "support-agent" component. All traces from this block are automatically added to that run.

You can also use the dataset context manager for more granular control:

with promptic_sdk.ai_component("support-agent"):
    # Production traffic — no dataset tagging
    agent.run(user_query)

    # Evaluation run — tagged
    with promptic_sdk.dataset("nightly-eval"):
        for query in test_queries:
            agent.run(query)

Tracing workflows with custom spans

When you do need it, wrap your workflow stages in custom OpenTelemetry spans. Auto-instrumented LLM and tool spans automatically nest under whichever custom span is active.

The recommended pattern:

1. Wrap the whole run in one root workflow span inside ai_component(...).
2. Add a child task span for each meaningful stage of the pipeline.
3. Record each stage's input and output as span attributes so the trace reads as a transformation, not just a list of LLM calls.

import json
import promptic_sdk
from opentelemetry import trace

promptic_sdk.init()
tracer = trace.get_tracer(__name__)

with promptic_sdk.ai_component("support-agent"):
    with tracer.start_as_current_span("answer_question") as root:
        root.set_attribute("traceloop.span.kind", "workflow")
        root.set_attribute("traceloop.entity.input", json.dumps(user_input))

        with tracer.start_as_current_span("retrieve_context") as span:
            span.set_attribute("traceloop.span.kind", "task")
            span.set_attribute("traceloop.entity.input", json.dumps(query))
            context = retrieve(query)
            span.set_attribute("traceloop.entity.output", json.dumps(context))

        with tracer.start_as_current_span("generate_answer") as span:
            span.set_attribute("traceloop.span.kind", "task")
            # The auto-instrumented LLM call nests under this task span
            answer = llm_call(context)

        root.set_attribute("traceloop.entity.output", json.dumps(answer))

Span attribute conventions:

A few tips:

• Use semantic span names (retrieve_context, rerank_results) rather than generic function names, so repeated stages stay distinguishable in the trace view.

• For large payloads, log a small preview plus a count rather than the full object — traces are not designed to store data:

  span.set_attribute(
      "traceloop.entity.output",
      json.dumps({
          "items": items[:5],
          "item_count": len(items),
          "additional_item_count": max(len(items) - 5, 0),
      }),
  )

The result is a single trace where the root workflow span carries the structured input and output, task spans appear as its children, and every auto-instrumented LLM or tool call nests under the stage that triggered it.

LangGraph and deepagents support

pip install promptic-sdk[langchain] installs OpenLLMetry's opentelemetry-instrumentation-langchain (≥0.60), which covers LangChain chains, LangGraph (create_agent), and deepagents with subagents. It emits the official OpenTelemetry GenAI semantic conventions (gen_ai.operation.name, gen_ai.tool.definitions, gen_ai.tool.name, gen_ai.usage.*) — the same schema Promptic's trace parser uses for every framework, so agent-evaluation insights (loops, tool errors, unused tools) work identically for flat agents and multi-agent graphs.

No extra configuration required.

Alternative: LangSmith OTel bridge

If you prefer LangSmith's built-in OTel exporter (for hybrid setups where traces also flow to LangSmith), set these env vars before calling init():

LANGSMITH_TRACING=true
LANGSMITH_OTEL_ENABLED=true

The SDK will propagate Promptic tags (promptic.ai_component, promptic.dataset, promptic.run) through LangSmith's run metadata so bridged spans are still linked to the right component. Note that the LangSmith bridge does not emit tool definitions, so the "unused tools" evaluator insight will not fire on LangSmith-bridged traces.

Custom OpenTelemetry instrumentors

Since Promptic uses standard OpenTelemetry, you can add any OTel instrumentor alongside the built-in ones:

import promptic_sdk
from opentelemetry.instrumentation.httpx import HTTPXInstrumentor

promptic_sdk.init()
HTTPXInstrumentor().instrument()  # Also trace HTTP calls

What's captured

Each trace includes: