Traces help you understand your system, particularly when it contains multiple steps, such as in RAG, LLM chains, and agents. In the monitoring mode of an Openlayer project, you can view the traces for the live requests your AI system receives. This allows you to log the inputs, outputs, latency, and other metadata such as cost and number of tokens associated with every step of your system. This guide shows how you can set up tracing with Openlayer’s SDKs to achieve a result similar to the one below.
If you use a framework that supports OpenTelemetry (OTel), you can export traces directly to Openlayer without needing to use Openlayer’s SDKs. See the OpenTelemetry integration for more details.
Requests and traces

​
How to set up tracing

You must use one of Openlayer’s SDKs to trace your system. After installing the SDK in your language of choice, follow the steps:
If you prefer, feel free to refer to a notebook example. Our templates gallery also has complete sample projects that show how tracing works for development and monitoring.
1

Set environment variables

Openlayer needs to know where to upload the traces to. This information is in the following environment variables:
Shell
OPENLAYER_API_KEY=YOUR_OPENLAYER_API_KEY
OPENLAYER_INFERENCE_PIPELINE_ID=YOUR_OPENLAYER_INFERENCE_PIPELINE_ID
2

Annotate the code you want to trace

Annotate all the functions you want to trace with Openlayer’s SDK.
import openai
from openlayer.lib import trace, trace_openai, update_current_trace, update_current_step

# Wrap the OpenAI client Openlayer's `trace_openai`
openai_client = trace_openai(openai.OpenAI(api_key="sk-..."))

# Decorate all the functions you want to trace
@trace()
def main(user_query: str) -> str:
    context = retrieve_context(user_query)
    answer = generate_answer(user_query, context)
    return answer

@trace()
def retrieve_context(user_query: str) -> str:
    return "Some context"

@trace()
def generate_answer(user_query: str, context: str) -> str:
    result = openai_client.chat.completions.create(
        messages=[{"role": "user", "content": user_query + " " + context}],
        model="gpt-4o"
    )
    return result.choices[0].message.content
The traced generate_answer function in the example above uses an OpenAI LLM. However, tracing also works for other LLM providers. If you set up any of the streamlined approaches described in the Publishing data guide, it will get added to the trace as well.
3

Use the annotated code

All data that goes through the decorated code is automatically streamed to the Openlayer platform, where your tests and alerts are defined.In the example above, if we call main:
main("What's the meaning of life?")
the resulting trace would be:TraceThe main function has two nested steps: retrieve_context, and generate_answer. The generate_answer has a chat completion call within it. The cost, number of tokens, latency, and other metadata are all computed automatically behind the scenes.

​
Updating Trace Metadata and Custom Inference IDs

These features are available in Openlayer Python SDK v0.2.0a91+ and enable advanced request correlation scenarios and dynamic metadata enrichment.
You can dynamically update trace metadata during execution to add context like user IDs, session variables, custom tags, or set custom inference IDs. Openlayer provides two main functions for this:
  • update_current_trace() - Updates trace-level metadata and inference IDs
  • update_current_step() - Updates current step/span metadata

​
Setting Custom Inference IDs

By default, Openlayer automatically generates unique UUIDs for each trace. However, you can set custom inference IDs to enable request correlation with external systems, batch processing, and audit trails. To set a custom inference ID, use the update_current_trace function within any traced function: 
import time
from openlayer.lib import trace, update_current_trace

@trace()
def process*request(user_id: str): # Set a custom inference ID for request correlation
custom_id = f"req*{user*id}*{int(time.time())}"
update_current_trace(inferenceId=custom_id)

    # Your processing logic here
    return "processed"

​
Common Use Cases for Custom Inference IDs

  • Request Correlation: Link traces with external API request IDs or session identifiers
  • Batch Processing: Group related requests with consistent correlation patterns
  • System Integration: Use existing request identifiers from upstream systems
  • Audit Trails: Maintain consistent identifiers across distributed systems
  • Debugging: Easily correlate logs and traces using familiar identifiers

​
Setting Trace-Level Metadata

You can enrich traces with contextual information like user IDs, session variables, or custom tags using update_current_trace(): 
from openlayer.lib import trace, update_current_trace

@trace()
def process_user_request(user_id: str, session_id: str): # Add user context to the trace
update_current_trace(
user_id=user_id,
session_id=session_id,
environment="production",
feature_flag="new_algorithm_v2"
)

    result = perform_analysis()
    return result

@trace()
def perform_analysis(): # This step inherits the trace metadata
return "analysis completed"

​
Setting Step-Level Metadata

You can also add metadata specific to individual steps using update_current_step(): 
from openlayer.lib import trace, update_current_trace, update_current_step

@trace()
def main_workflow(user_query: str):
update_current_trace(user_id="user_123", session_id="session_456")

    # Process through multiple steps
    context = retrieve_context(user_query)
    answer = generate_answer(user_query, context)
    return answer

@trace()
def retrieve_context(query: str): # Add step-specific metadata
update_current_step(
search_method="semantic_search",
query_length=len(query),
search_latency_target="<100ms"
)

    # Simulate retrieval
    context = "retrieved context"

    # Update with actual results
    update_current_step(
        documents_found=5,
        search_score=0.95,
        actual_latency="85ms"
    )

    return context

@trace()
def generate_answer(query: str, context: str):
update_current_step(
generation_model="gpt-4o",
context_length=len(context),
temperature=0.7
)

    # Generate response
    answer = "generated answer"

    # Add generation metadata
    update_current_step(
        tokens_used=150,
        cost_estimate=0.003,
        generation_time="1.2s"
    )

    return answer

​
Common Use Cases

  • Custom Inference IDs: Request correlation, batch processing, external system integration, audit trails, user feedback collection
  • Trace Metadata: User context, session tracking, A/B testing flags, environment information, business metrics
  • Step Metadata: Performance metrics, model parameters, intermediate results, error details, retry information

​
Using Custom Inference IDs for User Feedback and Signals

Custom inference IDs are particularly powerful when combined with Openlayer’s data updating capabilities. By setting meaningful inference IDs, you can easily add user feedback, ground truth labels, and other signals after the initial request: 
import time
from openlayer.lib import trace, update_current_trace

@trace()
def process*user_request(user_id: str, session_id: str, request_id: str = None): # Set a custom inference ID that you can use later for feedback
inference_id = request_id or f"user*{user*id}\_req*{int(time.time())}"
update_current_trace(
inferenceId=inference_id,
user_id=user_id,
session_id=session_id,
timestamp=int(time.time())
)

    # Store the inference_id for later use (e.g., in your database)
    store_inference_id_for_feedback(inference_id, user_id, session_id)

    result = generate_response(query)
    return result, inference_id

def store_inference_id_for_feedback(inference_id: str, user_id: str, session_id: str): # Save to your database for later feedback collection
database.save_request_metadata({
"inference_id": inference_id,
"user_id": user_id,
"session_id": session_id,
"created_at": time.time()
})
This approach enables powerful feedback loops where you can:
  • Collect user ratings and satisfaction scores after interaction
  • Add ground truth labels after expert review or verification
  • Update with business signals like conversion rates or success metrics
  • Enrich with contextual data discovered after the initial request
See the Updating Data guide for comprehensive examples of how to update previously published data using your custom inference IDs.
Trace metadata is set at the trace level and applies to all steps within that trace.Step metadata is specific to the current function/step being executed.Custom inference IDs are set at the trace level - if you don’t set one, Openlayer automatically generates a unique UUID.Metadata merging - calling update_current_trace() or update_current_step() multiple times will merge the new metadata with existing values.