Skip to main content

Documentation Index

Fetch the complete documentation index at: https://openlayer.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

The Openlayer Gateway is a single endpoint that sits between your apps and your LLM providers and enforces your rules on every request, in real time. It blocks calls that go over budget, stops prompt injection, and redacts PII before a request reaches a provider or a response reaches a user. Every request that flows through the gateway is also traced to your Openlayer project, so the monitoring you already rely on keeps working.

​
How it works

1

Point your app at the gateway

Keep your existing OpenAI- or Anthropic-compatible client. Point its base URL at the gateway and swap in a gateway key.For example:
from openai import OpenAI

client = OpenAI(
    base_url="https://your-gateway.example.com/v1",  # the gateway, not api.openai.com
    api_key="sk-olga-...", # a gateway key, not your provider key
)

response = client.responses.create(model="gpt-4o-mini", input="Hello!")
2

Enforce your rules

The gateway verifies the key, runs your content guardrails, and checks the request against your usage limits. Anything that trips a guardrail or exceeds a limit is blocked here, before it reaches a provider.
3

Route to a provider

It forwards the request to the provider you configured, and can override the model or translate between formats, so an OpenAI-style request runs against an Anthropic model, and vice versa.
4

Return and record

Output guardrails run on the way back, then the response streams to your app. Every request is also published as a trace in your Openlayer project, fire-and-forget, so tracing never slows or breaks a response.Every gateway request becomes a trace in Openlayer

​
What you get

Everything is configured once, in the gateway’s admin portal. There are no redeploys and no code changes in the apps sending traffic. The Openlayer Gateway admin portal

Access & keys

Issue and revoke keys for each app or teammate, group them into teams, and hand out self-serve invite links.

Budgets & usage

Cap spend, requests, or tokens per key or team, and see exactly what’s being consumed and what it costs.

Guardrails

Block or redact PII and stop prompt-injection attempts on inputs and outputs, before they reach a provider or a user.

Providers & models

Connect OpenAI, Anthropic, and Azure upstreams, route traffic between them, and call any model from any SDK.

​
FAQ

Almost nothing. You point your client at the gateway’s base URL and use a gateway key instead of a provider key. That’s it. Any client that speaks the OpenAI Responses API or the Anthropic Messages API works unchanged, including streaming.
They serve different jobs and you can use either or both.The gateway exists to enforce your rules on every request, in real time. Tracing comes as a byproduct: because traffic flows through the gateway, every request is also published to your Openlayer project without any SDK in your apps. That makes it the right path when you want governance plus baseline observability across many apps or languages at one chokepoint.The Openlayer SDK runs inside your code. It doesn’t enforce anything at the request boundary, but it captures detail the gateway can’t see from outside: nested spans for retrieval, tools, and sub-calls in a multi-step pipeline, plus any custom metadata. Reach for it when you need fine-grained traces of how a feature actually runs.A common setup is the gateway for governance and baseline traces, plus the SDK in the places that need the deeper view.
The gateway forwards traffic. It doesn’t keep your prompts and responses. It stores hashed API keys, per-day usage tallies, and guardrail event metadata (which guardrail triggered and what kind of entity, not the underlying text).The full request and response content is sent to your Openlayer project as a trace, where your data-retention and access controls apply. Openlayer can run on-premises or as SOC 2 Type II compliant SaaS.
Yes. When you route a request to a provider whose format differs from your client’s, the gateway translates the request, response, and stream between the OpenAI Responses API and the Anthropic Messages API. See Use any model from any SDK.
OpenAI, Anthropic, and Azure OpenAI / Foundry today. Any provider that exposes an OpenAI- or Anthropic-compatible API can be connected. See Connect providers.