Openlayer

Reference

Integrations

Changelog

Get started

Community

Log in

Introduction

Quickstart

Create projects

Create API keys

Explore the tests available on the platform

Tests overview

Understanding tests

Custom metrics

Character length

Class imbalance ratio

Column average

Conflicting labels

Column contains string

Correlated features

Data type validation

Duplicate rows

Empty features

Empty feature count

Features missing values

Feature values

Great expectations

Ill-formed rows

Is code

Is JSON

Null rows

Number of rows

Personal identifiable information (PII)

Predictive power score (PPS)

Quasi-constant features

Quasi-constant feature count

Special characters ratio

String validation

Valid URLs

Column drift

Column values match

Feature drift

Label/target drift

New categories

New labels

Size ratio

Training-validation leakage

Aggregate metrics

LLM evaluation

Max cost

Max latency

Max tokens

Mean cost

Mean latency

Mean tokens

Total cost

Total tokens

Learn how you can use Development mode with Git, the Openlayer CLI, or the Openlayer REST API

Development mode overview

Learn how to write the `openlayer.json` config for your project

openlayer.json

Learn how to write the `tests.json` config for your project

tests.json

Configuring output generation

GitHub Actions with Openlayer

Push and poll using the Openlayer CLI

Push and poll using the Openlayer REST API

Monitoring mode overview

Publishing data

Tracing

Annotating data

Evaluation and delay windows

Uploading a reference dataset

Updating data

LLM cost estimation

Glossary

Learn more about our data security practices and compliance measures.

Security

Templates

Monitoring

Overview

Python

TypeScript

Java

CLI global options

batch

export

help

inspect

install

link

login

metrics

push

validate

whoami

List projects

Create project

Retrieve a presigned url to post storage artifacts.

List the commits (project versions) in a project.

List project commits

Create a new commit (project version) in a project.

Create project commit

List the test results for a project commit (project version).

List commit test results

Create an inference pipeline in a project.

Create inference pipeline

List the inference pipelines in a project.

List inference pipelines

Publish an inference data point to an inference pipeline.

Publish inference

Update an inference data point in an inference pipeline.

List the latest test results for an inference pipeline.

List pipeline test results

OpenAI & Azure OpenAI

Anthropic

LangChain

Mistral AI

Groq

Amazon SageMaker

Label Studio

Great Expectations

Amazon S3

Databricks

Slack

GitHub

Ragas

Product updates

Get started

Set up tests

Test your system offline

Monitor your live system

Other resources

Introduction

Start now