Get started

First, you must create a project.

Navigate and log into Openlayer. Click the “plus” button next to “Projects,” on the sidebar, to create a new project.

The project is the place on the Openlayer platform that will house your models, data, and tests. You can learn more about projects here.

Select a template on the modal that appears. You can pattern-match based on the programming language, the framework, the AI/ML task type, or the use case.

Not sure which one to use? How about OpenAI in Python?

You must connect to your GitHub account after you select your template. This is important because Openlayer will clone a copy of the template into your GitHub account. Then, every new commit that includes changes to your AI system also gets pushed to Openlayer.

Make sure to provide the environment variables required to deploy the template. The example above requires an OpenAI API key, which is used to run the model on a validation set. The validation set for the templates only has three rows, so the costs are negligible.

After the GitHub repo is successfully connected, Openlayer will pull your latest commit and run a series of processing steps. These include validating your files, iterating over your datasets, and running your model to get its predictions.

In your GitHub repo, you can see that Openlayer started to run on your latest commit.

You can check out the logs on the Openlayer platform to understand the processing steps and debug any issues that arise. Once the commit finishes processing, you can move to the next step.

Tests materialize expectations around your model and data. Openlayer offers an ever-increasing suite of tests targetting different parts of your system.

After your first commit, a few general-purpose tests are automatically created. In this example, some are passing and others aren’t.

You are encouraged to collaborate with your team as you create more tests and define the success criteria for your system. To create more tests, navigate to the different test categories (“Integrity”, “Consistency”, and “Performance”) on the sidebar.

Your new commits to the Git repo will trigger your Openlayer tests.

This is how you make Openlayer part of your CI/CD system, targeting specifically the AI/ML components of your system.

The same template repo has all the information needed to use Openlayer’s monitoring mode and log your live requests to Openlayer.

In general, you’ll only need to set a few environment variables that specify your Openlayer API key and project name.

As your system receives live requests, they are streamed to the Openlayer platform, where you can see their traces, and define tests that run periodically on top of your data.

You can set up alerts to get notified immediately if your tests start failing.

Refer to the template README and the Monitoring overview for details.

Yes. To use Openlayer’s development mode, you only need to set up a way to push your artifacts to the Openlayer platform after each iteration of development. There are three ways to set up pushes to the Openlayer platform:

1. Git — exemplified in this guide with GitHub.
2. Openlayer CLI.
3. Openlayer REST API.

Check out the development mode Overview for details.

Yes. As your dataset grows in size, it is natural to not push it to Git. Instead, you can keep it elsewhere, such as in a blob store. You can still use Openlayer if your datasets are not in Git. Refer to the Configuring output generation guide for details.

Openlayer is SOC 2 Type II compliant, which means that we passed a series of external security assessments and adhere to the industry’s best security practices.

If you prefer, Openlayer also offers an on-prem solution. In this case, Openlayer is installed in your infrastructure and your data will never leave it. Reach out if this is a solution you are interested in.

In monitoring, instead of pushing artifacts after each round of development, you would be publishing production data to Openlayer. Openlayer tests, then, run at a regular cadence on the production data. Refer to the Monitoring examples and Monitoring overview for details.