Min latency

Definition

The min latency test ensures that the minimum latency for the data is within a given range.

Taxonomy

Task types: LLM, tabular classification, tabular regression, text classification.
Availability: .

Why it matters

The minimum latency represents the best-case performance of your model, indicating the fastest possible response time.
This metric helps identify the baseline performance capability and can reveal issues with caching, connection overhead, or model optimization.
Monitoring minimum latency is useful for understanding the lower bound of your system’s performance and ensuring it meets expectations for optimal conditions.

Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the min latency test:

[
  {
    "name": "Min latency above 100 msec",
    "description": "Make sure that the minimum latency is above 100 msec to ensure realistic measurements",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "minLatency",
        "operator": ">",
        "value": 100
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": true,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

[
  {
    "name": "Min latency above 100 msec",
    "description": "Make sure that the minimum latency is above 100 msec to ensure realistic measurements",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "minLatency",
        "operator": ">",
        "value": 100
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

Median latency

Mean squared error (MSE)

⌘I

Get started

Workspace setup

Governance

Observability

Offline testing

Tests

Gateway

Data quality monitoring

Administration

Notifications

Other resources

Definition

Taxonomy

Why it matters

Test configuration examples

​Definition

​Taxonomy

​Why it matters

​Test configuration examples

​Related

Definition

Taxonomy

Why it matters

Test configuration examples

Related