95th latency percentile

Definition

The 95th latency percentile test ensures that the 95th percentile latency for the data is within a given range.

Taxonomy

Task types: LLM, tabular classification, tabular regression, text classification.
Availability: .

Why it matters

The 95th percentile latency provides insight into the performance experienced by the vast majority of users, excluding only the worst 5% of cases.
This metric is commonly used in SLA definitions as it balances between covering most user experiences while filtering out extreme outliers.
Monitoring the 95th percentile helps ensure that nearly all users receive acceptable performance while being practical to achieve.

Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the 95th latency percentile test:

[
  {
    "name": "95th percentile latency below 8000 msec",
    "description": "Make sure that the 95th percentile latency is below 8000 msec",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "p95Latency",
        "operator": "<",
        "value": 8000
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": true,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

[
  {
    "name": "95th percentile latency below 8000 msec",
    "description": "Make sure that the 95th percentile latency is below 8000 msec",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "p95Latency",
        "operator": "<",
        "value": 8000
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

90th latency percentile

99th latency percentile

⌘I

Get started

Workspace setup

Governance

Observability

Offline testing

Tests

Gateway

Data quality monitoring

Administration

Notifications

Other resources

95th latency percentile

Definition

Taxonomy

Why it matters

Test configuration examples

​Definition

​Taxonomy

​Why it matters

​Test configuration examples

​Related

Definition

Taxonomy

Why it matters

Test configuration examples

Related