Definition
The edit distance test measures the minimum number of single-character insertions, deletions, or substitutions required to transform one string into another, serving as a measure of their similarity.Taxonomy
- Task types: LLM.
- Availability: and .
Why it matters
- Edit distance provides a character-level measure of how different two strings are, which is useful for evaluating text generation quality.
- This metric is particularly valuable when you need to measure fine-grained differences between generated and expected text.
- Lower edit distances indicate higher similarity between the generated output and the reference text.
Required columns
To compute this metric, your dataset must contain the following columns:- Outputs: The generated text from your LLM
- Ground truths: The reference/expected text to compare against
Test configuration examples
If you are writing atests.json, here are a few valid configurations for the edit distance test:
Related
- BLEU score test - Measure n-gram based text similarity.
- Exact match test - Assess identical string matches.
- Quasi-exact match test - Allow partial matches and variations.
- Aggregate metrics - Overview of all available metrics.