lpgcast.bench¶
The benchmark: a corpus of texts with the graphs they state, a scorer that
needs no model as judge, and the lpgcast-bench command.
Modules:
| Name | Description |
|---|---|
cli |
|
compare |
Paired comparison of two benchmark reports, episode by episode. |
corpus |
The benchmark corpus: texts, the schema they are cast against, and the graph |
oracle |
The oracle: a model that replies with the expected graph. |
report |
Rates from pooled counts, and the Markdown and JSON reports. |
run |
Run the corpus through a model, or the oracle, and score every episode. |
score |
Score a cast episode against its expected graph. |