Telemetry counters¶
This page lists every field and counter a run reports, and the code that emits it, so you can read a value you have not seen before or check that a change emits what it should. Reading run telemetry explains which fields answer which question.
Budget fields¶
BudgetTracker (ontocast/onto/state.py) is returned as outcome.metadata.budget by
/process and /process_unit and written as the run manifest's budget. Each
content unit runs with its own tracker, merged into the document's when the
unit finishes: fields and counters are summed, and duration keys ending in
_max keep the maximum.
| Field | Meaning |
|---|---|
calls_count |
Calls sent to the provider, including calls that timed out |
cache_hits |
Calls answered from the LLM response cache |
chars_sent, chars_received |
Prompt and answer characters, for sent and cached calls alike. A timed-out call adds its prompt |
input_tokens, output_tokens |
Tokens of calls sent to the provider |
cached_input_tokens, cached_output_tokens |
Tokens stored with cached answers. Cache entries written without usage add nothing |
reasoning_tokens |
Reasoning tokens of sent calls, inside output_tokens |
cache_read_input_tokens |
Input tokens of sent calls served from the provider's prompt cache, inside input_tokens |
cache_creation_input_tokens |
Input tokens of sent calls written to the provider's prompt cache |
replayed_reasoning_tokens, replayed_cache_read_input_tokens, replayed_cache_creation_input_tokens |
The same three, stored with cached answers |
ontology_triples_generated, ontology_operations_count |
Triples and update operations the ontology renders produced |
facts_triples_generated, facts_operations_count |
The same for facts renders |
node_durations |
Seconds per key; see Duration keys |
counters |
Event counts per key; see Budget counters |
tokens_by_phase |
Per LLMPhase (ontocast/onto/enum.py): calls, replays and the token counts of a RunMeter TokenTotals. The phase is set with use_llm_phase (ontocast/tool/llm.py), and an enclosing phase wins over an inner one |
Two fields are derived when the budget is serialized:
| Field | Formula | null when |
|---|---|---|
prefix_cache_hit_rate |
(cache_read_input_tokens + replayed_cache_read_input_tokens) / (input_tokens + cached_input_tokens) |
No input tokens were reported |
reasoning_share_of_output |
(reasoning_tokens + replayed_reasoning_tokens) / (output_tokens + cached_output_tokens) |
No output tokens were reported |
Both ratios cover sent and cached calls together: they describe the workload, not what this run paid for.
Duration keys¶
node_durations holds seconds. A plain pipeline node name is that node's wall
clock, recorded by the _timed wrapper in ontocast/stategraph/create.py:
Convert to Text, Chunk Text, Resolve Home Ontology, Plan Ontology,
Update Ontology, Normalize Ontology Updates, Structural Check,
Consistency Critic, Render Facts, Merge Facts, Resolve Coreference,
Validate Facts, Serialize.
The two fan-out nodes, Update Ontology and Render Facts, add keys written
in ontocast/stategraph/node_factories.py:
| Key | Meaning |
|---|---|
<node>/unit_sum |
Each unit's loop time, summed over units |
<node>/worker_wait |
Time units waited for a PARALLEL_WORKERS slot, summed |
<node>/loop_lag_total |
Time the event loop ran late while the units ran (ontocast/util/loop_lag.py) |
<node>/loop_lag_max |
The longest single delay |
BudgetTracker.parallel_efficiency(node) returns <node>/unit_sum divided by
the node's wall clock, the effective number of workers, which the end-of-run
log prints as Effective workers. Waiting on I/O yields the loop and adds no
lag, so lag measures only CPU work done on the event loop.
Stage keys, summed over every call or unit:
| Key | Written in | Time spent |
|---|---|---|
llm/provider |
ontocast/tool/llm.py |
Inside the provider call, including rate-limiter waits |
llm/inflight_wait |
ontocast/tool/llm.py |
Waiting for an LLM_MAX_INFLIGHT slot |
llm/cache_lookup |
ontocast/tool/llm.py |
Reading the LLM response cache |
ctx/merge_document_ontology |
ontocast/stategraph/context_resolver.py |
Merging the document's ontology for the facts fan-out |
ctx/union_document_ontology |
ontocast/stategraph/context_resolver.py |
Building the union chapter (ONTOLOGY_CONTEXT_SCOPE=document) |
ctx/working_graph_copy |
ontocast/stategraph/atomic.py |
Copying the ontology context into a unit's working graph |
prompt/ontology_chapter |
ontocast/agent/render_facts.py |
Building or fetching the facts ontology chapter |
repair/deterministic |
ontocast/agent/render_facts.py, ontocast/stategraph/atomic.py |
LLM-free repairs and finding collection in the unit loops |
ontology_validation/unit_findings |
ontocast/stategraph/atomic.py |
Validating an ontology unit's update |
For one fan-out node the time roughly adds up as
wall × effective workers ≈ llm/provider + llm/inflight_wait
+ <node>/worker_wait + <node>/loop_lag_total + other
Budget counters¶
budget.counters holds event counts.
| Counter | Written in | Counts |
|---|---|---|
llm/calls_timed |
ontocast/tool/llm.py |
Provider calls that returned |
llm/calls_failed |
ontocast/tool/llm.py |
Provider calls that raised, for any reason |
llm/timeouts |
ontocast/tool/llm.py |
Calls abandoned at LLM_REQUEST_TIMEOUT_SECONDS; part of llm/calls_failed |
llm/rate_limited |
ontocast/tool/llm.py |
Throttles that survived the SDK's retries; part of llm/calls_failed |
llm/calls_rejected |
ontocast/tool/llm.py |
Requests the provider refused as configured; part of llm/calls_failed. Any non-zero value means the run stopped |
llm/parse_retry |
ontocast/agent/common.py |
Calls re-sent because the previous answer did not parse or validate |
llm/json_bracket_repair |
ontocast/agent/common.py |
Answers recovered by correcting mismatched closing brackets |
llm/json_surplus_closer |
ontocast/agent/common.py |
Answers recovered by dropping closing brackets after a complete object |
llm/json_premature_closer |
ontocast/agent/common.py |
Answers recovered by dropping a closing brace written after the first member, before the rest |
llm/parse_abandoned |
ontocast/agent/common.py |
Calls given up after the retries ran out or the same JSON error recurred |
rdf/turtle_repair |
ontocast/onto/rdfgraph.py |
Turtle payloads that parsed only after repair (truncated statement, unknown prefix, update-query wrapper) |
rdf/jsonld_rdflib_fallback |
ontocast/onto/rdfgraph.py |
JSON-LD payloads that failed URDNA2015 normalization and were parsed by rdflib instead |
repair/compact_iri_literal |
ontocast/tool/facts_validation/literal_repair.py |
Compact IRIs written as plain strings on an IRI-valued position and coerced back to IRIs: in facts renders and their critic and completion patches, and on the schema predicates of ontology renders |
repair/entailed_type |
ontocast/tool/facts_validation/literal_repair.py |
rdf:type triples added from a predicate's rdfs:domain or rdfs:range, or on an untyped node from the one class its links list, in renders and in critic and completion patches |
repair/paired_object_property |
ontocast/tool/facts_validation/literal_repair.py |
Nodes moved off a well-known datatype property onto the object property its vocabulary pairs with it (sosa:hasSimpleResult to sosa:hasResult) |
repair/triple_reversed |
ontocast/tool/facts_validation/literal_repair.py |
Triples turned around in a facts render because both ends were outside the predicate's domain and range as written and fit them swapped |
llm/ontology_selection |
ontocast/agent/select_ontology_catalog.py |
Ontology selection calls, one per unit in selected_single_ontology mode |
chapter/text_chars_before |
ontocast/agent/render_facts.py |
Summed length of capped text literals before the text caps |
chapter/text_chars_after |
ontocast/agent/render_facts.py |
The same after the caps |
chapter/literals_clipped |
ontocast/agent/render_facts.py |
Literals the caps shortened |
chapter/text_over_budget |
ontocast/agent/render_facts.py |
Chapters still over ONTOLOGY_TEXT_TOTAL_BUDGET after every role was tightened |
chapter/triples_over_budget |
ontocast/agent/render_facts.py |
Facts chapters still over ONTOLOGY_CONTEXT_MAX_TRIPLES after condensing, passed through whole |
ctx/merge_document_ontology.calls |
ontocast/stategraph/context_resolver.py |
Document ontology merges; 1 per document |
ctx/facts_catalog_ontologies |
ontocast/stategraph/context_resolver.py |
Catalog ontologies added, read-only, to the document's facts context beside its home ontologies: those the ontology units read, and those a home imports or extends |
ctx/union_document_ontology.calls |
ontocast/stategraph/context_resolver.py |
Union chapters built; 1 per facts-only document with a document scope |
ctx/admitted_planned_terms |
ontocast/stategraph/context_resolver.py |
Terms the ontology plan reuses that were added to the facts ontology context |
ctx/admitted_named_terms |
ontocast/stategraph/context_resolver.py |
Catalog terms the facts prompt names (conformance rules, notes) whose definitions were added to the ontology context |
vector/transport_retry |
ontocast/tool/vector_store/qdrant.py |
Qdrant reads retried after a transport error; the third failure fails the unit |
<node>/unit_errors |
ontocast/stategraph/node_factories.py |
Units of a fan-out node that raised |
ontology/render_skipped_reuse_only |
ontocast/agent/render_ontology.py |
Ontology update renders answered without a call because the document's plan mints nothing |
ontology/update_rejected_over_budget |
ontocast/agent/render_ontology.py |
Ontology updates discarded for exceeding ONTOLOGY_MAX_TRIPLES |
ontology/deletes_dropped_insert_only |
ontocast/agent/render_ontology.py, ontocast/stategraph/node_factories.py |
Ontology delete triples dropped under ONTOLOGY_UPDATE_MODE=insert_only |
ontology/model_header_triples_dropped |
ontocast/agent/render_ontology.py |
owl:Ontology header triples a fresh render wrote and that were dropped: identity and the version header are written by code |
ontology/labels_derived |
ontocast/stategraph/atomic.py |
rdfs:label triples derived from local names for minted terms left unlabelled |
llm/ontology_identity |
ontocast/agent/resolve_home_ontology.py |
Home-ontology calls, one per document that can create vocabulary |
chunk/outline_windows_failed |
ontocast/tool/chunk/section_llm.py |
Outline windows of the model section tier whose call raised; those regions got no model vote |
serialize/ontology_identity_conflict |
ontocast/toolbox.py |
Ontology versions not written because their IRI is registered under another id; the facts are still written |
calls_count equals llm/calls_timed + llm/timeouts: a call that raised for
any other reason is not counted in calls_count. The
chapter/* counters are recorded once per chapter built, not per call that
reads it. chapter/triples_over_budget is recorded whenever a chapter stays
over its triple budget; the other four only when at least one text cap is set.
How prompts are built describes the caps.
Retrieval metrics¶
AgentState.retrieval_metrics is returned as outcome.metadata.retrieval_metrics and
written into the run manifest. Its top-level keys are the members of
RetrievalMetric in ontocast/onto/enum.py. A key appears only when the stage
that writes it ran.
Ontology context, written per unit in ontocast/stategraph/context_resolver.py
and merged onto the document, the last unit winning on shared keys:
| Key | Meaning |
|---|---|
ontology_context_mode |
The mode that resolved the context |
ontology_snapshot_triples |
Triples in the resolved context; for the facts fan-out, the shared document context when there is one |
empty_snapshot_reason |
Why a unit's context was empty |
patch_retrieval |
Retrieval detail in selected_vector_search_ontology mode; see below |
Ontology fan-out, written in ontocast/stategraph/node_factories.py:
| Key | Meaning |
|---|---|
ontology_writable_count, ontology_primary_units |
Distinct writable ontologies, and units assigned a primary one |
ontology_findings_residual, ontology_mandatory_residual |
Findings left on the ontology units after their loops, all and mandatory |
ontology_critic_calls, ontology_critic_accepted |
Ontology critic calls, and those whose verdict let the unit leave the loop |
ontology_critic_fixes_applied, ontology_critic_fixes_noop |
Critic fixes that reached the graph, and fixes that removed exactly what they added |
ontology_critic_patches_rolled_back |
Critic patch passes in which a fix was undone |
ontology_critic_units_skipped |
Units not sent to the critic: at one pass, units with no mandatory finding |
Facts fan-out, written in ontocast/stategraph/node_factories.py:
| Key | Meaning |
|---|---|
facts_anchor_count, facts_anchor_units |
Distinct anchor ontologies, and units assigned one |
facts_findings_residual, facts_mandatory_residual |
Findings left on the facts units after their loops, all and mandatory |
facts_mandatory_residual_by_kind |
The mandatory ones counted per finding kind |
facts_statements_without_subject_rendered, facts_statements_without_subject |
statement_without_subject findings: on the units' renders, and left after the critic and gap passes |
facts_critic_calls, facts_critic_accepted |
Facts critic calls, and those whose verdict let the unit leave the loop |
facts_critic_fixes_applied, facts_critic_fixes_residual, facts_critic_fixes_noop |
Fixes that reached the graph, fixes left for judgement, and fixes that removed exactly what they added |
facts_critic_fixes_rolled_back, facts_critic_patches_rolled_back |
Fixes undone for leaving the unit worse, and passes in which that happened |
facts_critic_fixes_junk_refused |
Inserts refused for minting a placeholder node |
facts_critic_fixes_unresolved_prefix |
Fixes naming a prefix nothing declares |
facts_critic_units_unreviewed, facts_critic_units_skipped |
Units whose critic call failed, and units not sent to the critic |
facts_completion_calls, facts_completion_triples_inserted, facts_completion_measurements_recovered |
The gap pass: calls, triples that stayed, and missed measurements found. Each pass's loop attempt also records n_subjects_linked |
facts_completion_detached_linked |
Instances linked to nothing before a gap pass and linked after it (n_detached_linked on the pass's loop attempt) |
facts_domain_repair_calls, facts_domain_repair_fixed, facts_domain_violations_dropped |
The domain repair: calls (loop attempts of kind domain_repair), statements given a property that fits their subject, and statements removed because they were still in domain violation when the unit left its loop (domain_violation_dropped repair records) |
Aggregation and validation, written in ontocast/stategraph/node_factories.py,
ontocast/stategraph/facts_gate.py and ontocast/tool/facts_validation/gate.py:
| Key | Meaning |
|---|---|
facts_rejected_merges |
Entity merges the aggregator's guards refused, for the graph returned |
facts_merge_rejections_by_reason |
Those refusals by reason (same_unit, sibling, literal_conflict, functional_iri_conflict, initials_conflict, role, type, lexical, cluster_veto); a pair refused for several reasons counts under each |
facts_merge_repair_passes, facts_merge_vetoes, facts_merge_repairs_rejected |
The un-merge repair: passes kept, merge pairs vetoed, passes reverted |
facts_coreference_pairs, facts_coreference_same, facts_coreference_different |
Coreference resolution: pairs put to the model, and those it judged one entity or two |
validated_without_ontology_context |
true when the facts were validated with an empty ontology context |
facts_validation_findings, facts_validation_errors |
Findings after validation, and those of error severity |
facts_shacl_violations_before, facts_shacl_violations_after |
SHACL violations around the automatic repair. Present only when the repair ran |
facts_shacl_repairs, facts_shacl_autofix_passes, facts_shacl_autofix_reverted |
Repairs applied, passes kept, and whether a pass was reverted. Present only when the repair ran |
facts_untyped_inferred |
Instances untyped after aggregation that the gate typed from the domains and ranges of their links, against the whole ontology context |
facts_untyped_dropped |
Instances dropped at the gate for carrying no concrete rdf:type after aggregation; their IRIs are AgentState.facts_untyped_dropped |
facts_self_loops_dropped |
Statements joining a node to itself (x p x) dropped at the gate, unless p is owl:sameAs, rdf:type or declared owl:ReflexiveProperty; each is a self_loop_dropped record in facts_gate_repairs |
facts_unlinked_dropped |
Instances dropped at the gate for stating nothing (linked to no other instance nor the document, no value of their own), plus values no statement carries, plus value-typed instances linked to nothing; each drop is a facts_gate_repairs record with its label and the triples removed |
facts_date_values_added |
Nodes standing for a date (object of a temporal property, typed with an OWL-Time class, or labelled with a whole date) that held only a label, given the typed ISO 8601 value the label names |
facts_links_removed |
Links between instances that the gate's SHACL prune and untyped drop removed |
facts_components, facts_isolated |
Connected groups of instances in the served facts (the document node joins what it links), and instances linked to no other instance (each holds a value of its own), in the served graph. The instances themselves are listed in the run manifest's facts_isolated (IRI, label, holds_value or other) |
facts_twin_labels |
Labels that two or more served instances share: duplicates aggregation left, or distinct things with one name |
structural_ontology_components_max |
The highest connected-component count among the ontologies the structural check examined |
consistency_conflicts |
Conflicts the consistency critic reported |
/process and /process_unit write the validation keys through the same
function, so the two routes report the same set for the same graph.
patch_retrieval¶
patch_retrieval is the retriever's own record
(ontocast/tool/vector_store/patch_retriever.py) of the unit that wrote last.
Its main keys:
| Key | Meaning |
|---|---|
query_count, top_k, effective_max_atoms |
Queries run, hits per query, and the term budget |
candidate_hits, threshold_rejected, atoms_after_dedupe, atoms_final |
The funnel from raw hits to terms kept |
seed_iris, seeds_by_ontology, relevance_by_ontology |
Terms kept, their count per ontology, and each ontology's best score |
lexical_trigger_hits, lexical_trigger_promoted, lexical_trigger_appended |
Terms found by exact name match, and how they entered the result |
module_closure_iris, module_closure_triples, module_closure_declined_iris |
Small ontologies included whole, what they added, and those the closure budget left out |
snapshot_triple_count |
Triples in the assembled context |
ontology_rank_diagnostics |
Per-query ontology ranking, when ONTOLOGY_PATCH_DUMP_ONTOLOGY_RANKS is on |
Retrieval explains the funnel.
Run manifest blocks¶
RunManifest (ontocast/onto/run_manifest.py) is written by
dump_run_manifest in ontocast/api/process_helpers.py as <name>.run.json
beside each <name>.facts.ttl, by ontocast process only. Fields that are null are
left out.
| Field | Contents |
|---|---|
source, line_number |
Input file name, and the line for a multi-record .jsonl |
ontocast_version, render_mode, current_domain, doc_iri, tenant, project |
What ran, and where its IRIs and graphs went |
loops |
The effective max_visits, facts_critic_passes, ontology_critic_passes |
llm |
Provider, model, temperature, the Ollama and reasoning settings, requests_per_second, max_retries, prompt_cache_key, max_inflight |
prompting |
llm_graph_format, llm_output_layout, the resolved ontology_chapter_format, ontology_context_scope, fanout_warmup_units, parallel_workers, embedding_model_name. Scope and warm-up are left out on the single-unit path |
budget |
The full budget, as above |
critic, ontology_critic |
Per loop: calls, accepted, score minimum, median, maximum and histogram, fix severity histograms, accept_reason_histogram, incumbent_accepted, patch passes, fix outcomes (fixes_applied, fixes_noop, fixes_rolled_back, fixes_junk_refused, fixes_unresolved_prefix, patches_rolled_back), units_unreviewed, units_skipped, triples deleted and inserted |
completion |
calls, units, subjects_inserted, subjects_rolled_back, triples_inserted, measurements_recovered |
selection |
target_sections, exclude_sections, summarize_sections, summary_max_sentences, bibliography_mode, non_content_mode, labeled_units, unlabeled_units, section_label_histogram, label_strength_chars (characters of the kept units with a strong, weak or no label), the units skipped as bibliography, undersized or non-content, route_histogram (<route>/<reason> → units and characters, for every prepared unit), with target_sections the characters it removed (allowlist_excluded_chars) and how many of them were unlabeled (allowlist_excluded_unlabeled_chars), and with CHUNK_SECTION_CLASSIFIER=llm the model tier's outline_labels (windows, windows_failed, errors by error class) |
validation_config |
context_from_units, json_mode, shapes_prompt_contract, shapes_prompt_selection, shapes_triples, shacl_inference, numeric_coverage_mandatory, and the length (not the text) of the facts user instruction |
ontology_reduce_metrics |
Counts from reducing the units' ontology updates: duplicate minted terms and their pairs, deletes dropped, unattributed inserts and deletes, fresh graphs unioned and their statements refiled or dropped, artifacts produced, and the apply step's own counts (apply_deletes_no_match, apply_header_triples_dropped, apply_structure_deletes_dropped, apply_structure_deletes_expanded), the planned sides and parents written over a render's (plan_sides_written, plan_parents_written), owned class sides restated as listed classes (owned_sides_listed), undeclared own IRIs in a side or parent rewritten or removed (undeclared_owned_references_rewritten, undeclared_owned_references_dropped), the minted properties' generic and missing domains and ranges after consolidation (generic_property_sides, missing_property_sides), and the minted terms whose every parent is generic (generic_parents) |
graph_metrics |
The written facts graph: nodes, edges, components, largest_component, isolated_nodes |
ontology_triples, facts_triples, facts_triples_serialized |
Ontology triples written; facts triples in memory, provenance included; facts triples in the .facts.ttl file |
retrieval_metrics |
The same mapping /process returns |
ontology_versions |
Ontology versions the run wrote (role created or updated) and read unchanged (role read), each with iri and version_iri |
home_ontology |
The document's home ontology: iri, ontology_id, namespace (every term the document minted is under it) and, for an extension ontology, extends. Absent when the ontology lane did not run |
unit_context_sources |
Ontology IRIs behind each unit's context, by lane (ontology, facts) and unit index |
effective_config, chunking_mode |
Every setting keyed by environment variable, credentials redacted; the chunker's mode after any fallback |
Score buckets in score_histogram are decades keyed like "70-79"; an empty
histogram with calls above zero means the critic returned no score that
could be read.