Skip to content

ontocast.stategraph.facts_gate

The post-aggregation facts invariant gate, shared by both entry paths.

Two callers reach this gate with the same contract and different capabilities:

  • the document graph, at VALIDATE_FACTS -- many units, so the un-merge repair is meaningful;
  • the single-unit path (/process_unit and the CLI --use-unit-pipeline batch), which does not run the graph and calls in after aggregation.

Un-merging re-aggregates retained units against each other, which has no meaning for one unit, so the single-unit caller passes merge_repair=False. Everything else -- validation arguments, SHACL autofix, conformance summary and the emitted metrics -- is identical, and used to be written out twice; a new validate_aggregated_facts argument added to one copy silently skipped the other entry path.

Attributes

MERGE_SIGNATURE_KINDS = frozenset({FactsValidationFindingKind.FUNCTIONAL_VIOLATION, FactsValidationFindingKind.SUSPECT_MULTI_VALUE, FactsValidationFindingKind.DEGENERATE_COREFERENCE}) module-attribute

logger = logging.getLogger(__name__) module-attribute

Classes

Functions:

drop_detached_values(graph, isolated, value_classes, fact_namespaces)

Remove every isolated instance typed as a value, with the parts it owns.

A value belongs to the statement that carries it; its type is one of value_classes (see :meth:~ontocast.tool.facts_validation.terms.CatalogRoles.value_classes). One linked to no instance nor the document has lost its statement, whatever numbers it holds, so it is not a fact. A value linked to anything, in either direction, is kept.

isolated is computed before values are named, so a value's parts are still blank nodes under it. Its parts go with it: the instances and blank nodes it reaches that nothing else points at.

Parameters:

Name Type Description Default
graph RDFGraph

Served facts graph, mutated in place.

required
isolated Sequence[str]

IRIs of the instances linked to nothing.

required
value_classes frozenset[str]

The classes whose instances are values.

required
fact_namespaces Sequence[str]

Namespaces holding minted instances.

required

Returns:

Type Description
list[GraphRepairRecord]

One unlinked_dropped record per dropped value, ordered by IRI: its

list[GraphRepairRecord]

label as target, and the triples removed with it.

Source code in ontocast/stategraph/facts_gate.py
def drop_detached_values(
    graph: RDFGraph,
    isolated: Sequence[str],
    value_classes: frozenset[str],
    fact_namespaces: Sequence[str],
) -> list[GraphRepairRecord]:
    """Remove every isolated instance typed as a value, with the parts it owns.

    A value belongs to the statement that carries it; its type is one of
    ``value_classes`` (see
    :meth:`~ontocast.tool.facts_validation.terms.CatalogRoles.value_classes`).
    One linked to no instance nor the document has lost its statement,
    whatever numbers it holds, so it is not a fact. A value linked to
    anything, in either direction, is kept.

    ``isolated`` is computed before values are named, so a value's parts are
    still blank nodes under it. Its parts go with it: the instances and blank
    nodes it reaches that nothing else points at.

    Args:
        graph: Served facts graph, mutated in place.
        isolated: IRIs of the instances linked to nothing.
        value_classes: The classes whose instances are values.
        fact_namespaces: Namespaces holding minted instances.

    Returns:
        One ``unlinked_dropped`` record per dropped value, ordered by IRI: its
        label as target, and the triples removed with it.
    """
    detached = sorted(
        (
            URIRef(iri)
            for iri in isolated
            if is_value_node(graph, URIRef(iri), value_classes)
        ),
        key=str,
    )
    records: list[GraphRepairRecord] = []
    for value in detached:
        parts = _owned_parts(graph, [value], fact_namespaces)
        label = graph.value(value, RDFS.label)
        records.append(
            GraphRepairRecord(
                kind=FactsGateRepairKind.UNLINKED_DROPPED,
                source=str(value),
                target=str(label) if label is not None else "",
                triple_count=_remove_instances(graph, [value, *sorted(parts, key=str)]),
            )
        )
    if records:
        logger.info(
            "Dropped %d value(s) linked to nothing from the served facts",
            len(records),
        )
    return records

drop_domain_violations(graph, findings)

Remove every statement a mandatory domain_violation finding names.

The finding names a subject and a property whose declared domain the subject's types contradict; every statement of that pair goes, with the blank nodes only it reached and the reifiers quoting it.

Parameters:

Name Type Description Default
graph RDFGraph

A unit's facts graph, mutated in place.

required
findings Sequence[FactsUnitFinding]

The unit's findings; other kinds are ignored.

required

Returns:

Type Description
list[GraphRepairRecord]

One domain_violation_dropped record per (subject, property) pair:

list[GraphRepairRecord]

the subject as source, the property as target.

Source code in ontocast/stategraph/facts_gate.py
def drop_domain_violations(
    graph: RDFGraph, findings: Sequence[FactsUnitFinding]
) -> list[GraphRepairRecord]:
    """Remove every statement a mandatory ``domain_violation`` finding names.

    The finding names a subject and a property whose declared domain the
    subject's types contradict; every statement of that pair goes, with the
    blank nodes only it reached and the reifiers quoting it.

    Args:
        graph: A unit's facts graph, mutated in place.
        findings: The unit's findings; other kinds are ignored.

    Returns:
        One ``domain_violation_dropped`` record per (subject, property) pair:
        the subject as source, the property as target.
    """
    records: list[GraphRepairRecord] = []
    pairs = sorted(
        {
            (finding.subject, finding.predicate)
            for finding in findings
            if finding.kind == FactsUnitFindingKind.DOMAIN_VIOLATION
            and finding.mandatory
            and finding.subject
            and finding.predicate
        }
    )
    for subject, predicate in pairs:
        triples = list(graph.triples((URIRef(subject), URIRef(predicate), None)))
        if not triples:
            continue
        orphan_candidates: set[BNode] = set()
        for triple in triples:
            graph.remove(triple)
            if isinstance(triple[2], BNode):
                orphan_candidates.add(triple[2])
            if graph.has_reifications:
                reifiers = graph.reifications.reifiers_of(triple)
                graph.reifications.remove_reifiers(reifiers)
                for reifier in reifiers:
                    for arc in list(graph.triples((reifier, None, None))):
                        graph.remove(arc)
        orphaned, orphan_triples = _remove_orphaned_blank_nodes(
            graph, orphan_candidates
        )
        drop_reifiers_mentioning(graph, orphaned)
        records.append(
            GraphRepairRecord(
                kind=FactsGateRepairKind.DOMAIN_VIOLATION_DROPPED,
                source=subject,
                target=predicate,
                triple_count=len(triples) + orphan_triples,
            )
        )
    return records

drop_self_loops(graph, ontology_graph)

Remove every statement x p x whose p is not declared reflexive.

A text never relates a thing to itself through an ordinary property: such a statement is a slip (the subject written twice) or two nodes a merge joined. It states nothing, so it goes; owl:sameAs, rdf:type and an owl:ReflexiveProperty the context declares are kept. A reifier goes with its statement.

Returns:

Type Description
list[GraphRepairRecord]

One self_loop_dropped record per statement.

Source code in ontocast/stategraph/facts_gate.py
def drop_self_loops(
    graph: RDFGraph, ontology_graph: RDFGraph | None
) -> list[GraphRepairRecord]:
    """Remove every statement ``x p x`` whose ``p`` is not declared reflexive.

    A text never relates a thing to itself through an ordinary property: such
    a statement is a slip (the subject written twice) or two nodes a merge
    joined. It states nothing, so it goes; ``owl:sameAs``, ``rdf:type`` and an
    ``owl:ReflexiveProperty`` the context declares are kept. A reifier goes
    with its statement.

    Returns:
        One ``self_loop_dropped`` record per statement.
    """
    context = ontology_graph if ontology_graph is not None else RDFGraph()
    loops = sorted(
        (
            (subject, predicate, obj)
            for subject, predicate, obj in graph
            if subject == obj
            and isinstance(predicate, URIRef)
            and predicate not in _REFLEXIVE_BY_DEFINITION
            and (predicate, RDF.type, OWL.ReflexiveProperty) not in context
        ),
        key=lambda triple: tuple(str(term) for term in triple),
    )
    records: list[GraphRepairRecord] = []
    for triple in loops:
        graph.remove(triple)
        records.append(
            GraphRepairRecord(
                kind=FactsGateRepairKind.SELF_LOOP_DROPPED,
                source=f"{triple[0]} {triple[1]}",
                target=str(triple[2]),
            )
        )
    if records:
        if graph.has_reifications:
            reifiers = {r for t in loops for r in graph.reifications.reifiers_of(t)}
            graph.reifications.remove_reifiers(reifiers)
            for reifier in reifiers:
                for arc in list(graph.triples((reifier, None, None))):
                    graph.remove(arc)
        logger.info("Dropped %d self-loop statement(s)", len(records))
    return records

drop_unlinked_instances(graph, fact_namespaces, document)

Remove every instance that states nothing, and every value no statement carries.

Served facts are statements: an instance linked to no other instance nor to document, holding no value of its own (:func:~ontocast.tool.facts_validation.connectivity.silent_instances), is a type and a name, a mention rather than a fact; a blank-node value nothing points to has lost its statement. Runs on the aggregated graph, so a mention another unit links has merged first.

Parameters:

Name Type Description Default
graph RDFGraph

Aggregated facts graph, mutated in place.

required
fact_namespaces Sequence[str]

Namespaces holding minted instances.

required
document URIRef | None

The document node; a link to it counts.

required

Returns:

Type Description
list[GraphRepairRecord]

One unlinked_dropped record per instance, then per carrierless

list[GraphRepairRecord]

value (its source the blank node).

Source code in ontocast/stategraph/facts_gate.py
def drop_unlinked_instances(
    graph: RDFGraph, fact_namespaces: Sequence[str], document: URIRef | None
) -> list[GraphRepairRecord]:
    """Remove every instance that states nothing, and every value no statement carries.

    Served facts are statements: an instance linked to no other instance nor
    to ``document``, holding no value of its own
    (:func:`~ontocast.tool.facts_validation.connectivity.silent_instances`),
    is a type and a name, a mention rather than a fact; a blank-node value
    nothing points to has lost its statement. Runs on the aggregated graph,
    so a mention another unit links has merged first.

    Args:
        graph: Aggregated facts graph, mutated in place.
        fact_namespaces: Namespaces holding minted instances.
        document: The document node; a link to it counts.

    Returns:
        One ``unlinked_dropped`` record per instance, then per carrierless
        value (its source the blank node).
    """
    records = _drop_recorded(
        graph,
        silent_instances(graph, fact_namespaces, document),
        FactsGateRepairKind.UNLINKED_DROPPED,
    )
    silent = len(records)
    for value in sorted(carrierless_values(graph), key=str):
        if (value, None, None) not in graph:
            continue
        label = graph.value(value, RDFS.label)
        orphaned, triple_count = _remove_orphaned_blank_nodes(graph, {value})
        drop_reifiers_mentioning(graph, orphaned)
        records.append(
            GraphRepairRecord(
                kind=FactsGateRepairKind.UNLINKED_DROPPED,
                source=str(value),
                target=str(label) if label is not None else "",
                triple_count=triple_count,
            )
        )
    if records:
        logger.info(
            "Dropped %d instance(s) stating nothing and %d value(s) with no "
            "carrier from the aggregated facts",
            silent,
            len(records) - silent,
        )
    return records

drop_untyped_instances(graph, fact_namespaces)

Remove every instance that has no concrete type, with its triples.

Runs on the aggregated graph so a type asserted in any unit counts. An untyped instance (:func:untyped_instances: a fact-namespace IRI in subject or IRI-object position whose rdf:type names no class outside the fact namespaces other than a metaclass or owl:Thing) loses every triple it is the subject or the object of; blank nodes reachable only through those triples go with them, and so do the RDF 1.2 reifiers quoting any removed statement. A typed neighbour keeps everything else.

Parameters:

Name Type Description Default
graph RDFGraph

Aggregated facts graph, mutated in place.

required
fact_namespaces Sequence[str]

Namespaces holding minted instances.

required

Returns:

Type Description
list[GraphRepairRecord]

One untyped_dropped record per instance, ordered by IRI.

Source code in ontocast/stategraph/facts_gate.py
def drop_untyped_instances(
    graph: RDFGraph, fact_namespaces: Sequence[str]
) -> list[GraphRepairRecord]:
    """Remove every instance that has no concrete type, with its triples.

    Runs on the aggregated graph so a type asserted in any unit counts. An
    untyped instance (:func:`untyped_instances`: a fact-namespace IRI in
    subject or IRI-object position whose ``rdf:type`` names no class outside
    the fact namespaces other than a metaclass or ``owl:Thing``) loses every
    triple it is the subject or the object of; blank nodes reachable only
    through those triples go with them, and so do the RDF 1.2 reifiers
    quoting any removed statement. A typed neighbour keeps everything else.

    Args:
        graph: Aggregated facts graph, mutated in place.
        fact_namespaces: Namespaces holding minted instances.

    Returns:
        One ``untyped_dropped`` record per instance, ordered by IRI.
    """
    records = _drop_recorded(
        graph,
        untyped_instances(graph, fact_namespaces),
        FactsGateRepairKind.UNTYPED_DROPPED,
    )
    if records:
        logger.info(
            "Dropped %d untyped instance(s) from the aggregated facts", len(records)
        )
    return records

run_facts_gate(state, ontology_graph, tools, *, merge_repair, document_metadata=None)

Validate the aggregated facts and apply the LLM-free repair stages.

Literal variants differing only in language tag or datatype are collapsed first (FACTS_LITERAL_VARIANT_DEDUPE), so validation counts values rather than spellings.

When merge_repair is set, merge-signature error findings whose subject resulted from an identity merge turn the offending cluster into pair vetoes and the retained facts units are re-aggregated, up to FACTS_MERGE_REPAIR_PASSES times. Then, on both paths, SHACL autofix (FACTS_SHACL_AUTOFIX) repairs constraint violations in code, instances still untyped take the types their links' domains and ranges entail in the whole ontology context (facts_untyped_inferred), and those still untyped are dropped (:func:drop_untyped_instances; facts_untyped_dropped), and so are instances that state nothing (:func:drop_unlinked_instances; facts_unlinked_dropped). The links these repairs removed (facts_links_removed), how the served instances hang together (facts_components, facts_isolated) and the labels several of them share (facts_twin_labels) are recorded. Residual findings stay on the state as telemetry.

Parameters:

Name Type Description Default
state AgentState

Document state carrying aggregated_facts and the facts units.

required
ontology_graph RDFGraph

Ontology context the facts are validated against.

required
tools ToolBox

Dependency container; only the aggregator is used, and only when merge_repair is set.

required
merge_repair bool

Whether to run the un-merge repair loop. False for the single-unit path, where re-aggregating one unit is a no-op.

required
document_metadata dict | None

Provenance metadata for re-aggregation. Required when merge_repair is set.

None

Raises:

Type Description
ValueError

If merge_repair is set without document_metadata.

Source code in ontocast/stategraph/facts_gate.py
def run_facts_gate(
    state: AgentState,
    ontology_graph: RDFGraph,
    tools: ToolBox,
    *,
    merge_repair: bool,
    document_metadata: dict | None = None,
) -> None:
    """Validate the aggregated facts and apply the LLM-free repair stages.

    Literal variants differing only in language tag or datatype are collapsed
    first (``FACTS_LITERAL_VARIANT_DEDUPE``), so validation counts values
    rather than spellings.

    When ``merge_repair`` is set, merge-signature error findings whose subject
    resulted from an identity merge turn the offending cluster into pair vetoes
    and the retained facts units are re-aggregated, up to
    ``FACTS_MERGE_REPAIR_PASSES`` times. Then, on both paths, SHACL autofix
    (``FACTS_SHACL_AUTOFIX``) repairs constraint violations in code, instances
    still untyped take the types their links' domains and ranges entail in the
    whole ontology context (``facts_untyped_inferred``), and those still
    untyped are dropped (:func:`drop_untyped_instances`;
    ``facts_untyped_dropped``), and so are instances that state nothing
    (:func:`drop_unlinked_instances`; ``facts_unlinked_dropped``). The links
    these repairs removed
    (``facts_links_removed``), how the served instances hang together
    (``facts_components``, ``facts_isolated``) and the labels several of them
    share (``facts_twin_labels``) are recorded. Residual findings stay on the
    state as telemetry.

    Args:
        state: Document state carrying ``aggregated_facts`` and the facts units.
        ontology_graph: Ontology context the facts are validated against.
        tools: Dependency container; only the aggregator is used, and only when
            ``merge_repair`` is set.
        merge_repair: Whether to run the un-merge repair loop. False for the
            single-unit path, where re-aggregating one unit is a no-op.
        document_metadata: Provenance metadata for re-aggregation. Required
            when ``merge_repair`` is set.

    Raises:
        ValueError: If ``merge_repair`` is set without ``document_metadata``.
    """
    if merge_repair and document_metadata is None:
        raise ValueError("merge_repair=True requires document_metadata")

    facts_validation = tools.config.get_tool_config().facts_validation
    shapes_graph = collect_shacl_shapes(ontology_graph, tools.shapes_catalog.graph())
    # The check must see what the unit validator sees, or it reports
    # contradictions no unit ever raised: the whole catalog rather than the
    # union of retrieved snapshots (a term a snapshot omitted is still a
    # term), and the shapes-contract exemptions the unit loop applies.
    _, contract_terms, _ = tools.shapes_prompt_contract()
    catalog_terms = tools.ontology_manager.catalog_terms()
    contradictions = shacl_catalog_contradictions(
        shapes_graph,
        ontology_graph,
        policy=ValidationPolicy(
            additional_standard_namespaces=tuple(
                facts_validation.additional_standard_namespaces
            ),
            quantity_fallback_vocabulary=facts_validation.quantity_fallback_vocabulary,
            code_predicates=tuple(facts_validation.code_predicates),
            contract_exempt_terms=contract_terms,
            vocabulary=facts_validation.vocabulary,
        ),
        catalog_terms=catalog_terms,
    )
    if contradictions:
        # Data cannot satisfy both sides: the shapes demand these properties
        # while the unit validator's mandatory findings order the renderer to
        # remove them. This is a catalog/configuration error, and it silently
        # destroys extracted data (observed live: qudt:numericValue on the
        # catalog). Loud, per document, until the catalog declares the
        # term or the deployment exempts its namespace.
        logger.error(
            "SHACL shapes require %d propert%s the term validator would flag "
            "as unknown: %s. Declare %s in the catalog, or exempt via "
            "FACTS_ADDITIONAL_STANDARD_NAMESPACES / the quantity fallback "
            "vocabulary.",
            len(contradictions),
            "y" if len(contradictions) == 1 else "ies",
            ", ".join(f"<{term}>" for term in contradictions),
            "it" if len(contradictions) == 1 else "them",
        )
    fact_namespaces = [DEFAULT_IRI, str(state.doc_iri), state.doc_namespace or ""]

    def run_validation() -> FactsValidationReport:
        return validate_aggregated_facts(
            state.aggregated_facts,
            ontology_graph,
            shapes_graph=shapes_graph,
            fact_namespaces=fact_namespaces,
            suspect_multi_value_severity=facts_validation.suspect_multi_value_severity,
            functional_min_single_support=(
                facts_validation.functional_min_single_support
            ),
            quantity_fallback_vocabulary=facts_validation.quantity_fallback_vocabulary,
            shacl_inference=facts_validation.shacl_inference,
            shacl_advanced=facts_validation.shacl_advanced,
            shacl_max_triples=facts_validation.shacl_max_triples,
            key_supported_subjects=state.aggregation_key_clusters,
            cross_unit_pairs=(
                state.aggregation_cross_unit_pairs
                if facts_validation.suspect_multi_value_require_cross_unit
                else None
            ),
            non_catalog_severity=(
                "error" if facts_validation.vocabulary == "strict" else "warning"
            ),
            additional_standard_namespaces=tuple(
                facts_validation.additional_standard_namespaces
            ),
            catalog_terms=catalog_terms,
        )

    def merge_signature_errors(report: FactsValidationReport) -> int:
        return sum(
            1
            for finding in report.error_findings
            if finding.kind in MERGE_SIGNATURE_KINDS
        )

    # Collapse literal variants ("X"@en / "X"^^xsd:string / "X") before
    # validating, so multi-value checks count values rather than spellings.
    dedupe_records = (
        dedupe_literal_variants(state.aggregated_facts, fact_namespaces)
        if facts_validation.literal_variant_dedupe
        else []
    )

    report = run_validation()
    vetoes: set[frozenset[URIRef]] = set()
    repair_passes = 0
    rejected_repairs = 0
    # Last *accepted* re-aggregation, i.e. the one whose graph is served.
    last_result: AggregationResult | None = None

    while merge_repair and repair_passes < facts_validation.merge_repair_passes:
        previous_errors = merge_signature_errors(report)
        if not previous_errors:
            break
        new_vetoes = vetoes_from_findings(
            report.error_findings, state.aggregation_clusters, state.facts_units
        )
        if not (new_vetoes - vetoes):
            break
        vetoes |= new_vetoes
        logger.info(
            "Facts validation gate: %d merge-signature error finding(s), "
            "re-aggregating with %d merge veto pair(s)",
            previous_errors,
            len(vetoes),
        )
        result = tools.aggregator.postprocess_facts_units(
            units=state.facts_units,
            ontology_graph=ontology_graph,
            doc_iri=state.doc_iri,
            document_metadata=document_metadata,
            doc_namespace=state.doc_namespace,
            merge_vetoes=vetoes | _pairs(state.coreference_vetoes),
            confirmed_pairs=_pairs(state.coreference_confirmations),
        )
        # Un-merging is destructive: a veto splits correct merges too, so a
        # pass that does not strictly reduce the error count has traded real
        # coreference for nothing and must not be kept.
        previous_graph = state.aggregated_facts
        previous_clusters = state.aggregation_clusters
        previous_key_clusters = state.aggregation_key_clusters
        previous_cross_unit_pairs = state.aggregation_cross_unit_pairs
        state.aggregated_facts = result.graph
        state.aggregation_clusters = result.merged_clusters
        state.aggregation_key_clusters = result.key_supported_clusters
        # Re-aggregation rewrites subject IRIs, so the pairs the validator
        # keys on have to come from the pass it is about to score.
        state.aggregation_cross_unit_pairs = result.cross_unit_object_pairs
        candidate_report = run_validation()
        candidate_errors = merge_signature_errors(candidate_report)
        if candidate_errors >= previous_errors:
            logger.warning(
                "Facts validation gate: repair pass %d did not reduce "
                "merge-signature errors (%d -> %d); reverting to the "
                "pre-repair graph",
                repair_passes + 1,
                previous_errors,
                candidate_errors,
            )
            state.aggregated_facts = previous_graph
            state.aggregation_clusters = previous_clusters
            state.aggregation_key_clusters = previous_key_clusters
            state.aggregation_cross_unit_pairs = previous_cross_unit_pairs
            rejected_repairs += 1
            break
        repair_passes += 1
        last_result = result
        report = candidate_report

    if repair_passes and facts_validation.literal_variant_dedupe:
        # An accepted un-merge pass re-aggregated from the raw units, which
        # reintroduces the literal variants the pre-validation pass removed.
        dedupe_records.extend(
            dedupe_literal_variants(state.aggregated_facts, fact_namespaces)
        )

    # Shape-driven repair, in code: retype literals against sh:datatype,
    # resolve codes to catalog IRIs, drop placeholder nodes. Runs after
    # un-merging so it sees the graph that will actually be served, and
    # reuses the violations the reporting pass just computed.
    links_before = instance_links(state.aggregated_facts, fact_namespaces)
    repair_result = ShaclRepairResult(graph=state.aggregated_facts)
    if shapes_graph is not None:
        repair_result = apply_shacl_repairs(
            state.aggregated_facts,
            shapes_graph,
            ontology_graph,
            mode=facts_validation.shacl_autofix,
            passes=facts_validation.shacl_autofix_passes,
            fact_namespaces=fact_namespaces,
            code_predicates=facts_validation.code_predicates,
            inference=facts_validation.shacl_inference,
            advanced=facts_validation.shacl_advanced,
            max_triples=facts_validation.shacl_max_triples,
            initial_violations=(
                report.shacl_violations if report.shacl_evaluated else None
            ),
        )
        if repair_result.records:
            state.aggregated_facts = repair_result.graph

    # A node standing for a date says which date it is with a typed value.
    date_records = add_date_values(
        state.aggregated_facts, ontology_graph, fact_namespaces
    )
    state.retrieval_metrics[RetrievalMetric.FACTS_DATE_VALUES_ADDED] = len(date_records)

    # A statement joining a node to itself states nothing; dropped before the
    # instances that then state nothing are.
    loop_records = drop_self_loops(state.aggregated_facts, ontology_graph)
    state.retrieval_metrics[RetrievalMetric.FACTS_SELF_LOOPS_DROPPED] = len(
        loop_records
    )

    # Every instance the served graph holds is typed. An instance still
    # untyped once all units have contributed their types takes the class its
    # links entail; whatever stays untyped is removed, with its links.
    # Units materialise entailed types against their own context, which may
    # lack the domain or range the whole context declares.
    inferred_records: list[GraphRepairRecord] = []
    untyped_before = set(untyped_instances(state.aggregated_facts, fact_namespaces))
    if untyped_before:
        _, inferred_records = materialize_entailed_types(
            state.aggregated_facts, ontology_graph
        )
    state.retrieval_metrics[RetrievalMetric.FACTS_UNTYPED_INFERRED] = len(
        untyped_before - set(untyped_instances(state.aggregated_facts, fact_namespaces))
    )
    untyped_records = drop_untyped_instances(state.aggregated_facts, fact_namespaces)
    state.facts_untyped_dropped = [record.source for record in untyped_records][
        :_UNTYPED_DROPPED_STATE_CAP
    ]
    state.retrieval_metrics[RetrievalMetric.FACTS_UNTYPED_DROPPED] = len(
        untyped_records
    )
    unlinked_records = drop_unlinked_instances(
        state.aggregated_facts, fact_namespaces, state.doc_iri
    )
    # A value is part of the statement that carries it: one isolated with no
    # carrier is not a fact, whatever numbers it holds. Values are still
    # blank nodes under their carriers here, so a carried value is linked.
    unlinked_records.extend(
        drop_detached_values(
            state.aggregated_facts,
            sorted(
                str(node)
                for node in isolated_instances(
                    state.aggregated_facts, fact_namespaces, state.doc_iri
                )
            ),
            tools.get_atomic_tools().value_classes(),
            fact_namespaces,
        )
    )
    state.retrieval_metrics[RetrievalMetric.FACTS_UNLINKED_DROPPED] = len(
        unlinked_records
    )
    untyped_records.extend(unlinked_records)

    state.retrieval_metrics[RetrievalMetric.FACTS_LINKS_REMOVED] = links_lost(
        links_before, state.aggregated_facts
    )
    connectivity = facts_connectivity(
        state.aggregated_facts, fact_namespaces, document=state.doc_iri
    )
    state.retrieval_metrics[RetrievalMetric.FACTS_COMPONENTS] = connectivity.components
    state.retrieval_metrics[RetrievalMetric.FACTS_ISOLATED] = connectivity.isolated
    # Kept in the served facts and listed: what is still linked to nothing.
    state.facts_isolated_report = isolated_report(
        state.aggregated_facts,
        isolated_instances(state.aggregated_facts, fact_namespaces, state.doc_iri),
    )[:_ISOLATED_STATE_CAP]
    state.retrieval_metrics[RetrievalMetric.FACTS_TWIN_LABELS] = len(
        twin_labels(state.aggregated_facts, fact_namespaces, state.doc_iri)
    )
    # Values are blank nodes under their carriers until here, so the counts
    # above see them as parts of statements; the served graph names them.
    name_values(state.aggregated_facts)
    # The served graph shows each namespace under the prefix the prompts did.
    tools.prefix_table().bind(state.aggregated_facts)

    if (
        repair_result.records
        or untyped_records
        or inferred_records
        or date_records
        or loop_records
    ):
        report = run_validation()

    gate_repairs = (
        dedupe_records
        + repair_result.records
        + date_records
        + loop_records
        + inferred_records
        + untyped_records
    )
    if gate_repairs:
        state.facts_gate_repairs = gate_repairs

    state.facts_validation_findings = report.findings
    state.facts_conformance = summarize_conformance(
        report.findings,
        shacl_evaluated=report.shacl_evaluated,
        repairs=state.facts_gate_repairs,
        focus_nodes=count_shacl_focus_nodes(state.aggregated_facts, shapes_graph),
    )
    record_facts_gate_metrics(
        state.retrieval_metrics,
        report=report,
        repair_result=repair_result,
        ontology_context_empty=not len(ontology_graph),
    )
    if merge_repair:
        state.retrieval_metrics[RetrievalMetric.FACTS_MERGE_REPAIR_PASSES] = (
            repair_passes
        )
        state.retrieval_metrics[RetrievalMetric.FACTS_MERGE_VETOES] = len(vetoes)
        state.retrieval_metrics[RetrievalMetric.FACTS_MERGE_REPAIRS_REJECTED] = (
            rejected_repairs
        )
    if repair_passes and last_result is not None:
        # merge_facts recorded this against the pre-repair aggregation, so
        # it is stale once a veto pass has re-aggregated. Republish the
        # guard count for the graph that is actually served -- not the veto
        # count, which is a different quantity and is already published as
        # facts_merge_vetoes.
        state.retrieval_metrics[RetrievalMetric.FACTS_REJECTED_MERGES] = (
            last_result.rejected_merge_count
        )
    if report.error_findings:
        logger.warning(
            "Facts validation gate: %d error finding(s) remain after "
            "%d repair pass(es)",
            len(report.error_findings),
            repair_passes,
        )

stored_pairs(pairs)

Entity pairs as the state stores them: sorted (left, right) strings.

Source code in ontocast/stategraph/facts_gate.py
def stored_pairs(pairs: set[frozenset[URIRef]]) -> list[tuple[str, str]]:
    """Entity pairs as the state stores them: sorted ``(left, right)`` strings."""
    out: list[tuple[str, str]] = []
    for pair in pairs:
        left, right = sorted(str(iri) for iri in pair)
        out.append((left, right))
    return sorted(out)

vetoes_from_findings(findings, clusters, units=())

Pair vetoes for merge-signature error findings.

A finding implicates the cluster whose merge could have produced it. A merged subject collects the objects of all its members, so FUNCTIONAL_VIOLATION and SUSPECT_MULTI_VALUE implicate the subject; its objects are innocent (merging objects can only reduce the values a subject carries). DEGENERATE_COREFERENCE reports the pointing node as subject and the over-merged endpoint in values (range1 hasLowerBound v1 ; hasUpperBound v1); either merge can produce it, so it implicates both.

Within an implicated cluster only the members that disagree are split: two members are vetoed when every value one carried on the finding's predicate, in its own unit, conflicts with every value the other carried (for DEGENERATE_COREFERENCE: when they use, or are reached by, different ones of its predicates). Members that carried nothing there keep their merges. When units cannot attribute the disagreement, or are not given, every pair of the implicated cluster is vetoed.

Parameters:

Name Type Description Default
findings list[FactsValidationFinding]

Error findings from the validation report.

required
clusters dict[str, list[str]]

Identity-merge clusters keyed by final IRI; members are the entity IRIs of the unit graphs.

required
units Sequence[ContentUnit]

The aggregated units, whose graphs say which member carried which value.

()

Returns:

Type Description
set[frozenset[URIRef]]

Pairs of IRIs that must not be merged on a re-aggregation pass.

Source code in ontocast/stategraph/facts_gate.py
def vetoes_from_findings(
    findings: list[FactsValidationFinding],
    clusters: dict[str, list[str]],
    units: Sequence[ContentUnit] = (),
) -> set[frozenset[URIRef]]:
    """Pair vetoes for merge-signature error findings.

    A finding implicates the cluster whose merge could have produced it. A
    merged subject collects the objects of all its members, so
    FUNCTIONAL_VIOLATION and SUSPECT_MULTI_VALUE implicate the subject; its
    objects are innocent (merging objects can only reduce the values a subject
    carries). DEGENERATE_COREFERENCE reports the *pointing* node as subject and
    the over-merged endpoint in ``values`` (``range1 hasLowerBound v1 ;
    hasUpperBound v1``); either merge can produce it, so it implicates both.

    Within an implicated cluster only the members that disagree are split:
    two members are vetoed when every value one carried on the finding's
    predicate, in its own unit, conflicts with every value the other carried
    (for DEGENERATE_COREFERENCE: when they use, or are reached by, different
    ones of its predicates). Members that carried nothing there keep their
    merges. When ``units`` cannot attribute the disagreement, or are not
    given, every pair of the implicated cluster is vetoed.

    Args:
        findings: Error findings from the validation report.
        clusters: Identity-merge clusters keyed by final IRI; members are the
            entity IRIs of the unit graphs.
        units: The aggregated units, whose graphs say which member carried
            which value.

    Returns:
        Pairs of IRIs that must not be merged on a re-aggregation pass.
    """
    canonical = {
        member: min(members) for members in clusters.values() for member in members
    }
    graphs = [unit.graph for unit in units if unit.graph is not None]
    vetoes: set[frozenset[URIRef]] = set()
    for finding in findings:
        if finding.kind not in MERGE_SIGNATURE_KINDS:
            continue
        degenerate = finding.kind == FactsValidationFindingKind.DEGENERATE_COREFERENCE
        implicated = (
            [finding.subject, *finding.values] if degenerate else [finding.subject]
        )
        for candidate in implicated:
            members = clusters.get(candidate, [])
            if len(members) < 2:
                continue
            values: dict[str, set[tuple[str, str]]] = {}
            if degenerate:
                # A member's "value" is which of the finding's predicates it
                # uses (pointing side) or is reached by (endpoint side): one
                # endpoint is a lower bound, the other an upper.
                predicates = [URIRef(p) for p in finding.predicate.split(", ") if p]
                pointing = candidate == finding.subject
                for graph in graphs:
                    for member in members:
                        node = URIRef(member)
                        for predicate in predicates:
                            pattern = (
                                (node, predicate, None)
                                if pointing
                                else (None, predicate, node)
                            )
                            if pattern in graph:
                                values.setdefault(member, set()).add(
                                    ("node", str(predicate))
                                )
            else:
                predicate = URIRef(finding.predicate)
                for graph in graphs:
                    for member in members:
                        for obj in graph.objects(URIRef(member), predicate):
                            values.setdefault(member, set()).add(
                                _value_key(obj, canonical)
                            )
            pairs = _conflicting_pairs(members, values)
            if not pairs:
                pairs = {
                    frozenset((URIRef(left), URIRef(right)))
                    for left, right in combinations(sorted(members), 2)
                }
            vetoes |= pairs
    return vetoes