Skip to content

ontocast.tool.agg.coreference

Coreference candidates for the model to resolve, and what its verdicts do.

Units render in parallel and never see each other's entities, so whether two mentions from different units denote one thing is decided after aggregation. The symbolic tiers settle most pairs; the ones they cannot settle are put to the model, one call per document:

  • unconfirmed_merge -- merged on a plausible lexical tier only (label overlap, string similarity, an IRI-derived name), never on an equal label, initials, a shared key or a confirmation. A different verdict splits it.
  • near_miss -- similar enough to be a candidate, refused for lexical alone, with no guard saying the two are distinct. A same verdict merges.
  • shared_link -- two entities of one type from different units, the only two that point to one node through one predicate (both "applicant in" one case), with no name or embedding evidence: a role phrase and a name for one referent.

A verdict is evidence, not a decision: same merges like a shared key and different vetoes, and every distinctness guard still applies.

Attributes

__all__ = ['CoreferenceCandidate', 'coreference_candidates', 'describe_entity', 'verdict_pairs'] module-attribute

Classes

CoreferenceCandidate

Bases: BaseModel

A pair of unit-minted entities whose identity is put to the model.

kind says why: unconfirmed_merge (merged on a plausible lexical tier only), near_miss (refused for lexical alone) or shared_link (same type, different units, pointing to one node by one predicate).

Source code in ontocast/onto/model.py
class CoreferenceCandidate(BaseModel):
    """A pair of unit-minted entities whose identity is put to the model.

    ``kind`` says why: ``unconfirmed_merge`` (merged on a plausible lexical
    tier only), ``near_miss`` (refused for ``lexical`` alone) or
    ``shared_link`` (same type, different units, pointing to one node by one
    predicate).
    """

    left: str
    right: str
    kind: Literal["unconfirmed_merge", "near_miss", "shared_link"]
    similarity: float = 0.0

Attributes

kind instance-attribute
left instance-attribute
right instance-attribute
similarity = 0.0 class-attribute instance-attribute

Functions:

coreference_candidates(*, unconfirmed_merges, lexical_near_misses, units, final_uri, max_pairs)

The pairs to put to the model, at most max_pairs.

Only entities units minted are asked about; catalog terms are not, and neither are entities of a citation-metadata unit: authors and venues of a reference list would spend the cap on bibliography. The three kinds take turns, so none crowds the others out: the least similar unconfirmed merges first (the likeliest wrong), the most similar near misses first (the likeliest right), then shared links.

Parameters:

Name Type Description Default
unconfirmed_merges Sequence[tuple[str, str, float]]

AggregationResult.unconfirmed_merges.

required
lexical_near_misses Sequence[tuple[str, str, float]]

AggregationResult.lexical_near_misses.

required
units Sequence[ContentUnit]

The aggregated units, their graphs unit-scoped.

required
final_uri dict[str, str]

Source entity IRI -> its final IRI after aggregation.

required
max_pairs int

Cap on the pairs returned.

required
Source code in ontocast/tool/agg/coreference.py
def coreference_candidates(
    *,
    unconfirmed_merges: Sequence[tuple[str, str, float]],
    lexical_near_misses: Sequence[tuple[str, str, float]],
    units: Sequence[ContentUnit],
    final_uri: dict[str, str],
    max_pairs: int,
) -> list[CoreferenceCandidate]:
    """The pairs to put to the model, at most ``max_pairs``.

    Only entities units minted are asked about; catalog terms are not, and
    neither are entities of a citation-metadata unit: authors and venues of a
    reference list would spend the cap on bibliography. The
    three kinds take turns, so none crowds the others out: the least similar
    unconfirmed merges first (the likeliest wrong), the most similar near
    misses first (the likeliest right), then shared links.

    Args:
        unconfirmed_merges: ``AggregationResult.unconfirmed_merges``.
        lexical_near_misses: ``AggregationResult.lexical_near_misses``.
        units: The aggregated units, their graphs unit-scoped.
        final_uri: Source entity IRI -> its final IRI after aggregation.
        max_pairs: Cap on the pairs returned.
    """
    merges = [
        CoreferenceCandidate(
            left=left, right=right, kind="unconfirmed_merge", similarity=score
        )
        for left, right, score in sorted(unconfirmed_merges, key=lambda m: m[2])
    ]
    near = [
        CoreferenceCandidate(left=left, right=right, kind="near_miss", similarity=score)
        for left, right, score in sorted(lexical_near_misses, key=lambda m: -m[2])
    ]
    links = _shared_link_candidates(units, final_uri)

    def from_citations(iri: str) -> bool:
        unit = _unit_of(iri, units)
        return unit is not None and unit.is_citation_metadata

    def minted(candidates: list[CoreferenceCandidate]) -> list[CoreferenceCandidate]:
        return [
            c
            for c in candidates
            if unit_scope_index(c.left) is not None
            and unit_scope_index(c.right) is not None
            and not from_citations(c.left)
            and not from_citations(c.right)
        ]

    asked: set[frozenset[str]] = set()
    out: list[CoreferenceCandidate] = []
    for round_ in zip_longest(minted(merges), minted(near), minted(links)):
        for candidate in round_:
            if candidate is None or len(out) >= max_pairs:
                continue
            pair = frozenset((candidate.left, candidate.right))
            if pair in asked:
                continue
            asked.add(pair)
            out.append(candidate)
    return out

describe_entity(iri, units)

What the model sees of one entity: names, types, values, links, a sentence.

Read from the graph of the unit that minted it, so each side of a pair is shown as its own unit stated it. The IRI is never shown: a local name is an artefact of minting, two units can mint the same one for unrelated things, and the model would take it as evidence.

Source code in ontocast/tool/agg/coreference.py
def describe_entity(iri: str, units: Sequence[ContentUnit]) -> str:
    """What the model sees of one entity: names, types, values, links, a sentence.

    Read from the graph of the unit that minted it, so each side of a pair is
    shown as its own unit stated it. The IRI is never shown: a local name is
    an artefact of minting, two units can mint the same one for unrelated
    things, and the model would take it as evidence.
    """
    unit = _unit_of(iri, units)
    if unit is None or unit.graph is None:
        return "(not described)"
    graph = unit.graph
    node = URIRef(iri)
    labels = sorted(str(o) for o in graph.objects(node, RDFS.label))
    types = sorted(_local(o) for o in graph.objects(node, RDF.type))
    values: list[str] = []
    relations: list[str] = []
    for predicate, obj in sorted(graph.predicate_objects(node), key=str):
        if predicate in (RDF.type, RDFS.label):
            continue
        if isinstance(obj, RDFLiteral):
            if len(values) < _MAX_VALUES:
                values.append(f"{_local(predicate)} = {str(obj)[:60]}")
        elif isinstance(obj, URIRef) and len(relations) < _MAX_RELATIONS:
            name = next(
                iter(sorted(str(o) for o in graph.objects(obj, RDFS.label))), ""
            )
            relations.append(f"{_local(predicate)} -> {name or _local(obj)}")
    parts: list[str] = []
    if labels:
        parts.append("labels: " + "; ".join(f'"{label}"' for label in labels))
    if types:
        parts.append("types: " + ", ".join(types))
    if values:
        parts.append("values: " + "; ".join(values))
    if relations:
        parts.append("relations: " + "; ".join(relations))
    sentence = _source_sentence(unit.text, labels)
    if sentence:
        parts.append(f'source: "{sentence}"')
    return "\n    ".join(parts) or "(not described)"

verdict_pairs(candidates, verdicts)

(vetoes, confirmations) from verdicts keyed by 1-based pair number.

same confirms a pair, different vetoes it; unsure and pairs with no verdict change nothing.

Source code in ontocast/tool/agg/coreference.py
def verdict_pairs(
    candidates: Sequence[CoreferenceCandidate],
    verdicts: dict[int, str],
) -> tuple[set[frozenset[URIRef]], set[frozenset[URIRef]]]:
    """``(vetoes, confirmations)`` from verdicts keyed by 1-based pair number.

    ``same`` confirms a pair, ``different`` vetoes it; ``unsure`` and pairs
    with no verdict change nothing.
    """
    vetoes: set[frozenset[URIRef]] = set()
    confirmations: set[frozenset[URIRef]] = set()
    for number, candidate in enumerate(candidates, start=1):
        pair = frozenset((URIRef(candidate.left), URIRef(candidate.right)))
        verdict = verdicts.get(number)
        if verdict == "same":
            confirmations.add(pair)
        elif verdict == "different":
            vetoes.add(pair)
    return vetoes, confirmations