Skip to content

graflo.link.reconcile

Keep a graph's stated relations to what its statements resolve to.

A producer may write each affirmed relation as an edge when it states it, or leave every stated edge to reconcile; either way :func:reconcile reads every relation statement (:func:~graflo.link.assertions.read_statements), resolves them under the schema's slot rules (:func:~graflo.link.statements.resolve) as_of a time, by default the graph's own (:func:~graflo.link.assertions.graph_time), and materialises each resolved relation slot:

  • a stored edge that statements back but that does not hold is deleted;
  • a value that holds and is not stored is written, through the writer, so a merge: replace edge replaces its source's other target. A planned value deleted while pending comes back once it is due.

An edge no statement backs is never deleted here; writing a replace edge may still displace it, as the edge declares. On a connection that cannot delete edges the stale ones stay and are counted as kept, and the value that holds in a single-valued slot is written again, so it is the slot's last.

An exclusive edge whose source already holds another target is refused by the writer; the stored target stays and the refusal is reported, not raised.

A written edge carries the reserved fields its statements' assertions carry (a list field takes the union), so it is visible where they are; a value whose assertions do not all carry them is left unwritten and counted.

Attributes

Pair = tuple[str, str] module-attribute

(source key, target key), each as :func:entity_key gives it.

__all__ = ['ReconcileReport', 'reconcile'] module-attribute

Classes

ReconcileReport

Bases: ConfigBaseModel

What one reconcile weighed, removed and wrote, per relation.

Source code in graflo/link/reconcile.py
class ReconcileReport(ConfigBaseModel):
    """What one reconcile weighed, removed and wrote, per relation."""

    as_of: datetime | None = None
    statements: int = 0
    conflicts: int = 0
    removed: dict[str, int] = Field(default_factory=dict)
    kept: dict[str, int] = Field(
        default_factory=dict,
        description="Stale edges left in place: the connection cannot delete edges.",
    )
    written: dict[str, int] = Field(
        default_factory=dict,
        description="Values sent to the writer; those it refused are under ``refused``.",
    )
    unwritten: dict[str, int] = Field(
        default_factory=dict,
        description="Values that hold but whose statements carry no reserved fields to write them with.",
    )
    refused: list[str] = Field(
        default_factory=list,
        description=(
            "Values of an exclusive relation the stored graph refused, one line "
            "each: the source holds another target, which stays."
        ),
    )

Attributes

as_of = None class-attribute instance-attribute
conflicts = 0 class-attribute instance-attribute
kept = Field(default_factory=dict, description='Stale edges left in place: the connection cannot delete edges.') class-attribute instance-attribute
refused = Field(default_factory=list, description='Values of an exclusive relation the stored graph refused, one line each: the source holds another target, which stays.') class-attribute instance-attribute
removed = Field(default_factory=dict) class-attribute instance-attribute
statements = 0 class-attribute instance-attribute
unwritten = Field(default_factory=dict, description='Values that hold but whose statements carry no reserved fields to write them with.') class-attribute instance-attribute
written = Field(default_factory=dict, description='Values sent to the writer; those it refused are under ``refused``.') class-attribute instance-attribute

Functions:

reconcile(schema, conn_conf, *, as_of=None, thread_field=None) async

Delete stated edges that do not hold and write those that do but are missing.

Parameters:

Name Type Description Default
schema Schema

The graph's logical schema, lifted with Assertion.

required
conn_conf DBConfig

The graph's database.

required
as_of datetime | None

When to resolve at; the graph's own time when left out.

None
thread_field str | None

As for :func:~graflo.link.assertions.read_statements.

None
Source code in graflo/link/reconcile.py
async def reconcile(
    schema: Schema,
    conn_conf: DBConfig,
    *,
    as_of: datetime | None = None,
    thread_field: str | None = None,
) -> ReconcileReport:
    """Delete stated edges that do not hold and write those that do but are missing.

    Args:
        schema: The graph's logical schema, lifted with ``Assertion``.
        conn_conf: The graph's database.
        as_of: When to resolve at; the graph's own time when left out.
        thread_field: As for :func:`~graflo.link.assertions.read_statements`.
    """
    if ASSERTION not in schema.core_schema.vertex_config.vertex_set:
        return ReconcileReport()
    edges = {
        (edge.source, str(edge.relation), edge.target): edge
        for edge in schema.core_schema.edge_config.values()
        if edge.relation is not None
    }
    by_key = {_edge_key(edge): edge for edge in edges.values()}
    rules = slot_rules(schema)
    read = await asyncio.to_thread(_read, schema, conn_conf, edges, as_of, thread_field)
    relations = [s for s in read.found.statements if s.kind == "relation"]
    result = resolve(relations, rules, as_of=read.when)
    report = ReconcileReport(
        as_of=read.when, statements=len(relations), conflicts=len(result.conflicts)
    )
    current: dict[str, dict[Pair, list[str]]] = defaultdict(dict)
    for fact in result.current:
        edge = edges.get((fact.vertex_type, fact.slot, fact.target_type or ""))
        if edge is not None:
            current[_edge_key(edge)][fact.key, str(fact.value)] = fact.supports

    deletes: list[tuple[Edge, list[Pair]]] = []
    to_write: dict[str, dict[Any, list[Any]]] = defaultdict(lambda: defaultdict(list))
    stamps: dict[str, dict[str, Any] | None] = {}
    for key, pairs in sorted(read.stated.items()):
        edge = by_key[key]
        relation = str(edge.relation)
        holds = current.get(key, {})
        stored = read.stored.get(key, set())
        stale = sorted((stored & pairs) - set(holds))
        if stale and read.deletes:
            deletes.append((edge, stale))
            _count(report.removed, relation, len(stale))
        else:
            _count(report.kept, relation, len(stale))
        rule = rules.get((edge.source, relation))
        # Where stale targets stay, the value that holds is written again, last.
        again = (
            {s for s, _ in stale}
            if not read.deletes and rule is not None and rule.single
            else set()
        )
        for (source, target), supports in sorted(holds.items()):
            if (source, target) in stored and source not in again:
                continue
            stamp = _stamp(edge, supports, read.found)
            if stamp is None:
                _count(report.unwritten, relation, 1)
                continue
            label = json.dumps(stamp, sort_keys=True, default=str)
            stamps[label] = stamp or None
            to_write[label][edge.edge_id].append(
                (json.loads(source), json.loads(target), {})
            )
            _count(report.written, relation, 1)

    if deletes:
        await asyncio.to_thread(_delete, schema, conn_conf, deletes)
    for label, rows in sorted(to_write.items()):
        try:
            await write_container(
                schema,
                GraphContainer(edges=dict(rows)),
                conn_conf,
                stamp=stamps[label],
            )
        except WriteRefusal as refusal:
            report.refused.extend(refusal.subjects)
    return report