Two people changed the same manifest. How do I merge their changes and keep the history?¶
A person registry keys each person by an internal id. Two people change its
manifest, starting from the same version. One decides that the social security
number identifies a person and adds a flag, ssn_verified. The other decides
that the email identifies a person and adds email_verified.
If the manifest is saved as one version after another, the second change overwrites the first. You want to keep both changes as history. You want to see the one real disagreement, which key identifies a person, next to what the manifest looked like before either change. Everything else should combine without a question, and the result should be recorded with both changes as its parents.
GraFlo records each change of a manifest as a commit, and merges two branches against the commit they both start from: a three-way merge. Combining two manifests that share no history is a different operation, the union shown in Combine two manifests.
flowchart LR
base["97382d5d<br>track when a person was first seen"]
ssn["f42a0133<br>key people by SSN"]
email["78397faf<br>key people by email"]
merged["f26ae87f<br>merge 78397faf into f42a0133"]
base --> ssn --> merged
base --> email --> mergedWhat you need¶
- GraFlo installed (
pip install graflo). No database is needed.
The data¶
There are no data files: a commit records a change of the manifest. The base,
manifest.yaml, declares one vertex type:
- name: person
properties:
- name: id
type: STRING
- name: ssn
type: STRING
- name: email
type: STRING
identity: [id]
Steps¶
1. Record the two branches¶
commit kind ops label
97382d5d edit 1 track when a person was first seen
78397faf edit 2 key people by email (head)
f42a0133 edit 2 key people by SSN (head)
2 heads: the history has forked. Run merge_branches.py to merge them.
stored: artifacts/commits
build_history.py records three commits. A commit is a
list of evolution operations (changes to the manifest) plus its parents:
by_ssn = build_commit(
after_shared,
[
_rekey("ssn"),
AddVertexPropertiesOp(additions={"person": ["ssn_verified"]}),
],
parents=[shared.id],
label="key people by SSN",
created_at=STAMP,
)
_rekey("ssn") makes ssn the identity of person and keeps id as a plain
property. The email branch is the same with email. Both have the same parent,
97382d5d, which adds created_at. Each commit id is computed from the
commit's operations and parents, so you get the ids shown here. The history
now has two heads: both changes are kept, and neither overwrites the other.
2. Merge the branches and decide the conflict¶
left : f42a0133 key people by SSN
right : 78397faf key people by email
base : 97382d5d
1 conflict(s):
slot : vertex/person/identity
reason : both sides changed this slot differently
left : ['replace_identity']
right : ['replace_identity']
base : identity ['id']
merged (took left):
identity : ['ssn']
properties: ['id', 'ssn', 'email', 'created_at', 'ssn_verified', 'email_verified']
hash : fb66c0ec806b
merge commit: f26ae87f
parents : f42a0133, 78397faf
recipe : d7e4607459e1
heads after merging: 1
stored: artifacts/commits
merge_branches.py finds the base, the commit both
branches start from, and compares each branch with it place by place. GraFlo
calls such a place a slot. Both branches change the slot
vertex/person/identity, in different ways, so that is a conflict, shown with
the key it had in the base: ['id']. The two new properties are in different
slots and are combined without a question.
The script decides the conflict by taking the left side, the SSN branch. The
merged manifest keys people by ssn and has both ssn_verified and
email_verified. The merge commit f26ae87f has both branches as parents and
stores the decision. Each branch had to add a property the other does not
touch: if the key were the only change, taking one side would reproduce that
side exactly, and GraFlo refuses to record a merge commit that changes nothing.
3. Change a branch and merge again¶
left branch advanced: person gains 'nickname'
[...]
replayed the stored decision:
note: 1 recorded resolution(s) replayed: vertex/person/identity
merged (stored decision):
identity : ['ssn']
properties: ['id', 'ssn', 'email', 'created_at', 'ssn_verified', 'nickname', 'email_verified']
hash : 1cb2ab519fc0
The SSN branch moves on: person gains nickname. The same conflict comes
back, and the decision stored in the merge commit is applied again instead of
asking you. Only a conflict in a new slot would need a new decision. A stored
decision whose slot no longer conflicts is reported as not needed and is not
applied, because applying it could undo a change nobody disputes. This run
stores nothing.
What you should see¶
artifacts/commits/ holds one YAML file per commit, and
graflo log shows the history with the parents of each commit:
commit kind rev? ops label
97382d5d edit True 1 track when a person was first seen
78397faf edit True 2 key people by email
└─ parents: 97382d5d
f42a0133 edit True 2 key people by SSN
└─ parents: 97382d5d
f26ae87f merge3 True 1 merge 78397faf into f42a0133 (head)
└─ parents: f42a0133, 78397faf
The merge commit stores one operation, the addition of email_verified: the
change from its first parent, the SSN branch, to the merged manifest. That is what lets
graflo verify --base manifest.yaml --store artifacts/commits replay the whole
history from the base (history replays cleanly (4 commit(s), 1 head(s))), and
graflo checkout --base manifest.yaml --store artifacts/commits f26ae87f
rebuild the merged manifest (manifest hash: fb66c0ec806b).
uv run python merge_branches.py --take right decides for the email branch
instead and stores that merge commit in place of f26ae87f: the identity
becomes ['email'], and the hash (e294e3f07eb7) and the merge commit
(6daeb51f) change with it.
Also possible¶
- The same merge from the shell, on a store that holds the two branches:
graflo merge3 f42a0133 78397faf --base manifest.yaml --store artifacts/commitslists the conflict and stops; add--take leftto record the merge commit. uv run python merge_branches.py --plot-dir figsdraws the slot figure above andfigs/merge-history.svg, the history with the base marked.graflo merge3draws the same with--plotand--plot-history.
What to read next¶
- Track state and measurements: turn a plain schema into one that records how facts change over time.
- Merging two branches: slots, conflicts and merge commits in detail.
- Tracked merges: how a stored decision is replayed.
Files¶
The example lives in examples/22-version-control.
manifest.yaml
# The base manifest: a person keyed by an internal id. Two people will branch
# from it and disagree about what identifies a person.
schema:
metadata:
name: people
version: 1.0.0
graph:
vertex_config:
vertices:
- name: person
properties:
- name: id
type: STRING
- name: ssn
type: STRING
- name: email
type: STRING
identity: [id]
edge_config:
edges: []
ingestion_model:
resources:
- name: people
pipeline:
- vertex: person
transforms: []
build_history.py
"""Two people changed the same manifest. How do I merge their changes and keep the history?
Records the history of ``manifest.yaml`` as commits: one change both people
start from, then one branch that keys people by SSN and one that keys them by
email. Writes one YAML file per commit to ``artifacts/commits/`` and prints
the log. ``merge_branches.py`` merges the two branches. Run it from this
directory:
uv run python build_history.py # write artifacts/commits/
uv run python build_history.py --show # print the stored log only
"""
from __future__ import annotations
import os
from pathlib import Path
import click
from suthing import FileHandle
from graflo import GraphManifest
from graflo.architecture.evolution import (
FileCommitStore,
History,
apply_evolution,
build_commit,
)
from graflo.architecture.evolution.ops import (
AddVertexPropertiesOp,
IdentityReplacement,
NaturalIdentityTarget,
ReplaceIdentityOp,
)
EXAMPLE_DIR = Path(__file__).resolve().parent
MANIFEST_PATH = EXAMPLE_DIR / "manifest.yaml"
STORE_ROOT = EXAMPLE_DIR / "artifacts" / "commits"
#: A commit id is computed from its operations and its parents. The timestamp
#: is fixed so that the stored commit files are the same on every run.
STAMP = "2026-01-01T00:00:00+00:00"
def load_base() -> GraphManifest:
"""The manifest every commit in this history descends from."""
manifest = GraphManifest.from_config(FileHandle.load(MANIFEST_PATH))
manifest.finish_init()
return manifest
def _rekey(field: str) -> ReplaceIdentityOp:
"""Make *field* the identity of ``person``, keeping the old key as a property."""
return ReplaceIdentityOp(
replacements={
"person": IdentityReplacement(
to=NaturalIdentityTarget(identity=[field]), retire="keep"
)
}
)
def build_history() -> tuple[GraphManifest, History]:
"""The base manifest and the forked history over it.
Writes nothing: ``merge_branches.py`` imports this function, so both
scripts work on the same commits.
"""
base = load_base()
# The change both branches start from: their common ancestor.
shared = build_commit(
base,
[AddVertexPropertiesOp(additions={"person": ["created_at"]})],
label="track when a person was first seen",
created_at=STAMP,
)
after_shared = apply_evolution(
base, list(shared.ops), bump_version=False, finish_init=False
)
# Each branch changes the key and also adds a property the other branch
# does not touch. If the key were the only change, taking one side would
# reproduce that side exactly, and a merge commit would have nothing to
# record.
by_ssn = build_commit(
after_shared,
[
_rekey("ssn"),
AddVertexPropertiesOp(additions={"person": ["ssn_verified"]}),
],
parents=[shared.id],
label="key people by SSN",
created_at=STAMP,
)
by_email = build_commit(
after_shared,
[
_rekey("email"),
AddVertexPropertiesOp(additions={"person": ["email_verified"]}),
],
parents=[shared.id],
label="key people by email",
created_at=STAMP,
)
return base, History(commits=[shared, by_ssn, by_email])
def print_log(history: History) -> None:
"""Print the history, oldest first, marking the heads."""
heads = {head.id for head in history.heads()}
click.echo(f"{'commit':<10} {'kind':<8} {'ops':<4} label")
for commit in history.topological():
marker = " (head)" if commit.id in heads else ""
click.echo(
f"{commit.short():<10} {commit.kind:<8} {len(commit.ops):<4} "
f"{commit.label}{marker}"
)
if len(heads) > 1:
click.echo(
f"\n{len(heads)} heads: the history has forked. "
"Run merge_branches.py to merge them."
)
@click.command()
@click.option(
"--store",
type=click.Path(path_type=Path),
default=STORE_ROOT,
show_default=True,
help="Where the commits are written.",
)
@click.option("--show", is_flag=True, help="Print the stored log without rewriting it.")
def main(store: Path, show: bool) -> None:
"""Record the history, or print the one already recorded."""
if show:
print_log(FileCommitStore(store).load())
return
_base, history = build_history()
FileCommitStore(store).save(history)
print_log(history)
click.echo(f"\nstored: {os.path.relpath(store)}")
if __name__ == "__main__":
main()
merge_branches.py
"""Merge the two branches recorded by ``build_history.py`` against their base.
Finds the commit both branches start from, runs a three-way merge, prints the
conflict, resolves it by taking one side, and records a merge commit with both
parents in ``artifacts/commits/``. Run it from this directory:
uv run python merge_branches.py # take the SSN branch
uv run python merge_branches.py --take right # take the email branch
uv run python merge_branches.py --advance-left # change a branch, merge again
uv run python merge_branches.py --plot-dir figs # draw the slots and the history
``--advance-left`` adds a property to the SSN branch and merges again with the
decision stored in the merge commit, so the conflict is not asked a second
time. It stores nothing.
"""
from __future__ import annotations
import os
from pathlib import Path
import click
from build_history import STAMP, build_history
from graflo import GraphManifest
from graflo.architecture.evolution import (
FileCommitStore,
History,
MergeRecipe,
MergeRecipeRef,
apply_evolution,
build_multi_parent_commit,
build_recipe,
checkout,
describe_slot,
find_merge_base,
manifest_hash,
merge_three_way,
re_merge,
take_left,
take_right,
)
from graflo.architecture.evolution.ops import AddVertexPropertiesOp
from graflo.architecture.schema.vertex import Vertex
EXAMPLE_DIR = Path(__file__).resolve().parent
STORE_ROOT = EXAMPLE_DIR / "artifacts" / "commits"
def _person(manifest: GraphManifest) -> Vertex:
"""The ``person`` vertex of *manifest*."""
schema = manifest.graph_schema
assert schema is not None
return schema.core_schema.vertex_config.vertices[0]
def _print_merged(merged: GraphManifest, how: str) -> None:
"""Print the key, the properties and the hash of the merged manifest."""
person = _person(merged)
click.echo(f"\nmerged ({how}):")
click.echo(f" identity : {person.identity}")
click.echo(f" properties: {person.property_names}")
click.echo(f" hash : {manifest_hash(merged)[:12]}")
def _stored_recipe(store: Path) -> MergeRecipe:
"""The decision recorded in the merge commit of the stored history."""
for commit in FileCommitStore(store).load().commits:
if commit.merge_recipe is not None:
return MergeRecipe.model_validate(commit.merge_recipe.payload)
raise click.ClickException(
"no merge commit is stored yet; run merge_branches.py without flags first"
)
@click.command()
@click.option(
"--plot-dir",
type=click.Path(path_type=Path),
default=None,
help="Draw the slot tree and the commit history into this directory.",
)
@click.option(
"--take",
type=click.Choice(["left", "right"]),
default="left",
show_default=True,
help="Which side wins the conflict.",
)
@click.option(
"--advance-left",
is_flag=True,
help="Change the left branch, then merge again with the stored decision.",
)
@click.option(
"--store",
type=click.Path(path_type=Path),
default=STORE_ROOT,
show_default=True,
)
def main(plot_dir: Path | None, take: str, advance_left: bool, store: Path) -> None:
"""Merge the two heads, resolving the conflict on the identity."""
base, history = build_history()
heads = history.heads()
if len(heads) != 2:
raise click.ClickException(
f"expected a forked history with two heads, found {len(heads)}"
)
# Sorted by label, so that the SSN branch is always the left side.
left_commit, right_commit = sorted(heads, key=lambda commit: commit.label or "")
click.echo(f"left : {left_commit.short()} {left_commit.label}")
click.echo(f"right : {right_commit.short()} {right_commit.label}")
base_id = find_merge_base(history, left_commit.id, right_commit.id)
if base_id is None:
raise click.ClickException(
"the two heads share no ancestor; combine unrelated manifests with "
"a union (`graflo merge`) instead"
)
click.echo(f"base : {base_id[:8]}\n")
ancestor = checkout(base, history, base_id)
left_state = checkout(base, history, left_commit.id)
right_state = checkout(base, history, right_commit.id)
if advance_left:
left_state = apply_evolution(
left_state,
[AddVertexPropertiesOp(additions={"person": ["nickname"]})],
bump_version=False,
finish_init=False,
)
click.echo("left branch advanced: person gains 'nickname'\n")
# The merge without a decision: it stops at the conflict.
merged, result = merge_three_way(ancestor, left_state, right_state)
if plot_dir is not None:
# Drawn from the unresolved result, where the conflict is still open.
# Drawing writes nothing to the store.
_plot(
result,
history,
plot_dir,
heads=[left_commit.id, right_commit.id],
merge_base=base_id,
)
return
if merged is not None:
raise click.ClickException("expected a conflict; the example is stale")
click.echo(f"{len(result.conflicts)} conflict(s):")
for conflict in result.conflicts:
click.echo(f" slot : {describe_slot(conflict.slot_key)}")
click.echo(f" reason : {conflict.reason}")
click.echo(f" left : {[op.op for op in conflict.left_ops]}")
click.echo(f" right : {[op.op for op in conflict.right_ops]}")
click.echo(f" base : identity {conflict.base_excerpt.get('identity')}")
if advance_left:
merged, result = re_merge(
_stored_recipe(store), ancestor, left_state, right_state
)
click.echo("\nreplayed the stored decision:")
for warning in result.warnings:
click.echo(f" note: {warning}")
if merged is None:
raise click.ClickException("the merge did not resolve")
_print_merged(merged, "stored decision")
click.echo("\n(not stored: --advance-left merges a changed branch)")
return
# The decision, kept in a recipe so that a later merge can replay it.
chooser = take_left if take == "left" else take_right
resolutions = [chooser(conflict) for conflict in result.conflicts]
recipe = build_recipe(ancestor, left_state, right_state, resolutions=resolutions)
merged, result = merge_three_way(
ancestor, left_state, right_state, resolutions=resolutions
)
if merged is None:
raise click.ClickException("the merge did not resolve")
_print_merged(merged, f"took {take}")
# The merge commit: both heads as parents, the recipe attached.
commit = build_multi_parent_commit(
left_state,
merged,
parents=[left_commit.id, right_commit.id],
label=f"merge {right_commit.short()} into {left_commit.short()}",
created_at=STAMP,
merge_recipe=MergeRecipeRef(
hash=recipe.content_hash(), kind=recipe.kind, payload=recipe.to_dict()
),
)
click.echo(f"\nmerge commit: {commit.short()}")
click.echo(f" parents : {', '.join(p[:8] for p in commit.parents)}")
click.echo(f" recipe : {recipe.content_hash()[:12]}")
merged_history = History(commits=[*history.commits, commit])
FileCommitStore(store).save(merged_history)
click.echo(f"\nheads after merging: {len(merged_history.heads())}")
click.echo(f"stored: {os.path.relpath(store)}")
def _plot(
result,
history: History,
plot_dir: Path,
*,
heads: list[str],
merge_base: str,
) -> None:
"""Draw where the branches met, and the history they met on."""
from graflo.architecture.evolution.preview import build_merge3_preview
from graflo.plot.merge3 import plot_history, plot_merge3_preview
preview = build_merge3_preview(result)
slots = plot_merge3_preview(preview, plot_dir / "merge-slots.svg")
lineage = plot_history(
history, plot_dir / "merge-history.svg", heads=heads, merge_base=merge_base
)
click.echo(f"slot tree ({preview.conflicts} contested) -> {os.path.relpath(slots)}")
click.echo(f"history -> {os.path.relpath(lineage)}\n")
if __name__ == "__main__":
main()