Skip to content

graflo.hq.bulk

Bulk load: cast and stage a whole input, fold it by identity, write it once.

A bulk run loads an empty target. Every vertex and edge is folded to one row under the upsert merge rules before anything is written, so each backend needs only a create-only write. The fold runs in DuckDB, from the optional bulk extra; importing this package does not load it.

Modules:

Name Description
encode

Rows on disk: typed identity keys, lossless values, compressed JSON Lines.

fold

Fold staged runs by identity in DuckDB: one row per vertex and per edge.

ledger

The run ledger: which phases and chunks of a bulk or sync run are done, durably.

partition

Rounds of edge cells that share no endpoint vertex (mix and batch).

refusals

What a bulk run cannot load: steps that read the target while ingesting.

run

Run a bulk load: open the staging directory, fold, write, verify.

stage

Stage cast batches: one run per vertex type and per concrete edge, on disk.