Skip to content

Backend indexes

Every upsert looks up the stored vertex with the same identity, and without an index on the identity fields each lookup scans the whole vertex type. This page is for anyone who writes to a graph database with GraFlo: it says which indexes GraFlo creates on each backend when it defines the schema, which ones you declare yourself, and how Neo4j, Memgraph and FalkorDB match edges. After reading it you know what to put in the db_profile block and what to leave to GraFlo.

Identity indexes and secondary indexes

GraFlo distinguishes two kinds of vertex index:

  • The identity index covers the fields a vertex type upserts on: its natural key, or id for hash, funnel, blank and assigned vertices (see Vertex identity). GraFlo creates it on every database backend. You do not declare it.
  • Secondary indexes speed up other lookups. You declare them in the database profile of the manifest, schema.db_profile: per vertex type under vertex_indexes, per edge under edge_specs[*].indexes.
schema:
    db_profile:
        vertex_indexes:
            machine:
            -   fields: [plant, model]
        edge_specs:
        -   source: work_order
            target: machine
            relation: targets
            indexes:
            -   fields: [opened_at]

An index entry takes fields and, optionally, unique (default false), type (persistent, the default, hash, skiplist or fulltext) and name. Every backend builds persistent, hash and skiplist as a plain index. Only ArangoDB builds fulltext; on any other database a fulltext index is refused when the schema is applied, with the vertex or edge it is declared on. The file backend builds no index and ignores it. With unique: true, the index is a uniqueness constraint over its fields together, and a write that repeats their values is rejected.

An index over several fields is one index over them, in the order you list them; put the field your lookups filter on most first. Some backends cannot build every index: TigerGraph builds no edge index, no index over several fields and no uniqueness constraint, Memgraph builds edge indexes over one field and without uniqueness, and NebulaGraph builds no uniqueness constraint. An index you declare that the backend cannot build is refused when the schema is applied, naming the vertex or edge and the fields. An index GraFlo derives itself, for an identity or a secondary identity, is skipped there with a warning instead.

sparse and deduplicate are ArangoDB index options; the other backends ignore them.

An edge_specs entry can also set relation_name, the name the database stores the relation under. TigerGraph native inverses are set per relation in db_profile.native_inverses, not on a spec; see Directed, undirected, and bidirectional edges.

Indexes from secondary identities

Every secondary identity on a vertex adds one non-unique index over its fields to db_profile.vertex_indexes when the schema is loaded. You do not declare it. If vertex_indexes already has an index on the same fields, GraFlo keeps yours and adds nothing.

The index is more than an optimization. Endpoint lookups filter on those fields, and on NebulaGraph a filtered lookup cannot run without a tag index.

The index is non-unique because a secondary identity is only softly unique: a unique index would reject exactly the duplicate values that the ambiguity policy exists to handle. An index you declare yourself on the same fields keeps the unique setting you gave it.

TigerGraph accepts indexes on a single field only. A composite secondary identity logs a warning and gets no index there; the lookup does not depend on it, because GraFlo finds TigerGraph endpoints with an interpreted GSQL query.

NebulaGraph refuses a tag index on a string column unless the index gives a length. GraFlo gives every string column in a tag index a length of 256. That includes untyped properties and DATETIME and UUID properties, which NebulaGraph stores as strings. When NebulaGraph rejects an index anyway, GraFlo logs a warning, because every filtered read on that tag then fails with IndexNotFound.

Per backend

For the same limits at a glance, beside the other features of each backend, see Choosing a backend.

Backend Identity index Declared vertex indexes Declared edge indexes
Neo4j A uniqueness constraint over the identity fields, created with the other indexes. If the server refuses a composite constraint, a plain index with a warning CREATE INDEX, or a uniqueness constraint when unique: true; a constraint the server refuses is an error A relationship index, or a uniqueness constraint when unique: true
Memgraph (3.2 or later) One index over the identity fields and a uniqueness constraint over them, created with the other indexes One index over the fields, plus a uniqueness constraint when unique: true CREATE EDGE INDEX ON :<relation>(<field>), one field only, never unique
FalkorDB Each identity field indexed and a uniqueness constraint over them, created with the other indexes Each field indexed (FalkorDB keeps one index per label), plus a uniqueness constraint when unique: true The same, on the relationship type
NebulaGraph A tag index over the identity fields, always created, because LOOKUP and filtered MATCH need it A tag index; never unique An edge index; never unique
ArangoDB A unique persistent index over the identity fields, created with the collection; none when the identity is _key, which ArangoDB indexes itself As declared, with unique and type applied As declared, on each edge collection
TigerGraph The primary key (PRIMARY_ID, or PRIMARY KEY for a composite identity), which TigerGraph indexes itself One field only, never unique None
PostgreSQL The vertex table's PRIMARY KEY CREATE INDEX, or CREATE UNIQUE INDEX when unique: true The same, over the edge's property columns. Every edge table also gets an index on its target columns, and a unique index over its endpoint columns and the edge's identity properties, which covers its identities
GraFlo file backend None None None

On ArangoDB, TigerGraph and PostgreSQL the identity is covered when the collection, vertex type or table is created. On Neo4j, Memgraph, FalkorDB and NebulaGraph, GraFlo creates the identity index together with the declared indexes when it defines the schema.

On PostgreSQL an index name longer than 63 bytes, the identifier limit, is shortened to a head and an 8-hex digest of the whole name, so two long names sharing a prefix stay two indexes.

Defining indexes without a schema

The low-level define_vertex_indexes call takes the schema as an optional argument, and only the schema says which fields form each identity. Called without it, Neo4j, Memgraph and FalkorDB get no identity index and GraFlo logs a warning; NebulaGraph gets one anyway, and PostgreSQL gets no secondary indexes. GraphEngine.define_schema and define_and_ingest always pass the schema. Pass it yourself when you call define_vertex_indexes or define_indexes directly.

Writing never creates an index, on any backend. Upserting into a database whose schema was not defined is still correct, but each lookup scans.

Edge upserts and MERGE (Neo4j, Memgraph, FalkorDB)

An edge is identified by its endpoints and its relation. On the Cypher backends it is written with MERGE: the endpoints are matched on their vertex identities, and the relationship on its type. Without identities there is one edge per pair of endpoints and relation, and the last record's properties overwrite the stored ones. Properties are written, never matched on.

To keep several edges between the same two vertices, declare which properties tell them apart: identities: [[source, target, since]] merges on since as well. GraFlo takes those names from the first entry of identities, leaving out the source and target tokens and turning a relation token into the relationship's relation property. A record that lacks one of them matches an edge that lacks it too.

PostgreSQL keys every edge table on the same properties, less a relation token, since each relation has its own table: a unique index over the endpoint columns and the identity properties, NULLS NOT DISTINCT, so it needs PostgreSQL 15 or later. An edge write that finds its key taken updates the edge's other properties. An edge table whose key is missing or covers other columns, as one created by an earlier version, gets its key at schema definition or on its first ingest. That fails with a clear error if the table holds rows the key would merge, such as parallel edges told apart by a property no identity names.

An edge's identities also feed edge indexes, which are defined with the schema. The MERGE key is chosen separately, when the graph is written. Keep both in line with the uniqueness you intend for the edge.