graflo.data_source.rdf¶
RDF data source hierarchy.
Provides two concrete data sources that share a common abstract parent:
- :class:
RdfFileDataSource– reads local RDF files (Turtle, RDF/XML, N3, JSON-LD, …) via rdflib. - :class:
SparqlEndpointDataSource– queries a remote SPARQL endpoint (e.g. Apache Fuseki) via SPARQLWrapper.
Both convert RDF triples into flat dictionaries grouped by subject URI, one
dict per rdf:Class instance. Each document has _uri (full subject URI)
and _key (URI local name — the fragment or last path segment). All other
identity decisions (e.g. using a domain-specific literal as the storage key)
belong in the schema layer, not here.
Two kinds of subject have no IRI of their own to key on, and both sources treat them alike:
- A blank node is keyed on its content:
_uriis_:<key>and_keythe key, the same on every read. A reference to it carries that_uri. - IRIs joined by
owl:sameAsare one document under the smallest IRI; the others are listed in_same_asand references to them are rewritten.same_as="keep"reads the statements as an ordinary property instead.
Uses rdflib and SPARQLWrapper, which are core dependencies of
graflo (see pyproject.toml).
Attributes¶
RDF_TYPE = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type'
module-attribute
¶
SparqlDataSource = SparqlEndpointDataSource
module-attribute
¶
logger = logging.getLogger(__name__)
module-attribute
¶
Classes¶
RdfDataSource
¶
Bases: AbstractDataSource, ABC
Abstract base for RDF data sources (file and endpoint).
Captures the fields and batch-yielding logic shared by both
:class:RdfFileDataSource and :class:SparqlEndpointDataSource.
Attributes:
| Name | Type | Description |
|---|---|---|
rdf_class |
str | None
|
Optional URI of the |
same_as |
Literal['collapse', 'keep']
|
|
Source code in graflo/data_source/rdf.py
Attributes¶
rdf_class = Field(default=None, description='URI of the rdf:Class to filter by')
class-attribute
instance-attribute
¶
same_as = Field(default='collapse', description='How owl:sameAs statements between IRIs are read.')
class-attribute
instance-attribute
¶
source_type = DataSourceType.SPARQL
class-attribute
instance-attribute
¶
typed_objects = Field(default_factory=list, description='Properties (local names) whose objects are also read by class, as ``<property>@<Class>``: for a property with several ranges.')
class-attribute
instance-attribute
¶
RdfFileDataSource
¶
Bases: RdfDataSource
Data source for local RDF files.
Parses RDF files using rdflib and yields flat dictionaries grouped by
subject URI. Optionally filters by rdf_class so that only instances
of a specific class are returned.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
Path
|
Path to the RDF file. |
rdf_format |
str | None
|
Explicit rdflib format string (e.g. |
Source code in graflo/data_source/rdf.py
Attributes¶
path
instance-attribute
¶
rdf_format = Field(default=None, description='rdflib serialization format')
class-attribute
instance-attribute
¶
Methods:¶
iter_batches(batch_size=1000, limit=None)
¶
Parse the RDF file and yield batches of flat dictionaries.
Source code in graflo/data_source/rdf.py
SparqlEndpointDataSource
¶
Bases: RdfDataSource
Data source that reads from a SPARQL endpoint.
Uses SPARQLWrapper to query an endpoint and returns flat dictionaries
grouped by subject.
Attributes:
| Name | Type | Description |
|---|---|---|
config |
SparqlSourceConfig
|
SPARQL source configuration. |
Source code in graflo/data_source/rdf.py
536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 | |
Attributes¶
config
instance-attribute
¶
Methods:¶
iter_batches(batch_size=1000, limit=None)
¶
Query the SPARQL endpoint and yield batches of flat dictionaries.
Paginates with SPARQL LIMIT/OFFSET on bindings (triple rows), merges rows into subject documents in a streaming fashion, and stops fetching once limit subjects have been yielded (when set).
Two more reads happen when the graph calls for them. The owl:sameAs
statements are read first (unless same_as is keep); the members
of a component are held back and yielded last, as one document. The
graph's blank-node triples are read when the first blank node is met,
which keys blank nodes on their content; this relies on the endpoint
giving a blank node the same label in every query of the read.
Source code in graflo/data_source/rdf.py
581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 | |
SparqlSourceConfig
¶
Bases: ConfigBaseModel
Configuration for a SPARQL endpoint data source.
Attributes:
| Name | Type | Description |
|---|---|---|
endpoint_url |
str
|
Full SPARQL query endpoint URL
(e.g. |
rdf_class |
str | None
|
URI of the rdf:Class whose instances to fetch |
graph_uri |
str | None
|
Named graph to restrict the query to (optional) |
sparql_query |
str | None
|
Custom SPARQL query override (optional) |
username |
str | None
|
HTTP basic-auth username (optional) |
password |
str | None
|
HTTP basic-auth password (optional) |
page_size |
int
|
Number of results per SPARQL LIMIT/OFFSET page |
Source code in graflo/data_source/rdf.py
Attributes¶
endpoint_url
instance-attribute
¶
graph_uri = None
class-attribute
instance-attribute
¶
page_size = Field(default=10000, description='SPARQL pagination page size')
class-attribute
instance-attribute
¶
password = None
class-attribute
instance-attribute
¶
rdf_class = None
class-attribute
instance-attribute
¶
sparql_query = None
class-attribute
instance-attribute
¶
username = None
class-attribute
instance-attribute
¶
Methods:¶
blank_node_query(offset=0)
¶
Every triple whose subject is a blank node.
build_query(offset=0, limit=None)
¶
Build a SPARQL SELECT query.
If sparql_query is set it is returned with LIMIT/OFFSET appended. Otherwise generates::
SELECT ?s ?p ?o WHERE { ?s a <rdf_class> . ?s ?p ?o . }
Source code in graflo/data_source/rdf.py
same_as_query(offset=0)
¶
The owl:sameAs statements between IRIs, as ?s and ?o.
Source code in graflo/data_source/rdf.py
subjects_query(iris, offset=0)
¶
Every triple of the subjects iris.
Source code in graflo/data_source/rdf.py
types_query(iris, offset=0)
¶
The rdf:type of each of iris, as ?s and ?o.
Source code in graflo/data_source/rdf.py
Functions:¶
split_objects_by_type(docs, properties, types_of)
¶
Add <property>@<Class> beside each listed property, in place.
The new field holds the property's objects typed <Class> (the class
IRI's local name); an object of several classes is under each, one of none
under no field. types_of maps IRIs to their rdf:type IRIs.