ontocast.tool.chunk.structure¶
Document structure: the blocks of the markdown export and their kinds.
The markdown export is the document text everything else reads, so blocks are cut from it by markdown syntax: blank lines separate paragraphs, list bullets and citation markers start blocks, and heading lines and table rows are blocks of their own. A converted document adds what syntax cannot show: each docling item is serialized as the export serializes it, and a block whose text equals an item's text takes that item's kind and reference. So a caption, a footnote or a formula is known for what it is, and a unit can name the items it holds.
A block no item matches -- plain-text input, a paragraph a unit boundary cut through, text the converter joined differently -- keeps its syntax kind.
Attributes¶
BlockKinds = Mapping[str, ItemBlock]
module-attribute
¶
ENTRY_MARKER = re.compile('^\\s*(?:\\[\\d{1,4}\\]|\\(\\d{1,4}\\)|\\d{1,4}[.)])\\s+\\S')
module-attribute
¶
LIST_BULLET = re.compile('^\\s*[-*•·]\\s+')
module-attribute
¶
PLACEHOLDER = re.compile('<!--.*?-->', re.DOTALL)
module-attribute
¶
__all__ = ['ENTRY_MARKER', 'LIST_BULLET', 'PLACEHOLDER', 'Block', 'BlockKind', 'BlockKinds', 'DocumentStructure', 'ItemBlock', 'build_document_structure', 'docling_block_kinds', 'item_refs', 'split_blocks']
module-attribute
¶
Classes¶
Block
dataclass
¶
One block of a text, with offsets into that text.
Source code in ontocast/tool/chunk/structure.py
BlockKind
¶
Bases: StrEnum
What a block of the export is.
Source code in ontocast/tool/chunk/structure.py
Attributes¶
CAPTION = 'caption'
class-attribute
instance-attribute
¶
CODE = 'code'
class-attribute
instance-attribute
¶
FOOTNOTE = 'footnote'
class-attribute
instance-attribute
¶
FORMULA = 'formula'
class-attribute
instance-attribute
¶
HEADING = 'heading'
class-attribute
instance-attribute
¶
PICTURE = 'picture'
class-attribute
instance-attribute
¶
TABLE = 'table'
class-attribute
instance-attribute
¶
TEXT = 'text'
class-attribute
instance-attribute
¶
DocumentStructure
dataclass
¶
The document text and the docling kinds of its blocks.
Source code in ontocast/tool/chunk/structure.py
ItemBlock
¶
Bases: NamedTuple
The kind and docling reference of an item, keyed by its block text.
Source code in ontocast/tool/chunk/structure.py
Functions:¶
build_document_structure(doc)
¶
The markdown export of doc and the docling kinds of its blocks.
Source code in ontocast/tool/chunk/structure.py
docling_block_kinds(doc)
¶
Kind and reference of each body item, keyed by its normalised block text.
Items are serialized one by one with the serializer the markdown export uses, so their paragraphs are the export's blocks. An inline group is one item (its parts serialize differently on their own); a picture or table contributes its captions. The first item wins when two share a text.
Source code in ontocast/tool/chunk/structure.py
item_refs(text, kinds)
¶
References of the docling items whose blocks text holds whole, in order.
Source code in ontocast/tool/chunk/structure.py
split_blocks(text, kinds=None)
¶
Split a text into blocks, typed by docling items where they match.
Blank lines separate paragraphs. Inside a paragraph, a line that opens with a list bullet or a citation marker starts a new block, and lines without one continue the block before them, so a reference wrapped over two lines stays one entry. A paragraph that opens mid-sentence (lower case, a digit, a bracket, a DOI or URL) right after a block that opened with a marker continues that block too: converters split a wrapped entry that way. Heading lines and table rows are blocks of their own; a paragraph of placeholders only is a picture.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Markdown or plain text. |
required |
kinds
|
BlockKinds | None
|
Docling kinds by block text (:func: |
None
|
Returns:
| Type | Description |
|---|---|
list[Block]
|
Blocks in order, with offsets into |