ontocast.tool.chunk.sections¶
Document section spans for chunk prepare: outline spans plus front matter.
Attributes¶
ABSTRACT_FRONT_MATTER_MAX_CHARS = 6000
module-attribute
¶
__all__ = ['ABSTRACT_FRONT_MATTER_MAX_CHARS', 'DocumentOutline', 'HeadingNode', 'SectionSpan', 'detect_section_spans', 'document_text_for_section_tagging', 'inject_front_matter_spans']
module-attribute
¶
Classes¶
DocumentOutline
¶
Bases: BasePydanticModel
Ordered headings detected in a document, with the document length.
Source code in ontocast/onto/section_models.py
HeadingNode
¶
Bases: BasePydanticModel
One detected heading in the document outline.
Attributes:
| Name | Type | Description |
|---|---|---|
text |
str
|
Raw heading line as it appears in the document text. |
normalised |
str
|
Heading text after decoration/numbering stripping. |
start |
int
|
Character offset of the heading line itself. |
body_start |
int
|
Character offset just past the heading line. |
level |
int
|
Markdown heading depth (1 = top). Docling reports a flat level for PDF conversions, so levels place a heading only in documents whose levels vary. |
number |
str | None
|
The heading's section number ("2.1", "II", "A.2"), if any. |
parent |
int | None
|
Index in the outline of the heading this one sits under. |
tree_placed |
bool
|
Whether numbering or markdown levels place this heading;
|
sectionlike |
bool
|
Whether the heading reads as a generic section name rather than a descriptive subsection title or a document title. |
label |
str | None
|
Canonical section label, when resolved. |
source |
SectionLabelSource
|
How |
strength |
LabelStrength
|
Whether |
Source code in ontocast/onto/section_models.py
Attributes¶
body_start
instance-attribute
¶
label = None
class-attribute
instance-attribute
¶
level = 1
class-attribute
instance-attribute
¶
normalised
instance-attribute
¶
number = None
class-attribute
instance-attribute
¶
parent = None
class-attribute
instance-attribute
¶
sectionlike = True
class-attribute
instance-attribute
¶
source = SectionLabelSource.OUTLINE_UNRESOLVED
class-attribute
instance-attribute
¶
start
instance-attribute
¶
strength = LabelStrength.WEAK
class-attribute
instance-attribute
¶
text
instance-attribute
¶
tree_placed = False
class-attribute
instance-attribute
¶
SectionSpan
¶
Bases: BasePydanticModel
Character span of a document section with a normalised label.
label is None for a region whose section type is not (yet) known —
for example an unrecognised but section-like heading. Such a span is
explicitly unresolved rather than absent, which is what stops a neighbouring
label from being smeared across it.
Source code in ontocast/onto/section_models.py
Attributes¶
end
instance-attribute
¶
heading = Field(default=None, description='The heading line that opens the span; none for unheaded text.')
class-attribute
instance-attribute
¶
label = None
class-attribute
instance-attribute
¶
source = SectionLabelSource.OUTLINE_UNRESOLVED
class-attribute
instance-attribute
¶
start
instance-attribute
¶
strength = LabelStrength.WEAK
class-attribute
instance-attribute
¶
Functions:¶
detect_section_spans(text, schema=None, *, include_text_headings=False, embed=None)
¶
Detect document sections and return a partition of the text into spans.
Every heading closes the preceding span, so an unrecognised heading yields an
explicitly unresolved (label=None) span instead of letting the previous
label run on to the next recognised heading.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Document text (the markdown export). |
required |
schema
|
SectionLabelSchema | None
|
Section label schema; the manifest default when omitted. |
None
|
include_text_headings
|
bool
|
Enable the plain-text heading heuristic for documents with no markdown heading structure. |
False
|
embed
|
TextEmbedder | None
|
The chunker's embedder, for the embedding heading tier. |
None
|
Returns:
| Type | Description |
|---|---|
list[SectionSpan]
|
Section spans tiling |
Source code in ontocast/tool/chunk/sections.py
document_text_for_section_tagging(doc)
¶
inject_front_matter_spans(spans, text, schema, *, min_gap_chars=80, max_gap_chars=ABSTRACT_FRONT_MATTER_MAX_CHARS)
¶
Label unheaded front matter before the first labeled section as abstract.
Leading unresolved spans (a title block, an unrecognised banner heading) are
skipped when locating the first labeled section, so front matter is still
recovered on documents whose first recognised section is not an IMRaD
opener -- papers that jump straight to Results are common.