ontocast.tool.chunk.outline¶
Document outline detection for chunk preparation.
The outline is the backbone of section classification: every heading in the document closes the preceding section, whether or not the heading maps to a known label. The previous span builder ended each span at the next recognised heading, so a single unrecognised heading let one label smear across the rest of the document -- and because the label was stamped onto segments at split time, no later tier could correct it.
Two heading kinds must be told apart, because they need opposite treatment:
- generic section names (
Results,Experimental Section,References) start a new section; when unrecognised they open an explicitly unresolved span rather than inheriting the previous label; - descriptive subsection titles and document titles (
Cooperative ensemble breaks population-inversion limitation) sit inside a section and must inherit its label, or their body text is lost from the parent section.
A section-like heading that no pattern or keyword names may still be named by
its nearest label in embedding space (:mod:ontocast.tool.chunk.heading_embedding),
weakly, when the chunker's model is available.
Where the document states its hierarchy, the tree decides: a heading's section number ("2.1" sits under "2"), or its markdown level in documents whose levels vary. Docling reports one level for every PDF heading, so many documents state nothing; there the discriminator is heading genericity, the number of content words after stopword removal.
Subject-domain-agnostic but, like :mod:ontocast.tool.chunk.bibliography, not
language-agnostic: the stopword list and the sentence-punctuation cues are
English/Latin-script. The failure mode is a miss (a heading treated as
descriptive), not a misfire.
Attributes¶
SECTIONLIKE_MAX_CONTENT_WORDS = 3
module-attribute
¶
__all__ = ['SECTIONLIKE_MAX_CONTENT_WORDS', 'build_document_outline', 'content_words', 'detect_headings', 'format_outline', 'heading_is_sectionlike', 'heading_number', 'inherit_subsection_labels', 'label_outline', 'link_heading_tree', 'markdown_headings', 'outline_to_spans', 'text_headings']
module-attribute
¶
logger = logging.getLogger(__name__)
module-attribute
¶
Classes¶
Functions:¶
build_document_outline(text, schema, *, include_text_headings=False, embed=None)
¶
Detect headings and resolve the labels obtainable from heading text alone.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Document text (the markdown export). |
required |
schema
|
SectionLabelSchema
|
Section label schema. |
required |
include_text_headings
|
bool
|
Detect plain-text headings in a document with no markdown headings. |
False
|
embed
|
TextEmbedder | None
|
The chunker's embedder; when given, section-like headings the lexical tiers leave unnamed take their nearest label, weakly. |
None
|
Source code in ontocast/tool/chunk/outline.py
content_words(normalised)
¶
Topical words of a normalised heading, with closed-class words removed.
Source code in ontocast/tool/chunk/outline.py
detect_headings(text, *, include_text_headings=False)
¶
Detect document headings, falling back to the text heuristic when empty.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Document text (the markdown export used for section tagging). |
required |
include_text_headings
|
bool
|
Enable the plain-text heading heuristic for documents with no markdown heading structure at all. |
False
|
Returns:
| Type | Description |
|---|---|
list[HeadingNode]
|
Headings ordered by character offset. |
Source code in ontocast/tool/chunk/outline.py
format_outline(outline)
¶
Render the outline as human-readable lines for logging and the CLI.
Source code in ontocast/tool/chunk/outline.py
heading_is_sectionlike(normalised)
¶
Whether a heading reads as a generic section name.
Section-like headings start a new section and, when unrecognised, leave the span explicitly unresolved. Non-section-like headings (descriptive subsection titles, document titles) inherit the enclosing section's label.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
normalised
|
str
|
Heading text after :func: |
required |
Returns:
| Type | Description |
|---|---|
bool
|
True when the heading is a short, generic section name. |
Source code in ontocast/tool/chunk/outline.py
heading_number(text)
¶
The section number a raw heading opens with ("2.1", "II", "A.2"), if any.
Source code in ontocast/tool/chunk/outline.py
inherit_subsection_labels(outline, schema)
¶
Propagate the enclosing section's label onto headings inside it.
A heading the tree places under a parent takes the parent's label, and its strength, unless it names its own: the number or level states that it is part of the parent. A heading the tree places at top level opens a section of its own and inherits nothing. An unplaced heading that is not section-like (a descriptive subsection title) sits inside the section opened by the last section-like or placed heading, and takes its label weakly: only its wording says it belongs there. A terminal section (a reference list) has no subsections.
Source code in ontocast/tool/chunk/outline.py
label_outline(outline, schema)
¶
Assign labels to outline headings from the schema (mutates in place).
Runs the anchored-pattern tier, then the keyword recall tier. Both are
gated on sectionlike: applying keyword matching to a descriptive
subsection title mislabels it from an incidental word -- a title containing
"limitation" is not a limitations section.
A subtitled heading ("Appendix A: Derivation of the rate") is judged by its head: when the head is section-like and names a label, the heading opens that section instead of reading as a descriptive title.
Source code in ontocast/tool/chunk/outline.py
link_heading_tree(outline)
¶
Place headings in a tree where the document states one (mutates in place).
A section number places a heading: "2.1" hangs under the nearest "2" before it, and a one-part number is top level. A numbered heading whose parent number is absent stays unplaced. In a document whose markdown levels vary, the remaining headings hang under the nearest shallower heading before them. Unplaced headings are judged by genericity.
Source code in ontocast/tool/chunk/outline.py
markdown_headings(text)
¶
Detect #-prefixed headings in the markdown export.
This is the complete structural signal available: docling renders every
SECTION_HEADER item as a markdown heading line, so scanning the export
finds exactly the items a docling walk would, with exact character offsets.
Source code in ontocast/tool/chunk/outline.py
outline_to_spans(outline)
¶
Convert an outline into a partition of the document into section spans.
Every heading closes the preceding span, recognised or not. The returned
spans tile [0, text_len) with no gaps and no overlaps; a span whose
section type is unknown carries label=None rather than being merged into
its neighbour.
Source code in ontocast/tool/chunk/outline.py
text_headings(text)
¶
Detect headings in documents that carry no markdown heading structure.
Only blank-line-delimited short lines that are upper-case or explicitly numbered qualify, which keeps the detector from firing inside prose. Used as a fallback when the structural scan finds nothing.