Skip to content

ontocast.config.section_labels

Load versioned section-label schemas from YAML in this package.

Classes

RouteClass

Bases: StrEnum

What a section of this label holds, for routing before extraction.

content sections are extracted. boilerplate sections are dropped by default when the label is strong and the text states no measurement. non_content sections (front or back matter: author blocks, notes, availability and licence statements) are flagged, or dropped, per CHUNK_NON_CONTENT_MODE under the same conditions. citations sections are reference lists, routed by CHUNK_BIBLIOGRAPHY_MODE when their content agrees. The class never decides a drop on its own.

Source code in ontocast/config/section_labels/__init__.py
class RouteClass(StrEnum):
    """What a section of this label holds, for routing before extraction.

    ``content`` sections are extracted. ``boilerplate`` sections are dropped
    by default when the label is strong and the text states no measurement.
    ``non_content`` sections (front or back matter: author blocks, notes,
    availability and licence statements) are flagged, or dropped, per
    ``CHUNK_NON_CONTENT_MODE`` under the same conditions. ``citations``
    sections are reference lists, routed by ``CHUNK_BIBLIOGRAPHY_MODE`` when
    their content agrees. The class never decides a drop on its own.
    """

    CONTENT = "content"
    BOILERPLATE = "boilerplate"
    NON_CONTENT = "non_content"
    CITATIONS = "citations"

Attributes

BOILERPLATE = 'boilerplate' class-attribute instance-attribute
CITATIONS = 'citations' class-attribute instance-attribute
CONTENT = 'content' class-attribute instance-attribute
NON_CONTENT = 'non_content' class-attribute instance-attribute

SchemaManifestEntry

Bases: BaseModel

Source code in ontocast/config/section_labels/__init__.py
class SchemaManifestEntry(BaseModel):
    id: str
    file: str

Attributes

file instance-attribute
id instance-attribute

SectionLabelDef

Bases: BaseModel

One canonical section label, its heading patterns and recall keywords.

heading_patterns are high-precision anchored regexes. keywords are the recall tier: whole-word phrases that identify the label inside a compound or decorated heading ("Results and Discussion", "Experimental Section") which the anchored patterns cannot match.

Source code in ontocast/config/section_labels/__init__.py
class SectionLabelDef(BaseModel):
    """One canonical section label, its heading patterns and recall keywords.

    ``heading_patterns`` are high-precision anchored regexes. ``keywords`` are
    the recall tier: whole-word phrases that identify the label inside a
    compound or decorated heading ("Results and Discussion", "Experimental
    Section") which the anchored patterns cannot match.
    """

    model_config = ConfigDict(extra="forbid")

    id: str
    route_class: RouteClass = RouteClass.CONTENT
    terminal: bool = Field(
        default=False,
        description=(
            "A flat list with no subsections (a reference list): descriptive "
            "headings after it never inherit the label, and a keyword names it "
            "only as a whole conjunct of a heading ('Notes and References'), "
            "never inside a longer phrase ('Comparison with literature "
            "references')."
        ),
    )
    description: str = Field(
        min_length=1,
        description=(
            "What a section of this type does in the document, by its role "
            "rather than its subject: shown to the model that labels the "
            "outline."
        ),
    )
    heading_patterns: list[str] = Field(default_factory=list)
    keywords: list[str] = Field(default_factory=list)

    @property
    def compiled_keywords(self) -> tuple[re.Pattern[str], ...]:
        return tuple(_compile_keyword(keyword) for keyword in self.keywords)

Attributes

compiled_keywords property
description = Field(min_length=1, description='What a section of this type does in the document, by its role rather than its subject: shown to the model that labels the outline.') class-attribute instance-attribute
heading_patterns = Field(default_factory=list) class-attribute instance-attribute
id instance-attribute
keywords = Field(default_factory=list) class-attribute instance-attribute
model_config = ConfigDict(extra='forbid') class-attribute instance-attribute
route_class = RouteClass.CONTENT class-attribute instance-attribute
terminal = Field(default=False, description="A flat list with no subsections (a reference list): descriptive headings after it never inherit the label, and a keyword names it only as a whole conjunct of a heading ('Notes and References'), never inside a longer phrase ('Comparison with literature references').") class-attribute instance-attribute

SectionLabelManifest

Bases: BaseModel

Source code in ontocast/config/section_labels/__init__.py
class SectionLabelManifest(BaseModel):
    catalog_version: str
    default_schema: str
    schemas: list[SchemaManifestEntry]
    document_type_hints: dict[str, str] = Field(default_factory=dict)

Attributes

catalog_version instance-attribute
default_schema instance-attribute
document_type_hints = Field(default_factory=dict) class-attribute instance-attribute
schemas instance-attribute

SectionLabelSchema

Bases: BaseModel

Domain-specific section label vocabulary for one document type.

The catalog of schemas is a partition: every document belongs to exactly one cell, with general as the residual. document_profile is what makes that partition checkable -- if two profiles could describe the same document, the cells overlap and one of them is wrong. It is also the text the content-based detector embeds, so it describes the document type, unlike description, which describes this schema's headings.

Source code in ontocast/config/section_labels/__init__.py
class SectionLabelSchema(BaseModel):
    """Domain-specific section label vocabulary for one document type.

    The catalog of schemas is a **partition**: every document belongs to exactly
    one cell, with ``general`` as the residual. ``document_profile`` is what
    makes that partition checkable -- if two profiles could describe the same
    document, the cells overlap and one of them is wrong. It is also the text
    the content-based detector embeds, so it describes the *document type*,
    unlike ``description``, which describes this schema's headings.
    """

    model_config = ConfigDict(extra="forbid")

    schema_version: str
    id: str
    description: str = ""
    document_profile: str = Field(
        default="",
        description=(
            "One sentence describing the kind of document this schema covers, "
            "written to be true of no other schema in the catalog. Empty means "
            "the schema is not a detection candidate."
        ),
    )
    parent: str | None = Field(
        default=None,
        description=(
            "Reserved for a future document-type hierarchy (e.g. a thesis as a "
            "sub-type of academic). Unused today: the catalog is flat, and "
            "sub-types would blur the cells a flat detector must separate."
        ),
    )
    labels: list[SectionLabelDef]

    @property
    def compiled_patterns(self) -> tuple[tuple[str, re.Pattern[str]], ...]:
        compiled: list[tuple[str, re.Pattern[str]]] = []
        for label_def in self.labels:
            for pattern in label_def.heading_patterns:
                compiled.append((label_def.id, _compile_heading_pattern(pattern)))
        return tuple(compiled)

Attributes

compiled_patterns property
description = '' class-attribute instance-attribute
document_profile = Field(default='', description='One sentence describing the kind of document this schema covers, written to be true of no other schema in the catalog. Empty means the schema is not a detection candidate.') class-attribute instance-attribute
id instance-attribute
labels instance-attribute
model_config = ConfigDict(extra='forbid') class-attribute instance-attribute
parent = Field(default=None, description='Reserved for a future document-type hierarchy (e.g. a thesis as a sub-type of academic). Unused today: the catalog is flat, and sub-types would blur the cells a flat detector must separate.') class-attribute instance-attribute
schema_version instance-attribute

Functions:

all_known_label_ids() cached

Source code in ontocast/config/section_labels/__init__.py
@lru_cache(maxsize=1)
def all_known_label_ids() -> frozenset[str]:
    manifest = load_manifest()
    ids: set[str] = set()
    for entry in manifest.schemas:
        schema = load_section_label_schema(entry.id)
        ids.update(canonical_labels(schema))
    return frozenset(ids)

canonical_labels(schema)

Source code in ontocast/config/section_labels/__init__.py
def canonical_labels(schema: SectionLabelSchema) -> tuple[str, ...]:
    return tuple(label_def.id for label_def in schema.labels)

clear_section_label_caches()

Clear loader caches (for tests).

Source code in ontocast/config/section_labels/__init__.py
def clear_section_label_caches() -> None:
    """Clear loader caches (for tests)."""
    load_manifest.cache_clear()
    load_section_label_schema.cache_clear()
    all_known_label_ids.cache_clear()
    _hint_matchers.cache_clear()

get_default_section_schema()

Source code in ontocast/config/section_labels/__init__.py
def get_default_section_schema() -> SectionLabelSchema:
    manifest = load_manifest()
    return load_section_label_schema(manifest.default_schema)

heading_head(normalised)

The head of a subtitled heading ("Appendix A: Derivation ..." -> "Appendix A").

Returns:

Type Description
str | None

The part before the first colon or spaced dash, or None when the

str | None

heading has no subtitle.

Source code in ontocast/config/section_labels/__init__.py
def heading_head(normalised: str) -> str | None:
    """The head of a subtitled heading ("Appendix A: Derivation ..." -> "Appendix A").

    Returns:
        The part before the first colon or spaced dash, or ``None`` when the
        heading has no subtitle.
    """
    parts = _SUBTITLE.split(normalised, maxsplit=1)
    if len(parts) != 2 or not parts[0].strip() or not parts[1].strip():
        return None
    return parts[0].strip()

label_is_terminal(label, schema)

Whether a label names a flat list that no subsection can belong to.

Source code in ontocast/config/section_labels/__init__.py
def label_is_terminal(label: str | None, schema: SectionLabelSchema) -> bool:
    """Whether a label names a flat list that no subsection can belong to."""
    if label is None:
        return False
    return any(
        label_def.terminal for label_def in schema.labels if label_def.id == label
    )

label_route_class(label, schema)

Route class of a label in this schema; content for unknown labels.

Source code in ontocast/config/section_labels/__init__.py
def label_route_class(label: str | None, schema: SectionLabelSchema) -> RouteClass:
    """Route class of a label in this schema; ``content`` for unknown labels."""
    if label is None:
        return RouteClass.CONTENT
    wanted = label.strip().lower()
    for label_def in schema.labels:
        if label_def.id == wanted:
            return label_def.route_class
    return RouteClass.CONTENT

labels_of_class(schema, route_class)

Label ids of one route class, in schema order.

Source code in ontocast/config/section_labels/__init__.py
def labels_of_class(schema: SectionLabelSchema, route_class: RouteClass) -> list[str]:
    """Label ids of one route class, in schema order."""
    return [
        label_def.id
        for label_def in schema.labels
        if label_def.route_class is route_class
    ]

load_manifest() cached

Source code in ontocast/config/section_labels/__init__.py
@lru_cache(maxsize=1)
def load_manifest() -> SectionLabelManifest:
    path = _labels_dir() / "manifest.yaml"
    raw = yaml.safe_load(path.read_text(encoding="utf-8"))
    return SectionLabelManifest.model_validate(raw)

load_section_label_schema(schema_id) cached

Source code in ontocast/config/section_labels/__init__.py
@lru_cache(maxsize=16)
def load_section_label_schema(schema_id: str) -> SectionLabelSchema:
    manifest = load_manifest()
    entry = next((item for item in manifest.schemas if item.id == schema_id), None)
    if entry is None:
        known = ", ".join(item.id for item in manifest.schemas)
        raise ValueError(f"Unknown section schema {schema_id!r}; known: {known}")
    path = _labels_dir() / entry.file
    raw = yaml.safe_load(path.read_text(encoding="utf-8"))
    schema = SectionLabelSchema.model_validate(raw)
    if schema.id != schema_id:
        raise ValueError(
            f"Schema file {entry.file} has id {schema.id!r}, expected {schema_id!r}"
        )
    return schema

match_heading_keywords(line, schema)

Match a heading by keyword, for compound and non-canonical headings.

Resolution follows the heading's head noun, in order:

  1. a terminal label whose keyword is a whole conjunct ("Notes and References" is references); a terminal label matches no other way;
  2. a keyword ending the head phrase -- the first conjunct cut before its first preposition ("Characterization Results" is results, "Summary of Results" a summary, "Results and Discussion" results);
  3. any keyword in the head phrase, rightmost first ("Experimental Section" is methods);
  4. the earliest keyword anywhere in the heading.

Ties go to the longer keyword, then to schema order. English noun-phrase structure is assumed, as the keywords themselves are English.

Parameters:

Name Type Description Default
line str

Raw heading line.

required
schema SectionLabelSchema

Active section label schema.

required

Returns:

Type Description
str | None

The label, or None when no keyword matches.

Source code in ontocast/config/section_labels/__init__.py
def match_heading_keywords(line: str, schema: SectionLabelSchema) -> str | None:
    """Match a heading by keyword, for compound and non-canonical headings.

    Resolution follows the heading's head noun, in order:

    1. a terminal label whose keyword is a whole conjunct ("Notes and
       References" is references); a terminal label matches no other way;
    2. a keyword ending the head phrase -- the first conjunct cut before its
       first preposition ("Characterization Results" is results, "Summary of
       Results" a summary, "Results and Discussion" results);
    3. any keyword in the head phrase, rightmost first ("Experimental
       Section" is methods);
    4. the earliest keyword anywhere in the heading.

    Ties go to the longer keyword, then to schema order. English noun-phrase
    structure is assumed, as the keywords themselves are English.

    Args:
        line: Raw heading line.
        schema: Active section label schema.

    Returns:
        The label, or ``None`` when no keyword matches.
    """
    normalised = normalise_heading_line(line)
    if not normalised or len(normalised) > _MAX_HEADING_LINE_LEN:
        return None

    conjuncts = _conjuncts(normalised)
    for label_def in schema.labels:
        if label_def.terminal and any(
            pattern.fullmatch(conjunct)
            for pattern in label_def.compiled_keywords
            for conjunct in conjuncts
        ):
            return label_def.id

    head = _head_phrase(normalised)
    ending: tuple[int, int, str] | None = None
    inside: tuple[int, int, int, str] | None = None
    anywhere: tuple[int, int, int, str] | None = None
    for index, label_def in enumerate(schema.labels):
        if label_def.terminal:
            continue
        for pattern in label_def.compiled_keywords:
            in_head = pattern.search(head)
            if in_head is not None:
                length = in_head.end() - in_head.start()
                if in_head.end() == len(head):
                    candidate = (-length, index, label_def.id)
                    if ending is None or candidate < ending:
                        ending = candidate
                inside_candidate = (-in_head.end(), -length, index, label_def.id)
                if inside is None or inside_candidate < inside:
                    inside = inside_candidate
            found = pattern.search(normalised)
            if found is not None:
                any_candidate = (
                    found.start(),
                    -(found.end() - found.start()),
                    index,
                    label_def.id,
                )
                if anywhere is None or any_candidate < anywhere:
                    anywhere = any_candidate
    if ending is not None:
        return ending[2]
    if inside is not None:
        return inside[3]
    if anywhere is not None:
        return anywhere[3]
    return None

match_heading_line(line, schema)

Match a heading against the schema's anchored patterns (high precision).

Deliberately exact: this function also gates segment coalescing, where a fuzzy match on a body first line would join distinct sections. Recall lives in :func:match_heading_keywords.

Source code in ontocast/config/section_labels/__init__.py
def match_heading_line(line: str, schema: SectionLabelSchema) -> str | None:
    """Match a heading against the schema's anchored patterns (high precision).

    Deliberately exact: this function also gates segment coalescing, where a
    fuzzy match on a body first line would join distinct sections. Recall lives
    in :func:`match_heading_keywords`.
    """
    normalised = normalise_heading_line(line)
    if not normalised or len(normalised) > _MAX_HEADING_LINE_LEN:
        return None
    for label, pattern in schema.compiled_patterns:
        if pattern.match(normalised):
            return label
    return None

match_heading_name(line, schema)

Match a heading that consists of one of a label's keywords and nothing else.

"Author Contributions" is a name of the acknowledgements section, as exactly as an anchored pattern; "Funding Mechanisms" only contains one.

Source code in ontocast/config/section_labels/__init__.py
def match_heading_name(line: str, schema: SectionLabelSchema) -> str | None:
    """Match a heading that consists of one of a label's keywords and nothing else.

    "Author Contributions" *is* a name of the acknowledgements section, as
    exactly as an anchored pattern; "Funding Mechanisms" only contains one.
    """
    normalised = normalise_heading_line(line)
    if not normalised or len(normalised) > _MAX_HEADING_LINE_LEN:
        return None
    for label_def in schema.labels:
        for pattern in label_def.compiled_keywords:
            if pattern.fullmatch(normalised):
                return label_def.id
    return None

normalise_heading_line(line)

Reduce a raw heading line to its bare section name.

Strips markdown syntax, publisher decoration glyphs, extraction artefacts and section numbering, so that "## ■ REFERENCES" and "2.1 Synthesis of films" reach the matchers as "REFERENCES" and "Synthesis of films".

Source code in ontocast/config/section_labels/__init__.py
def normalise_heading_line(line: str) -> str:
    """Reduce a raw heading line to its bare section name.

    Strips markdown syntax, publisher decoration glyphs, extraction artefacts
    and section numbering, so that "## ■ REFERENCES" and "2.1 Synthesis of
    films" reach the matchers as "REFERENCES" and "Synthesis of films".
    """
    stripped = unicodedata.normalize("NFKC", line).strip()
    if stripped.startswith("#"):
        stripped = stripped.lstrip("#").strip()
    stripped = _MARKDOWN_EMPHASIS.sub("", stripped).strip()
    stripped = _LEADING_DECORATION.sub("", stripped)
    stripped = _TRAILING_DECORATION.sub("", stripped)
    stripped = _ARTEFACT_TOKEN.sub("", stripped)
    stripped = _STRUCTURAL_PREFIX.sub("", stripped).strip()
    stripped = _DIGIT_NUMBERING.sub("", stripped)
    stripped = _ALPHA_NUMBERING.sub("", stripped)
    return stripped.strip()

normalise_llm_label(raw, schema)

Source code in ontocast/config/section_labels/__init__.py
def normalise_llm_label(raw: str | None, schema: SectionLabelSchema) -> str | None:
    if raw is None:
        return None
    cleaned = raw.strip().lower().replace(" ", "_").replace("-", "_")
    allowed = set(canonical_labels(schema))
    if cleaned in allowed:
        return cleaned
    return None

normalise_user_section_label(raw, *, schema_id=None)

Map user-supplied section name to a canonical label.

Source code in ontocast/config/section_labels/__init__.py
def normalise_user_section_label(
    raw: str,
    *,
    schema_id: str | None = None,
) -> str | None:
    """Map user-supplied section name to a canonical label."""
    if raw.strip() == "*":
        return "*"

    cleaned = raw.strip().lower().replace(" ", "_").replace("-", "_")
    if cleaned in all_known_label_ids():
        return cleaned

    resolved_id = resolve_section_schema_id(
        section_schema_id=schema_id,
        document_type_hint=None,
    )
    schema = load_section_label_schema(resolved_id)
    if cleaned in canonical_labels(schema):
        return cleaned

    matched = match_heading_line(raw, schema)
    if matched is not None:
        return matched

    for entry in load_manifest().schemas:
        other = load_section_label_schema(entry.id)
        matched = match_heading_line(raw, other)
        if matched is not None:
            return matched

    return None

resolve_heading_label(line, schema)

Resolve a heading to a label via patterns, then keywords.

A heading that is a keyword and nothing else resolves as a pattern.

Returns:

Type Description
tuple[str, str] | None

(label, source) where source is "heading_pattern" or

tuple[str, str] | None

"heading_keyword"; None when the heading is unrecognised.

Source code in ontocast/config/section_labels/__init__.py
def resolve_heading_label(
    line: str, schema: SectionLabelSchema
) -> tuple[str, str] | None:
    """Resolve a heading to a label via patterns, then keywords.

    A heading that is a keyword and nothing else resolves as a pattern.

    Returns:
        ``(label, source)`` where source is ``"heading_pattern"`` or
        ``"heading_keyword"``; ``None`` when the heading is unrecognised.
    """
    exact = match_heading_line(line, schema) or match_heading_name(line, schema)
    if exact is not None:
        return exact, "heading_pattern"
    keyword = match_heading_keywords(line, schema)
    if keyword is not None:
        return keyword, "heading_keyword"
    return None

resolve_section_schema_id(*, section_schema_id=None, document_type_hint=None)

Pick schema: explicit id, then hint substring match, then manifest default.

Source code in ontocast/config/section_labels/__init__.py
def resolve_section_schema_id(
    *,
    section_schema_id: str | None = None,
    document_type_hint: str | None = None,
) -> str:
    """Pick schema: explicit id, then hint substring match, then manifest default."""
    manifest = load_manifest()
    if section_schema_id and section_schema_id.strip():
        schema_id = section_schema_id.strip().lower()
        load_section_label_schema(schema_id)
        return schema_id

    from_hint = schema_id_from_hint(document_type_hint)
    if from_hint is not None:
        return from_hint

    return manifest.default_schema

schema_id_from_hint(document_type_hint)

Schema a free-text document-type hint maps to, or None if it maps to none.

Distinct from :func:resolve_section_schema_id, which cannot express "the caller told us nothing": it returns the manifest default both for an unmatched hint and for no hint at all. Automatic detection must run in exactly those cases, so it needs this finer answer.

Source code in ontocast/config/section_labels/__init__.py
def schema_id_from_hint(document_type_hint: str | None) -> str | None:
    """Schema a free-text document-type hint maps to, or ``None`` if it maps to none.

    Distinct from :func:`resolve_section_schema_id`, which cannot express "the
    caller told us nothing": it returns the manifest default both for an
    unmatched hint and for no hint at all. Automatic detection must run in
    exactly those cases, so it needs this finer answer.
    """
    if not document_type_hint or not document_type_hint.strip():
        return None
    hint_lower = document_type_hint.strip().lower()
    for pattern, schema_id in _hint_matchers():
        if pattern.search(hint_lower):
            return schema_id
    return None