ontocast.tool.chunk.bibliography¶
Reference-list detection at block level, and trimming of citation runs.
A unit is split into blocks (:func:ontocast.tool.chunk.structure.split_blocks)
and each text or footnote block is tested as one cited work, or as part of one
when an entry runs over several blocks; captions, formulas, code, pictures and
table rows never are. Units are judged by the share of their blocks that are citation entries,
not by counting citation tokens over the whole unit, so a table that cites its
sources or a discussion that enumerates prior studies is not a reference list.
A reference list does not sit inside prose, so only a run of entries at a unit's start or end is trimmed into a unit of its own; a run in the middle is left alone.
Subject-domain-agnostic but not language- or script-agnostic: markers,
et al., venue abbreviations and the author shapes are Western conventions,
and author-year styles that put the year last rely on the venue cue alone. The
failure mode is a miss -- a reference list extracted as content -- never a
dropped section.
Attributes¶
CONTENT_ONLY_SHARE = 0.8
module-attribute
¶
LABEL_AGREEMENT_SHARE = 0.3
module-attribute
¶
MIN_LIST_ENTRIES = 3
module-attribute
¶
MIN_TRIM_ENTRIES = 5
module-attribute
¶
__all__ = ['CONTENT_ONLY_SHARE', 'LABEL_AGREEMENT_SHARE', 'MIN_LIST_ENTRIES', 'MIN_TRIM_ENTRIES', 'citation_counts', 'citation_share', 'is_citation_entry', 'looks_like_bibliography', 'trim_citation_runs']
module-attribute
¶
Classes¶
Functions:¶
citation_counts(text, kinds=None)
¶
(entries, counted_blocks): citation entries among the unit's text blocks.
Headings, table rows and pictures are not counted, so a table never counts; captions, formulas and code count but are never entries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Unit text. |
required |
kinds
|
BlockKinds | None
|
Docling block kinds of the document, when it was converted. |
None
|
Source code in ontocast/tool/chunk/bibliography.py
citation_share(text, kinds=None)
¶
is_citation_entry(text)
¶
Whether one block reads as a single cited work.
It opens with a citation marker or an author shape, carries a publication year and a venue cue (DOI, venue token or phrase, volume and pages, or an abbreviated venue), has the length of one entry, and states no measurement outside its links and DOIs. The last test separates "Smith et al. (2019) stored films at 85 °C ..." from a reference.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
One block. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
True when the block is a citation entry. |
Source code in ontocast/tool/chunk/bibliography.py
looks_like_bibliography(text, kinds=None)
¶
Whether the content alone makes a unit a reference list.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Unit text. |
required |
kinds
|
BlockKinds | None
|
Docling block kinds of the document, when it was converted. |
None
|
Returns:
| Type | Description |
|---|---|
bool
|
True when at least :data: |
bool
|
|
Source code in ontocast/tool/chunk/bibliography.py
trim_citation_runs(text, kinds=None)
¶
Cut citation runs at a unit's start or end into pieces of their own.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Unit text. |
required |
kinds
|
BlockKinds | None
|
Docling block kinds of the document, when it was converted. |
None
|
Returns:
| Type | Description |
|---|---|
list[tuple[str, bool]]
|
|
list[tuple[str, bool]]
|
no run at either edge comes back whole as |
list[tuple[str, bool]]
|
that is all one run comes back as |