ontocast.tool.sentence_transformer¶
One process-wide cache of guarded local sentence-transformer encoders.
Three independent subsystems load local sentence-transformers: semantic chunking
(:mod:ontocast.tool.chunk.chunker), retrieval embeddings
(:mod:ontocast.tool.vector_store.embedding) and entity clustering
(:mod:ontocast.tool.agg.clustering). Point two of them at the same checkpoint
and a per-subsystem cache means the same weights resident twice; point all three
at it and it is three times.
Caching alone is not enough, though, and getting it half-right is worse than not
sharing at all: once two subsystems hold the same model object, a lock that
lives in one of them protects nothing. So the cache hands out a
:class:SharedEncoder that owns the model and the lock that serialises it.
Every consumer encodes through that one guarded path, and — unlike a single
process-wide lock — two different checkpoints never serialise against each other.
Sharing weights is not the same as sharing semantics: retrieval applies
EmbeddingConfig document/query prefixes and clustering and chunking do not.
The three consumers are interchangeable in what they load, not in what they
mean.
Attributes¶
logger = logging.getLogger(__name__)
module-attribute
¶
Classes¶
SharedEncoder
¶
A process-shared SentenceTransformer and the only way to use it.
The model is private: every use of it or its tokenizer goes through this
handle, under one lock. A HuggingFace fast tokenizer keeps its truncation
setting as mutable state, and encode (which truncates) and
:meth:token_lengths (which must not) each set it. Two threads doing so at
once raise RuntimeError: Already borrowed from the tokenizer. The lock
also bounds peak memory: every concurrent encode allocates its own
activation batch, and the unit fan-out is PARALLEL_WORKERS wide.
Source code in ontocast/tool/sentence_transformer.py
Attributes¶
device = device
instance-attribute
¶
max_seq_length
property
¶
Tokens the model sees before truncating; None if it states none.
model_name = model_name
instance-attribute
¶
Methods:¶
__init__(model_name, model, *, device=None)
¶
Wrap a loaded model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name
|
str
|
Checkpoint id this was loaded from. |
required |
model
|
Any
|
The loaded |
required |
device
|
str | None
|
Device it was requested on; |
None
|
encode(texts, **kwargs)
¶
Encode texts, serialised against other users of this model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
texts
|
list[str]
|
Strings to encode. |
required |
**kwargs
|
Any
|
Passed through to |
{}
|
Returns:
| Type | Description |
|---|---|
Any
|
Whatever |
token_lengths(texts)
¶
Word pieces per text, special tokens included, without truncation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
texts
|
list[str]
|
Strings to measure, exactly as they would be encoded. |
required |
Returns:
| Type | Description |
|---|---|
list[int]
|
list[int]: One length per text. |
SharedSentenceTransformerEmbeddings
¶
Bases: Embeddings
LangChain Embeddings view over a :class:SharedEncoder.
langchain_huggingface.HuggingFaceEmbeddings always constructs its own
SentenceTransformer (its model config forbids extra fields, so a
prebuilt model cannot be injected), which is why semantic chunking used to
hold a second copy of a checkpoint the process already had resident. This
adapter is the whole of the interface that class provided to us.
Source code in ontocast/tool/sentence_transformer.py
Functions:¶
get_shared_encoder(model_name, *, device=None, feature='Local sentence-transformer models')
¶
Return the process-wide guarded encoder for (model_name, device).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name
|
str
|
HuggingFace model id or local path. |
required |
device
|
str | None
|
Device to load on; |
None
|
feature
|
str
|
What needs the model, used in the missing-dependency message. |
'Local sentence-transformer models'
|
Returns:
| Name | Type | Description |
|---|---|---|
SharedEncoder |
SharedEncoder
|
The shared handle. |