createEmbeddingPipeline
@forge/monorepo / backend/src / createEmbeddingPipeline
Function: createEmbeddingPipeline()
createEmbeddingPipeline(
deps):object
Defined in: backend/src/knowledge/index.ts:111
Parameters
deps
Returns
freshnessTargetMs
freshnessTargetMs:
number=FRESHNESS_TARGET_MS
model
model:
EmbeddingModelRef=deps.embeddings.model
indexSource()
indexSource(
context,input):Promise<IndexResult>
Index one source, replacing whatever was there.
Replace rather than append: a changed document's old chunks must stop being searchable, because a stale chunk is a citation pointing at text that is no longer in the document.
Parameters
context
tenantId
input
Returns
Promise<IndexResult>
reindexBatch()
reindexBatch(
context,input):Promise<{reindexed:number;remaining:number;skipped:number; }>
One page of a re-index — AC-5.
Deliberately not a loop over everything. A caller runs this repeatedly and stops when remaining is
zero, which means an interruption at any point loses at most one page's work and no bookkeeping: the
next call re-derives the work list from what is stored. There is no cursor to persist and therefore no
cursor to lose.
reload fetches a source's current blocks. It is the caller's, because where a source's content lives
differs by type — an attachment's is an extracted document, an artifact's is its content — and this
module has no business knowing all of them.
Parameters
context
tenantId
input
limit
number
reload
(source) => Promise<{ authSubject: string; blocks: readonly DocumentBlock[]; } | null>
Returns
Promise<{ reindexed: number; remaining: number; skipped: number; }>