createEvalHarness
@forge/monorepo / backend/src / createEvalHarness
Function: createEvalHarness()
createEvalHarness(
deps):object
Defined in: backend/src/evaluation/index.ts:65
Parameters
deps
Returns
score()
score(
context,input):Promise<HarnessRunResult>
Score every case and store the run.
The whole dataset in one run, because a partial run cannot gate a release and a run that skipped what it could not grade would gate on a shrinking subset without saying so.
Parameters
context
input
cases
readonly EvalCase[]
release
string
Returns
Promise<HarnessRunResult>