Grader

@forge/monorepo

---

@forge/monorepo / backend/src / Grader

Interface: Grader

Defined in: backend/src/evaluation/graders.ts:65

Properties

deterministic

readonly deterministic: boolean

Defined in: backend/src/evaluation/graders.ts:77

True when no model is called. Asserted by test, not trusted.

---

id

readonly id: string

Defined in: backend/src/evaluation/graders.ts:66

---

kinds

readonly kinds: readonly string[]

Defined in: backend/src/evaluation/graders.ts:75

The expectation kinds this grader decides.

---

version

readonly version: string

Defined in: backend/src/evaluation/graders.ts:73

Bumped whenever the grader's judgement changes.

Stored on every result, because a score that moved after a grader edit is not a quality change — and without the version on the result the two are indistinguishable.

Methods

grade()

grade(input): Promise<EvalVerdict>

Defined in: backend/src/evaluation/graders.ts:78

Parameters

input

case

EvalCase

output

EvalOutput

Returns

Promise<EvalVerdict>