# Evaluations

> The pass/fail criteria a run is judged against, every field of a criterion, what each result does to the run, and how the results fold into a verdict.

An **evaluation** is a reusable set of pass/fail criteria. Each criterion watches one readable
channel inside a window of the run and judges its recorded samples against a fixed band or a
tolerance around another channel; each closes with a result of its own, and the run's verdict is
the fold of them. The engine judges live over the rows the datalog records, so a criterion's
channel and its reference channel must be columns of the procedure's datalog: the run file proves
what was judged. Evaluations live under Process > Models > model > Setup > **Evaluations** and
need the **Edit recipes & evaluations** permission.

## The Evaluations folder {#the-evaluations-folder}

Selecting the **Evaluations** row opens the folder page ("Pass/fail criteria a run is judged
against. The evaluation reads process channels.") with one card per evaluation, by name; clicking
a card opens it and the X removes it. With none, the page says "No evaluations yet. Add one."

| Command | What it does | Greyed when (situation) | Not drawn when (role) |
| --- | --- | --- | --- |
| **Evaluation** (detail bar, folder selected; right-click **Add evaluation**) | Adds `Evaluation <n>` with no criteria and selects it. | Any unit holds a run, or the runtime is stopped. | Without Edit recipes & evaluations. |

## The evaluation's page {#the-evaluations-page}

The header carries the name, the line "Pass/fail criteria a run is judged against." and a pill
counting the criteria. Every field autosaves, and every edit redraws the preview of the picked
criterion.

### Definition {#definition}

| Field | What it is | Values / default |
| --- | --- | --- |
| **Name** | What the evaluation is called in the tree, in a procedure's Evaluation picker and in the Evaluate now picker of a panel. | Free text; `Evaluation <n>` at creation. |
| **Notes** | Free text for whoever reads it. | Empty by default. |

### How the run is judged {#how-the-run-is-judged}

This card states the fold the engine applies, in its own order:

- **NOK** if any one criterion fails.
- **Indeterminate** if none failed and one could not be judged.
- **OK** only when every one of them passed.

"Severity changes none of that. It is read against the procedure's abort threshold and decides
only whether a failure also ends the run there." The card also warns before a run does: "One
criterion cannot be judged as configured, so a run would end Indeterminate." (or "`<n>` criteria
cannot be judged …"), and says "No procedure of this model runs against this evaluation yet."
while nothing references it.

## The criteria list {#the-criteria-list}

The list is headed **Criteria** and its **Criterion** button adds a row on the model's first
readable channel (anchor Whole run, mode Whole window, a fixed band with both sides open, severity Major, no
name) and picks it. Clicking a row opens it in the **Criterion** column; with nothing picked the
column says "Select a criterion to edit all of it in one place, with what each field decides
beside it." The list carries the whole criterion and scrolls sideways past the width of the panel.

| Column | What it holds |
| --- | --- |
| **Name** | The criterion's name, editable in place. |
| **Channel** | The readable channel judged. |
| **Anchor** | Whole run, Time window or Segment, each with the shape of the stretch it judges beside its name and its consequence as tooltip. |
| **Window** | The two settings the anchor needs: **Seg** and **Stab** for a Segment anchor, **From** and **To** for a Time window anchor, "Whole run" for the Whole run anchor. |
| **Mode** | Whole window or One point, with its shape. |
| **Limit** | Fixed band or Around the setpoint, with its shape. |
| **Band** | **Min** and **Max** for a fixed band ("blank leaves the side open"); the reference channel and **±** tolerance for a relative limit. |
| **Severity** | Minor, Major or Critical. |
| **Result** | What the criterion contributes to the run ("NOK fails the run", "NOK fails and stops the run", "Cannot be judged"), with the full sentence as tooltip. |
| Row action | **Remove** asks "Delete criterion": the name (or the channel's name), "This removes the criterion from the evaluation." |

## The criterion in full {#the-criterion-in-full}

The column opens with a preview: the picked criterion's band and the stretch it judges, over
declared axes with no sample near them (there is no run behind a configuration screen). A fixed
band is drawn in the channel's engineering values with room for a failure on each side; a relative
band is drawn as deviation from the reference channel, which stands as the line at zero. When
nothing can be drawn (no limit set, no reference picked) the frame says why instead of standing
empty. Under it, every field stands over a fixed legend and, where the answer moves with the
entries, the effect of what is in force.

| Field | What it is | Values / default | Effect line |
| --- | --- | --- | --- |
| **Name** | "Names this check wherever its result is shown." Empty falls back to the channel on the report; the column heads itself `Criterion <n>` until the row is named. | Free text; empty at creation. | |
| **Channel** | "The reading judged. It must be a datalog column of the procedure." | The model's readable channels; the first at creation. | |
| **Anchor** (WINDOW) | "Which stretch of the run this criterion is judged over." | **Whole run** (default), **Time window**, **Segment**. | Whole run: "Judged over the whole run. It needs no recipe." Time window: "Judged between two instants measured from run start." Segment: "Judged while the recipe is in that segment, after the delay. A hold extends it." |
| **Segment (0-based)** and **Stabilization (ms)** (Segment anchor) | "Which segment of the recipe is judged, and how long into it the window waits before opening." | Whole numbers; 0 and 0 by default. Neither field takes a negative number: the entry is refused, the note says so under the field, and the previous value comes back. | "Opens with segment 1." or "Opens 2 s into segment 1." |
| **From (ms)** and **To (ms, 0 = run end)** (Time window anchor) | "The two instants, counted from the start of the run, the window stands between." | Milliseconds; 0 and 0 by default. To = 0 means until the run ends; otherwise To must stand after From. | "From 5 s to the end of the run.", "From 5 s to 30 s.", or "To stands at or before From, so the window is empty." |
| **Mode** | "How much of that stretch the verdict rests on." | **Whole window** (default), **One point**. | Whole window: "Every sample in the window must pass." One point: "Only the last sample of the window decides." |
| **Limit** (LIMITS) | "What form the limit takes, which is also what the fields under it ask for." | **Fixed band** (default), **Around the setpoint**. | Fixed band: "A fixed engineering band. Either side may be left open." Around the setpoint: "A band around another channel, sampled on the same row." |
| **Min** and **Max** (fixed band) | "The engineering values the reading has to stay between. A side left empty stays open." | Numbers or empty; both empty at creation. | "No limit set, so every reading passes.", "Passes at 20 or above.", "Passes at 80 or below.", "Passes between 20 and 80.", or "Min stands above Max, so this cannot be judged and a run would end Indeterminate.", which is what the RESULT block under the row says too |
| **Reference** and **± Tolerance** (relative) | "The channel the reading is measured against, and how far from it it may sit." | A readable channel of the model and a number; none and 0 at creation. | "Passes within ±0.5 of Setpoint." or "Pick the channel the tolerance is measured around." |
| **Severity** | "Read against the procedure's abort threshold. It also sets the emphasis on the report." | **Minor**, **Major** (default), **Critical**. | |
| **RESULT** | What the criterion produces, and what that does to the run. | Read-only. | See below. |

### What the result says {#what-the-result-says}

| Line | Detail | When |
| --- | --- | --- |
| "NOK fails the run" | "A failure makes the run NOK." or, when some procedure runs the evaluation, "A failure makes the run NOK. The run still finishes, because no procedure aborts at this severity." | No procedure running this evaluation aborts at this criterion's severity. |
| "NOK fails and stops the run" | "A failure makes the run NOK and ends it there, because `<procedure>` aborts at this severity." | A procedure running this evaluation has Abort on NOK at or below this severity. |
| "Cannot be judged" | "As configured it cannot be judged, so it would leave the run Indeterminate." followed by the reason. | One of the reasons below holds. |

The reasons a criterion cannot be judged, asked at configuration time in the engine's own order:
"It watches no channel of this model."; "The channel it watches is not recorded by the datalog of
the procedure that runs it."; "It follows no reference channel of this model."; "To stands at or
before From, so the window is empty."; "Its limits are incomplete or the wrong way round."; "The
reference channel is not recorded by the datalog of the procedure that runs it." The datalog
reasons are asked only of the procedures that run the evaluation; an evaluation no procedure has
picked up yet is unfinished, not wrong.

## How criteria fold into a verdict {#how-criteria-fold-into-a-verdict}

Every recorded row inside an active window is classified once: **evaluated**, **unavailable** (a
missing value or a bad quality) or **invalid** (a good value that is not a finite number; a good
`NaN` stays auditable raw data but cannot certify a numeric criterion). A fixed limit needs the
watched value; a relative limit needs the watched and the reference value from the same row. The
counters Samples evaluated, Samples out, Samples unavailable and Samples invalid and the frozen set
of affected channels are recorded per criterion, and are also readable live under the unit's
`Procedure.*` points (see [The unit address space](process-addresses)).

Outcomes are strict about evidence. In **Window** mode any evaluated violation proves **NOK**;
otherwise an unavailable or invalid row, an interrupted observation or an empty window makes the
criterion **Indeterminate**; only complete in-band evidence makes it **OK**. **Point** mode decides
on the window's last active sample alone; earlier classifications stay in the counters. A
Temporal procedure re-anchors every criterion per cycle and counts the failed cycles. The run's
verdict folds in order: an aborted run or any NOK criterion makes it **NOK**; otherwise any
Indeterminate result makes it **Indeterminate**; results all OK make it **OK**; a run with no
evaluation ends with no verdict. Only a confirmed NOK triggers the procedure's abort threshold or
the failed-cycle tally.

## Where an evaluation is used {#where-an-evaluation-is-used}

| Surface | What it does with the evaluation |
| --- | --- |
| A procedure's **Judge** stage | Judges every run of the procedure; the **Abort on NOK ≥** picker arrives with it. The stage line says "Judges 4 criteria, and a Major NOK aborts the run." or "…, and a NOK never aborts the run." A procedure with an evaluation and no datalog does not validate. See [Procedures](process-procedures). |
| The **Single commands** card of a panel | **Evaluate now** records one row through a datalog and judges it against the evaluation; the verdict is recorded in Histories. The evaluation's channels must be columns of the chosen datalog. |
| Label commands | A marker may read a criterion's name, channel, limits, severity, outcome, error, window instants, sample counters, affected channels or worst excursion, for an explicit cycle. See [Actions](process-actions#markers). |
| Reports | The criteria section prints each criterion's result with the emphasis its severity sets. |

## Remove evaluation {#remove-evaluation}

**Remove evaluation** stands in the detail bar's danger cluster with the evaluation selected, on
its right-click menu, and as the X on the folder page's card. It asks first ("Delete evaluation",
"This removes the evaluation and its criteria."). Every procedure that judged by it empties its
Evaluation reference and ends its runs with no verdict. It needs Edit recipes & evaluations and is
greyed under the engineering lock and the stopped runtime.

## What an evaluation does not do {#what-an-evaluation-does-not-do}

- It judges nothing a datalog does not record, and nothing outside a run or a snapshot.
- It has no expression criteria: a criterion is a band or a tolerance on one channel.
- It differs between units in nothing: one set of limits serves every unit of the model.
- It ends no run by itself. Whether a NOK aborts the run is the procedure's Abort on NOK
  threshold, read against the criterion's severity.