Evaluations
The pass/fail criteria a run is judged against, every field of a criterion, what each result does to the run, and how the results fold into a verdict.
An evaluation is a reusable set of pass/fail criteria. Each criterion watches one readable channel inside a window of the run and judges its recorded samples against a fixed band or a tolerance around another channel; each closes with a result of its own, and the run's verdict is the fold of them. The engine judges live over the rows the datalog records, so a criterion's channel and its reference channel must be columns of the procedure's datalog: the run file proves what was judged. Evaluations live under Process > Models > model > Setup > Evaluations and need the Edit recipes & evaluations permission.
The Evaluations folder
Selecting the Evaluations row opens the folder page ("Pass/fail criteria a run is judged against. The evaluation reads process channels.") with one card per evaluation, by name; clicking a card opens it and the X removes it. With none, the page says "No evaluations yet. Add one."
| Command | What it does | Greyed when (situation) | Not drawn when (role) |
|---|---|---|---|
| Evaluation (detail bar, folder selected; right-click Add evaluation) | Adds Evaluation <n> with no criteria and selects it. |
Any unit holds a run, or the runtime is stopped. | Without Edit recipes & evaluations. |
The evaluation's page
The header carries the name, the line "Pass/fail criteria a run is judged against." and a pill counting the criteria. Every field autosaves, and every edit redraws the preview of the picked criterion.
Definition
| Field | What it is | Values / default |
|---|---|---|
| Name | What the evaluation is called in the tree, in a procedure's Evaluation picker and in the Evaluate now picker of a panel. | Free text; Evaluation <n> at creation. |
| Notes | Free text for whoever reads it. | Empty by default. |
How the run is judged
This card states the fold the engine applies, in its own order:
- NOK if any one criterion fails.
- Indeterminate if none failed and one could not be judged.
- OK only when every one of them passed.
"Severity changes none of that. It is read against the procedure's abort threshold and decides
only whether a failure also ends the run there." The card also warns before a run does: "One
criterion cannot be judged as configured, so a run would end Indeterminate." (or "<n> criteria
cannot be judged …"), and says "No procedure of this model runs against this evaluation yet."
while nothing references it.
The criteria list
The list is headed Criteria and its Criterion button adds a row on the model's first readable channel (anchor Whole run, mode Whole window, a fixed band with both sides open, severity Major, no name) and picks it. Clicking a row opens it in the Criterion column; with nothing picked the column says "Select a criterion to edit all of it in one place, with what each field decides beside it." The list carries the whole criterion and scrolls sideways past the width of the panel.
| Column | What it holds |
|---|---|
| Name | The criterion's name, editable in place. |
| Channel | The readable channel judged. |
| Anchor | Whole run, Time window or Segment, each with the shape of the stretch it judges beside its name and its consequence as tooltip. |
| Window | The two settings the anchor needs: Seg and Stab for a Segment anchor, From and To for a Time window anchor, "Whole run" for the Whole run anchor. |
| Mode | Whole window or One point, with its shape. |
| Limit | Fixed band or Around the setpoint, with its shape. |
| Band | Min and Max for a fixed band ("blank leaves the side open"); the reference channel and ± tolerance for a relative limit. |
| Severity | Minor, Major or Critical. |
| Result | What the criterion contributes to the run ("NOK fails the run", "NOK fails and stops the run", "Cannot be judged"), with the full sentence as tooltip. |
| Row action | Remove asks "Delete criterion": the name (or the channel's name), "This removes the criterion from the evaluation." |
The criterion in full
The column opens with a preview: the picked criterion's band and the stretch it judges, over declared axes with no sample near them (there is no run behind a configuration screen). A fixed band is drawn in the channel's engineering values with room for a failure on each side; a relative band is drawn as deviation from the reference channel, which stands as the line at zero. When nothing can be drawn (no limit set, no reference picked) the frame says why instead of standing empty. Under it, every field stands over a fixed legend and, where the answer moves with the entries, the effect of what is in force.
| Field | What it is | Values / default | Effect line |
|---|---|---|---|
| Name | "Names this check wherever its result is shown." Empty falls back to the channel on the report; the column heads itself Criterion <n> until the row is named. |
Free text; empty at creation. | |
| Channel | "The reading judged. It must be a datalog column of the procedure." | The model's readable channels; the first at creation. | |
| Anchor (WINDOW) | "Which stretch of the run this criterion is judged over." | Whole run (default), Time window, Segment. | Whole run: "Judged over the whole run. It needs no recipe." Time window: "Judged between two instants measured from run start." Segment: "Judged while the recipe is in that segment, after the delay. A hold extends it." |
| Segment (0-based) and Stabilization (ms) (Segment anchor) | "Which segment of the recipe is judged, and how long into it the window waits before opening." | Whole numbers; 0 and 0 by default. Neither field takes a negative number: the entry is refused, the note says so under the field, and the previous value comes back. | "Opens with segment 1." or "Opens 2 s into segment 1." |
| From (ms) and To (ms, 0 = run end) (Time window anchor) | "The two instants, counted from the start of the run, the window stands between." | Milliseconds; 0 and 0 by default. To = 0 means until the run ends; otherwise To must stand after From. | "From 5 s to the end of the run.", "From 5 s to 30 s.", or "To stands at or before From, so the window is empty." |
| Mode | "How much of that stretch the verdict rests on." | Whole window (default), One point. | Whole window: "Every sample in the window must pass." One point: "Only the last sample of the window decides." |
| Limit (LIMITS) | "What form the limit takes, which is also what the fields under it ask for." | Fixed band (default), Around the setpoint. | Fixed band: "A fixed engineering band. Either side may be left open." Around the setpoint: "A band around another channel, sampled on the same row." |
| Min and Max (fixed band) | "The engineering values the reading has to stay between. A side left empty stays open." | Numbers or empty; both empty at creation. | "No limit set, so every reading passes.", "Passes at 20 or above.", "Passes at 80 or below.", "Passes between 20 and 80.", or "Min stands above Max, so this cannot be judged and a run would end Indeterminate.", which is what the RESULT block under the row says too |
| Reference and ± Tolerance (relative) | "The channel the reading is measured against, and how far from it it may sit." | A readable channel of the model and a number; none and 0 at creation. | "Passes within ±0.5 of Setpoint." or "Pick the channel the tolerance is measured around." |
| Severity | "Read against the procedure's abort threshold. It also sets the emphasis on the report." | Minor, Major (default), Critical. | |
| RESULT | What the criterion produces, and what that does to the run. | Read-only. | See below. |
What the result says
| Line | Detail | When |
|---|---|---|
| "NOK fails the run" | "A failure makes the run NOK." or, when some procedure runs the evaluation, "A failure makes the run NOK. The run still finishes, because no procedure aborts at this severity." | No procedure running this evaluation aborts at this criterion's severity. |
| "NOK fails and stops the run" | "A failure makes the run NOK and ends it there, because <procedure> aborts at this severity." |
A procedure running this evaluation has Abort on NOK at or below this severity. |
| "Cannot be judged" | "As configured it cannot be judged, so it would leave the run Indeterminate." followed by the reason. | One of the reasons below holds. |
The reasons a criterion cannot be judged, asked at configuration time in the engine's own order: "It watches no channel of this model."; "The channel it watches is not recorded by the datalog of the procedure that runs it."; "It follows no reference channel of this model."; "To stands at or before From, so the window is empty."; "Its limits are incomplete or the wrong way round."; "The reference channel is not recorded by the datalog of the procedure that runs it." The datalog reasons are asked only of the procedures that run the evaluation; an evaluation no procedure has picked up yet is unfinished, not wrong.
How criteria fold into a verdict
Every recorded row inside an active window is classified once: evaluated, unavailable (a
missing value or a bad quality) or invalid (a good value that is not a finite number; a good
NaN stays auditable raw data but cannot certify a numeric criterion). A fixed limit needs the
watched value; a relative limit needs the watched and the reference value from the same row. The
counters Samples evaluated, Samples out, Samples unavailable and Samples invalid and the frozen set
of affected channels are recorded per criterion, and are also readable live under the unit's
Procedure.* points (see The unit address space).
Outcomes are strict about evidence. In Window mode any evaluated violation proves NOK; otherwise an unavailable or invalid row, an interrupted observation or an empty window makes the criterion Indeterminate; only complete in-band evidence makes it OK. Point mode decides on the window's last active sample alone; earlier classifications stay in the counters. A Temporal procedure re-anchors every criterion per cycle and counts the failed cycles. The run's verdict folds in order: an aborted run or any NOK criterion makes it NOK; otherwise any Indeterminate result makes it Indeterminate; results all OK make it OK; a run with no evaluation ends with no verdict. Only a confirmed NOK triggers the procedure's abort threshold or the failed-cycle tally.
Where an evaluation is used
| Surface | What it does with the evaluation |
|---|---|
| A procedure's Judge stage | Judges every run of the procedure; the Abort on NOK ≥ picker arrives with it. The stage line says "Judges 4 criteria, and a Major NOK aborts the run." or "…, and a NOK never aborts the run." A procedure with an evaluation and no datalog does not validate. See Procedures. |
| The Single commands card of a panel | Evaluate now records one row through a datalog and judges it against the evaluation; the verdict is recorded in Histories. The evaluation's channels must be columns of the chosen datalog. |
| Label commands | A marker may read a criterion's name, channel, limits, severity, outcome, error, window instants, sample counters, affected channels or worst excursion, for an explicit cycle. See Actions. |
| Reports | The criteria section prints each criterion's result with the emphasis its severity sets. |
Remove evaluation
Remove evaluation stands in the detail bar's danger cluster with the evaluation selected, on its right-click menu, and as the X on the folder page's card. It asks first ("Delete evaluation", "This removes the evaluation and its criteria."). Every procedure that judged by it empties its Evaluation reference and ends its runs with no verdict. It needs Edit recipes & evaluations and is greyed under the engineering lock and the stopped runtime.
What an evaluation does not do
- It judges nothing a datalog does not record, and nothing outside a run or a snapshot.
- It has no expression criteria: a criterion is a band or a tolerance on one channel.
- It differs between units in nothing: one set of limits serves every unit of the model.
- It ends no run by itself. Whether a NOK aborts the run is the procedure's Abort on NOK threshold, read against the criterion's severity.