CapabilitiesModel development and validation
Model development and validation
Define the fault or event a model is meant to recognise, then test whether it gives the plant enough warning to make a decision. Retain the model version, evaluation population and evidence supporting the release decision.
Product documentation · Updated 15 November 2026
- Who uses it
- Signal analysts, model engineers and reliability leads
#Define the target and prepare evidence
Specify whether the target is abnormal vibration, a likely fault type, an alarm-limit crossing or a verified component failure. These require different labels. Retain sampling conditions, operating regimes, sensor positions and inspection outcomes. Mark inconclusive inspections explicitly.
#Compare candidate models
The ORCAS design includes signal features such as frequency-domain measures, sequence modelling, fault classification and explanations of contributing features. Compare candidates against a simple baseline on held-out machines or time periods. Adjacent samples from one incident should stay together so the same event cannot inflate both training and evaluation scores.
#Qualify the model for use
Report false alerts per machine-week, missed confirmed events, warning lead time and the number of reviewed cases. Compare errors by machine type and operating regime. Document where evidence is sparse and the conditions that require a human review or a return to the previous model.
#Outcome and verification
- What the team gets
- A model qualification record containing the target event, version, baseline, eligible machines, test results and outstanding limitations.
- How to verify it
- Ask a reviewer to reproduce the evaluation on the held-out cases and trace one false alarm to the source data and field finding.