Probabilistic systems fail silently, drift without warning, and produce confident outputs that are structurally wrong. The failure modes are categorically different — and so are the governance requirements. Most organizations deploying AI cannot answer two questions their boards will eventually ask: what did we get for it, and how much should we trust it. This page describes the methodology, the standards, and the tools for managing AI development with documented, auditable quality control at every phase.
Conventional project tools record task status. They cannot infer the structural conditions that precede visible failure. Projects appear green on the dashboard, then enter crisis within weeks. The gap is not incidental — it is architectural.
Probabilistic systems degrade silently: distributional shift, calibration erosion, confabulation at inference time. No error message fires. The system keeps producing outputs. They are increasingly wrong. Standard QA processes do not detect this.
40% of agentic AI projects are projected to be cancelled by 2027 — not because the models failed, but because governance and escalation logic were never defined. No audit trail. No named human decision owner. No quality gate.
Six phases. A mandatory quality gate before every release. PRIMMS-GPT providing project intelligence across the full arc — from business need scoping to post-deployment regime-change detection.
The NIST AI Risk Management Framework (AI RMF 1.0) is the authoritative federal standard for AI risk governance. It defines the outcomes organizations need to achieve. It does not prescribe how to achieve them — that operational gap is precisely what RATIO fills, using the OCC audit methodology, Bayesian WoE review, and structured human accountability controls.
NIST organizes AI risk management around four core functions: Govern, Map, Measure, and Manage. Each function names an outcome. None of them specifies the operational methodology to achieve it — the audit procedures, the evidence standards, the human authority gates, the verdict criteria. That is the gap RATIO closes.
Our implementation translates each NIST function into a working system: role accountability structures your board can interrogate, use-case risk maps grounded in documented evidence, OCC-5C technical validation with WoE verdicts, and post-deployment governance controls with named human decision owners at every escalation point.
RATIO is not affiliated with or endorsed by NIST. We help organizations apply the public NIST AI RMF 1.0 to their specific operating environment using practitioner-tested methods.
The NIST AI RMF is intentionally non-prescriptive. It tells organizations what to achieve — not how. RATIO provides the how: the OCC audit standard, the WoE verdict methodology, and the human accountability architecture that make NIST compliance operational rather than aspirational.
RATIO · NIST AI RMF 1.0 Operationalization · Not affiliated with or endorsed by NISTConventional project management tools are record-keeping systems. They capture what has happened and display it. They do not infer the structural conditions — team confidence erosion, perception distortion in leadership, governance gap formation — that precede most project crises.
"The signals of structural deterioration accumulate invisibly, optimism bias suppresses their interpretation, and the window for low-cost correction closes before leadership recognises it."
Aaron — Seeing What the Dashboard Misses (2026)PRIMMS-GPT addresses this through three independent signal modalities fused through a Bayesian weight-of-evidence framework. Each modality carries information the others do not fully capture. Their combination produces a materially stronger evidence base than any single source.
Three specific failure modes characterise AI-assisted project management without a structured governance architecture:
PRIMMS-GPT is explicitly designed to counter all three AI-specific failure modes: multi-modal Bayesian signal fusion defeats proxy gaming; the advisory-only posture preserves practitioner judgment; the deterministic MATLAB signal layer operates independently of any LLM and has no social incentive to satisfy the project manager.
Each signal stream carries independent information about project risk. The fusion produces a Bayesian weight-of-evidence total — expressed in decibans, classified against Jeffreys bands — before any LLM interpretation is consulted.
Schedule state, days behind plan, jeopardy index, velocity-projected finish date, rundown gap, R² fit quality. Each signal mapped to a likelihood ratio and expressed in decibans. The issue closure ratio θ governs phase-exit thresholds: θ ≥ 0.95 for exit, θ < 0.70 is a deterministic HOLD.
Distributed team confidence ratings on a 1–6 scale, timestamped and stored as a time series. Mean, trend, and safe-band classification computed across each phase. VoT deterioration precedes schedule signal deterioration by 1–3 weeks — a structural leading indicator that no dashboard captures.
Status reports, steering committee documents, risk logs, meeting records — ingested via keyword and pattern extraction across 47 practitioner-validated risk categories including nine OCC-specific AI failure categories. CLASSIFIER_JEOPARDY (18 dB) fires when documentary evidence independently reaches decisive classification.
"A machine learning system applied to a real-time operational decision problem achieved 73% accuracy with one modality, 82% with two, and 100% with three. The implication for PRIMMS-GPT is direct: schedule data, VoT sentiment, and documentary text each carry independent information about project risk that the others do not fully capture."
Aaron — Seeing What the Dashboard Misses (2026) · citing Aaron (2019)Before any AI system is deployed, the full document and codebase are submitted to a structured adversarial review. Five quality dimensions are evaluated. A Bayesian weight-of-evidence verdict is issued. No deployment proceeds until all FATAL and WARN findings are resolved.
Does this output contradict itself or prior outputs from the same system? Internal coherence check across all claims.
Do the conclusions follow from the reasoning? Are there unsupported logical leaps between evidence and finding?
What is the strongest available counter-evidence? The adversarial challenger must identify it, cited to specific signals — not narrative.
Is every number traceable to a computation? Every claim traceable to evidence? Plausible-sounding outputs without a derivation chain are flagged.
Is stated confidence proportionate to the evidence base? Overconfidence is a primary AI failure mode — this dimension catches it before deployment.
"The illustrative example produces a schedule jeopardy classification at 65.5 dB (decisive evidence strength) and identifies a four-day actionable recovery window that was not surfaced through the conventional monitoring in use at the time."
Aaron — Seeing What the Dashboard Misses (2026) · AbstractThe classifier maps the convergent signal pattern to one of six empirically grounded failure modes. Archetype identification is the qualitative inference step that schedule analytics alone cannot perform — it requires the fusion of VoT sentiment with schedule and documentary evidence.
Each archetype has a characteristic recovery posture. The Cortical Hierarchy routes the classified signal pattern through a five-layer LLM interpretation architecture — feature detection, gestalt pattern recognition, temporal situation awareness, executive planning, and sponsor-ready briefing generation. The machine orients. The human decides.
At each phase boundary, PRIMMS-GPT aggregates the full session history into phase-level metrics and applies governing-equation thresholds before any LLM recommendation is considered. A HOLD from the θ calculation cannot be overridden by a persuasive status report.
| Gate | Phase Exit Question | θ Threshold | Governing Verdict |
|---|---|---|---|
| QG-1 | Is this the right problem for AI? | — | OCC scoping check complete |
| QG-2 | Are requirements sufficient to build against? | θ ≥ 0.70 | Conditional exit eligible |
| QG-3 | Is the architecture governance-sound? | θ ≥ 0.70 | Conditional exit eligible |
| QG-4 | Has the build met quality gates at each milestone? | θ ≥ 0.70 | Conditional exit eligible |
| QG-5 ⚑ | Has the OCC-5C pre-deployment audit been cleared? | θ ≥ 0.95 | EXIT eligible — FATAL/WARN resolved |
| QG-6 | Is ongoing governance in place? | θ ≥ 0.95 | Deploy authorised |