AI systems in production carry a distinct risk profile from traditional software systems. They can degrade in performance over time without obvious failure. They can produce outputs that are systematically biased in ways that are not immediately visible. They can be adversarially manipulated. And they can behave unexpectedly in real-world conditions that differ from their training environment. The AI Risk Taxonomy AI risks in production fall into five categories. Model risk is the risk that the AI model performs differently in production than in testing. Data risk is the risk that input data is inaccurate, incomplete, or manipulated in ways that degrade output quality or create biased outcomes. Governance risk is the risk that the AI system operates outside its defined parameters without detection. Operational risk is the risk that the AI system fails or becomes unavailable in ways that disrupt business operations. Regulatory risk is the risk that the AI system's operation creates compliance failures. Model Performance Monitoring Model performance monitoring is the foundation of AI risk management in production. At a minimum, performance monitoring should track output quality metrics: the accuracy of AI decisions compared to human review on a sampled basis, the rate of exceptions and escalations, and the consistency of outputs for similar inputs. An AI system that cannot be monitored should not be in production. Monitoring is not an optional governance enhancement. It is the mechanism by which production risk is managed. Data Drift and Model Drift Data drift occurs when the statistical properties of the data the AI system receives change over time. If a credit model is trained on data from a period of low unemployment and the economic environment changes significantly, the distribution of credit applications will change, potentially degrading model performance. Model drift occurs when the AI model's performance changes over time as the real-world relationship between inputs and outputs evolves. Both types of drift are predictable and detectable if monitoring is in place. Adversarial and Manipulation Risks AI systems can be deliberately manipulated by users who understand how they work and exploit that understanding to produce favourable outputs. In credit contexts, this might mean customers structuring their financial behaviour to optimise their credit score. Managing adversarial risks requires regularly updating AI models to incorporate the most recent behaviour patterns. Incident Response AI governance frameworks must include an AI-specific incident response plan. When an AI system produces evidence of systematic failure, bias, or regulatory non-compliance, the response must be fast, structured, and documented. The plan should cover immediate steps to limit harm, investigation procedures, remediation approach, and notification requirements.
Frequently Asked Questions
What are the five categories of AI risk in production?
Model risk (AI performs differently than in testing), data risk (input data inaccurate, incomplete or manipulated), governance risk (AI operates outside defined parameters without detection), operational risk (AI failure disrupts operations), and regulatory risk (AI creates compliance failures).
What is the difference between data drift and model drift?
Data drift occurs when the statistical properties of the data the AI receives change over time. Model drift occurs when the AI model's performance changes as the real-world relationship between inputs and outputs evolves. Both are predictable and manageable with appropriate monitoring.
What should an AI incident response plan include?
Immediate steps to limit harm, investigation procedures to identify the root cause, remediation approach to address the failure, and notification requirements including any regulatory reporting obligations.
How often should AI model performance be reviewed?
At a minimum: monthly for high-risk AI systems. The review should compare current performance metrics against the baseline established at deployment and examine the distribution of outputs for signs of drift or degradation.
Ready to act on this?
Start with the AI Workforce Blueprint™ — a fixed-price 2-3 week engagement that maps your specific opportunity and produces a board-ready roadmap.
Book a Blueprint Call →