How Model Risk Becomes A Major Risk — Knogrow.com

Models in modern business management play a major role. However, with the growing usage of multiple models, risk originated from inappropriate models and its usage has emerged as a major concern for risk managers in the form of Model risk. It is now one of the major risks to be managed in the risk management world. Enhanced dependency on various models has increased exposure to model risk to all counterparties. In the field of risk management study, it is now a major risk category. Let us understand how model risk has transformed into a major risk.

What are different risk types?

In the field of risk management study, two major categories of risks are commonly known.

1. Financial Risks
2. Non-Financial Risks

Financial risks are those where financial losses are visible, originated from financial transactions, and can be estimated. Whereas Non-financial risks may have huge latent loss potential but not visible and difficult to estimate properly. Common financial risks are market risk, credit risk, risks from operation and system failure, and some more. However fraudulent risk, cybersecurity risk, reputation risk, and legal risks are non-financial risk types, where the severity of the losses are much higher and time-varying.

Model risk overlaps with both financial and non-financial risks. Models are extensively used in risk assessment for financial risks which are the sources of model risk. Whereas, on the non-financial risk side, especially in fraud and AML (Anti-money laundering) risk, models are applied in risk detection and assessment. Where model risk is the major concern from risk managers for generating false positive/negative alerts.

What causes model risk?

As suggested by the name, Model risk originates from a model. Let us understand, what are the components of a model. A model starts with certain assumptions, develops on some statistical hypothesis, selects certain variables, establishes a model algorithm, and finally gets implemented through the system. So improper application or assumption in each step is the source of Model risk.

A model starts with an assumption. So, the assumptions must be aligned to the business problem. Models are to be highly customized with the purpose. Hence it is always advisable that the model needs to be specific. Sometimes we may see some generic all-in-one kind of model. Those are suitable to do some trend analysis but should not be used for critical decision-making input. It is always recommended to validate assumptions with business experts before start working further on the development of the model.

Error in statistical hypothesis may create model risk. There are numerous statistical approaches to address different business problems. However, the skill is to select the correct statical approach/hypothesis to solve the business problem that you are facing. If the design of the model not customized with a business problem, that will be the source of model risk. for better understanding, you can refer to my blog “How Analytics has transformed risk management“ at knogrow.com.

How development framework be a source of model risk?

Model algorithm and variable selection are major sources of model risk. Following errors are commonly seen in the case of inappropriate models.

• Sampling bias
• Over or underfitting
• Survivorship bias
• Missing variable bias
• Variable stationarity

Sampling bias — Sampling decision is a critical part of model development. A developer must ensure that the sample should be representative of the population. In most of the statistical models following sampling techniques are used

• Simple random sampling
• Systematic sampling
• Stratified sampling
• Multistage (cluster) sampling

If there is bias in sampling and it deviates from the population obviously model developed using the biased sample will increase model risk.

Over or underfitting — when a model explains all the noises of the dataset than the underlying statistical relationship, it performs very well in in-sample performance however out of sample performance being unsatisfactory is known as an overfitting problem. Overfitted models are unreliable and fragile for practical usage. Sometimes poor assumption and improper variable usage produce underfit models which are also unsuitable, hence both over and underfit models are a source of model risk.

Survivorship bias — when a model sample includes only the success cases then it is categorized as a survivorship biased sample. The most common cases are in the market/traded risk area for hedge funds where the performance of those successful hedge funds is only projected, without showing the actual picture of the industry. So, if an investor took a decision based on the statement that “hedge funds are always beating market index” will be incorrect as the reference decision model has huge model risk.

Variable stationarity — Model built by non-stationary variables will always produce unstable results and decisions taken based on those models will suffer from model risk.

Missing variable bias — Some models suffer from model risk when variable selection and data treatment processes are incorrect. Data availability plays a major role in model development. However, data completion, dropping of variables, and setting the standard of variable acceptance need important scrutiny from experts. Otherwise, the model will underperform due to the lack of crucial variables in the list.

Why models are so important today?

Today’s financial and non-financial markets are more complex and demand more data-driven decisions. The system, software, and complexity of business need a quick and correct response. Manual decision-making is not sufficient today to manage coverage and criticality of that task. Hence automation in the form of models has become important today. Organizations are capturing data at multiple customer touchpoints and deploying models at key decision areas for the following reasons.

Models interact with a lot of value chain at a time — modern models are deployed by IT and are part of a centralized information system. Hence a model can process a lot of information at a time and provide useful input/output to the entire organization’s ecosystem.

Models are performing multiple tasks with accuracy — The model selection is performed based on the accuracy ratio. Hence models are expected to perform with high accuracy, which is free of human error. Organisation prefers model for quick error-free performance. Moreover, models can take a huge workload that is impossible for a human being to perform.

Human aspects in model risk

Though it is expected that model should perform multiple complex tasks in a short time, but by design, models are dependent on human judgments to be created. Model risk originates from some human errors as well. Following are the areas where human error is responsible for model risk.

Selection of poor-quality data — Quality of data is particularly important for the model’s performance. Improper junk data is not permissible. If the modeler ignores data quality at the beginning, the resultant model cannot serve the purpose. Hence will generate model risk.

Improper assumptions — As we have discussed earlier, the model assumption is critical of the model’s performance. Lack of proper assumption leads to an incorrect model outcome.

Error in modelling code — If the developer is not efficient in writing correct and proper code model risk is on the card.

Erroneous implementation — If IT commits an error in implementing the model and the supervisor overlooks that, the Model risk is inevitable.

Incorrect documentation — A well-performing model must be well documented for repeat usage. If the developer does not write the document correctly and ignores the critical parts of the model’s working instruction, the model will be stale once the developer leaves the responsibility.

Inadequate statistics knowledge — Inadequate statistical knowledge will create risk in applying correct hypotheses and fixing bugs during production. It will also create a challenge in interpreting the validation outcome.

Insufficient understanding of the business problem — If the model owner and developer not clear about the business problem, then the selection of the model and its accessories will be erroneous and thus will create Model risk.

How model risks are managed?

As we discussed model risk has become a major source of risk today. Now the industry is venturing ways to manage it. Following are the probable ways to manage model risk within an organization. Further information is available at The evolution of model risk management.

Independent model review — independent model review is an integral part of model development. IMR team takes up model review task once the model primarily developed and produced output. They review the development data, assumptions, hypotheses, and model development codes. So that they can find out discrepancies in each step that we have already mentioned above as venerable areas of model risk.

Documentation — The most important way to manage model risk is proper documentation. A properly drafted document will explain each step of the development process. The document helps to understand the model, its usage, and sections where customization can be made. It also helps in the redeployment of the model. A robust model of today may be an improper model tomorrow as the scenario, historical drift and other business assumptions got changed over time. Hence recalibration of the model is required. Documentation plays a major role in model recalibration and an overall reduction in model risk.

Post-implementation validation and monitoring — As explained, that erroneous implementation of the model increases the model risk to a large extent. Hence post-implementation validation of model performance is a critical step for model risk mitigation. Validation exercise to be performed at regular intervals to calibrate the performance with changing business and economic environment. In most modern organizations post-implementation validation is a must to maintain model health.

Model review committee — Model governance and supervision is another important measure to monitor model performance. Most of the global organizations have dedicated high-powered committees for this purpose. This committee performs a periodical review of models and recommends changes, recalibration, and redevelopment decisions to the model owner. They also take the responsibility of model performance and provides risk acceptance to models for implementation.

External Audit review — Independent Audits by external auditors like big 4 is done to review model risk and further performance issues related to the models. This audit may be pre and post-implementation related, and audit rating and certification provide important insight to higher management and regulators.

Stress testing — Stress testing is a mandatory regulatory exercise for banks and financial institutions to examine the stress-bearing capability of the organization during extreme stress and liquidity crunch. Hypothetical scenarios are created, and models are operated using those scenarios. The performance is evaluated against the peer group performance benchmark to evaluate model sensitivity and responsive power. Model risks are identified through this exercise and the decision on recalibration or redevelopment of models gets decided at the management level.

Through this article, I have tried to explain, one of the critical risk areas of modern risk management. Organizations today are more vulnerable to model risk because of its latent nature.

Please visit my blogs at https://knogrow.com/ and provide comments and your opinion on “How Model Risk Becomes a Major Risk” and do share this article with your friends on Facebook, LinkedIn, and other social media.