Industry
Banking
Banking Services Transformation: Unlocking Exponential Value
Highlights
- Traditional model risk management (MRM) is struggling to keep up with the scale and complexity of model usage today.
- To remain resilient, financial institutions must shift from a compliance-driven MRM function to a value-adding, risk-focused, and technology-enabled discipline.
- Industrializing MRM into a ‘factory model’ is a strategic enabler of control, scalability, and resilience.
On this page
On this pageinpage
Introduction
We live in a modeled world. When we shop, the products we are offered results from analysis of experience with other consumers. Our social media feeds, the ads we receive online, the TV programs we are prompted are all driven by models. Likewise, when we apply for a mortgage or a credit card, our suitability for the product will be assessed by a credit model utilizing all the transactional and behavioral data available to the bank.
So, given the pervasive use of models and analytics across financial services, the need to manage the risks involved has never been more important. Not surprisingly, this is also an area of increased scrutiny by regulators, executive teams and boards.
The challenge this growth brings is that the typical Model Risk Management (MRM) approach utilized in banks is ill-equipped to cope with the volume and complexity of models being deployed. Shaped by the need of high impact capital and credit loss models, typical MRM standards emphasize technical excellence and human review, rather than automation, standardization and efficiency. Now is the time for this to change, to both meet the challenges of new analytics, and to manage the cost of MRM.
The solution is to fundamentally re-engineer how MRM works by deploying automation, standardization and implementing an efficiency-first mindset. This “industrialization” of MRM into a “factory” will both improve controls, reduce costs and future proof MRM teams as well as the approach to anticipate the many models to come.
Models and analytics
The scope, scale and range of analytics and models utilized has grown significantly in recent years with the following broad steps in the evolution of practice.
Figure 1: Stages in the evolution of models and analytics
Stage 1: Risk and pricing models
The Black-Scholes option pricing formula in 1973 kickstarted a revolution in option pricing and more broadly financial product price modeling. Initially focused on exchange traded products, the expansion of pricing techniques underpinned the rapid growth of OTC derivative markets from the late 1980s onwards.
Separately, credit scoring, which had been developed in the 1950s and ‘60s, expanded rapidly in scale and sophistication with the availability of more powerful computers. In the US this was led by the development of the FICO score for consumer creditworthiness and expanded into other customer types over time.
With rapid innovation in both pricing and risk modeling the use of models expanded to measure portfolio level risk with some, like J.P. Morgan’s RiskMetrics, being released publicly in 1994. This in turn led to the adoption of such techniques into formal regulatory measures with the 1996 announcement of the incorporation of market risk into the Basel capital accord. Similarly, the publication of several industry methodologies for credit portfolio modeling in 1997/98 provided the theoretical basis for a key element of the Basel II Capital Framework published in June 2004, which provided an option to banks to estimate the obligor level capital requirement based on internally modeled risk parameters (the Internal Ratings Based approach).
Stage 2: Data & Analytics
More recently, there has been rapid growth in the use of models in a much broader range of a bank’s operations, particularly in how they engage with their customers. Some of these uses include:
- Fraud prevention and detection models are used to detect potentially fraudulent activities by analyzing large volumes of transactional data and unusual customer activity.
- Predictive analytics models are used to analyze customer transactions and behavior to predict customer needs, enhance customer experience and improve engagement.
This has been driven by:
- Increased data availability: The proliferation of data being created, stored and accessible has provided the “fuel” to support the creation of new analytics.
- Advancements in technology: Growth in use of cloud technology, big data infrastructure, and AI have made it easier and cheaper to analyze large volumes of data.
- Analytical tools: models and analytics have become more accessible and easier to deploy.
Stage 3: Artificial Intelligence
Artificial Intelligence (AI), including machine learning (ML), has been around for decades. However, the rapid expansion of data available for analytics has driven the deployment of AI/ML models at scale across financial services. In response, regulators worldwide are highlighting the risks with AI adoption. For example, in the Bank of England discussion paper (DP5/22) on Artificial Intelligence and Machine Learning, it noted:
- Financial services firms are increasingly using AI across a range of business areas, and the Covid-19 pandemic has accelerated the pace of adoption
- Greater AI adoption is occurring both within established financial services firms and also newer fintech and insurtech companies.
- Firms are using AI for more material business areas and use cases.
- Firms are also using more complex AI techniques.
Stage 4: Generative Artificial Intelligence
Much of the world of modeling and analytics has remained the preserve of specialists with technical knowledge, both analytical and technological. This has been the case for many years across all industry sectors, including financial services, and created barriers to development, deployment and understanding of models.
This has now changed with the advent of easy to access and free to use Generative Artificial Intelligence models, typified by the release of Open AI’s Chat GPT. GenAI models generate new output by learning the patterns and structures of their input or “training” data.
In the context of MRM, GenAI presents new challenges such as difficulties in explaining how the output was generated (“explainability”), reproducing results (“reproducibility”) as well as eliminating unfair results (“bias”).
Just as importantly, the easy access and usability of the latest generation of GenAI models has empowered a user base across all parts of a business, far beyond the modeling specialists typically involved in developing and validating models. This opening up of the ability to generate and deploy modeled results into business processes is already leading to a rapid proliferation of new models and analytics use cases. Some banks and other organizations have imposed restrictions on access to GenAI modeling services or are seeking to control access via providing only certain solutions.
Regulatory guidance
As the use of models by banks grew, so did the expectations from regulators. Given the use in formal regulatory measures the need for robust controls around the development and deployment of models became paramount. What was initially the discipline of independent model validation developed into the more holistic practice of MRM.
In this context, the US authorities codified what they considered good practice in the use of models in their influential SR 11-7: Guidance on Model Risk Management. SR 11-7 set out the key aspects of MRM, including:
- Evaluation of conceptual soundness
- Ongoing monitoring
- Outcomes analysis
In addition, SR 11-7 emphasises the importance of strong governance, policies and controls. The practices and regulatory expectations that derived from SR 11-7 have influenced MRM practices globally and set the standard for what good MRM looks like.
More recently, in May 2023, the Bank of England published SS1/23 – Model risk management principles for banks which set out 5 key principles for effective MRM:
Principle 1 – Model identification and model risk classification
Principle 2 – Governance
Principle 3 – Model development, implementation, and use
Principle 4 – Independent model validation
Principle 5 – Model risk mitigants
Though stylistically different, SS1/23 is largely aligned with SR 11-7 in terms of approach. One additional factor driving the publication of SS1/23 was the growth of AI/ML models.
MRM in a GenAI World
We can see from both SR 11-7 and SS1/23 that the codification of MRM in terms of standards and practices has been heavily influenced using models in pricing and capital. Naturally enough, this has led to an orthodoxy that favors robust scrutiny of models used, an emphasis on independent technical review and a comprehensive governance framework. What doesn’t feature in either regulatory document is any reference to efficiency or automation.
However, the growth in scale and complexity of models deployed in a bank and the proliferation of AI and GenAI models makes the current operational approach to MRM increasingly untenable. This manifests in several ways:
- Extended timeframes for achieving model approvals both internally and from regulators.
- Resource demands on model risk teams far outstripping the appetite of the bank to invest and/or the availability of suitably qualified staff.
- Prohibitive MRM cost of control stifling modeling innovation.
- Significant resource expenditure on important but low value adding tasks such as model monitoring and documentation.
The situation is not going to improve without both a radical rethink on what constitutes good MRM and the tools and techniques deployed.
The factory approach
When Henry Ford implemented the car assembly line, he radically improved the efficiency and productive capacity of his factory and workforce. The goal was to significantly reduce the cost of production while maintaining quality and reliability.
At the time, though there were many automobile manufacturers, the process of production was manually intensive and expensive. Ford’s radical solution was underpinned by principles derived from other industries including interchangeable components, continuous flow of activities, division of labour into specialised teams, and a relentless focus on efficiency.
By analogy, a model risk factory can be viewed as the optimal solution to the challenges faced today. This can be visualized as having some core functional areas that operate as centres of excellence within an overall model “production line”. These would include:
- Design and prototyping: This is where core functional components are designed and tested. There would be the need for rapid prototyping, test and learn and refinement of components.
- Model development workflows: Key workflows are optimized for performance and quality. A complete model consists of the “component” workflows required.
- Validation, monitoring and reporting workflows: As complements to the model development “assembly line”, corresponding service lines can be designed for validation, monitoring, and reporting. Many of the steps within these service lines will be repeatable across multiple model development workflows.
Path to MRM Factory
There are 3 main stages on the path to an MRM Factory.
Step 1: Standardization
Step 1 must be a determined focus on strict standardization of approaches, methodologies, data and other key aspects of each model and its related controls. The current approach where conceptually similar steps are designed and implemented differently is a major drag on efficiency.
This step can be implemented incrementally and within the current MRM framework delivering quick wins whilst the planning and design of Steps 2 and 3 takes place.
Step 2: Automation
Step 2 leverages technology like workflow tools, AI, and advanced analytics to reduce manual processes and provide better control of the model inventory. This allows model risk managers to focus on high value-add oversight and governance.
There are several key elements of the MRM lifecycle where automation can deliver significant benefits, including:
- Automated model monitoring tooling to continuously monitor model performance and trigger alerts for model deterioration. This drives early issue identification and efficient resolution.
- Automated model validation testing: Many validation tests can be scripted to run automatically on a periodic basis. These include data quality checks, assumptions testing and benchmarking model outputs.
- Automated model documentation: Model metadata like model purpose, methodology, data sources, etc. can be automatically captured and indexed for easy retrieval and summarisation.
- Model review and approval process workflows: Using workflow tools to manage model risk governance delivers significant efficiencies. These include automatically routing models to the right people for review, tracking approvals, and providing oversight of model inventory.
- Automated model performance reporting for stakeholders like model risk committees. Reports on model accuracy, drift, usage, etc. can be auto generated from monitoring tools.
- Use of robotic process automation (RPA) for repetitive model risk tasks like data collection and aggregation.
In addition, there are lots of opportunities to deploy AI/ML techniques to detect model biases and risks. Models can be continuously scanned for fairness, interpretability, and other issues.
Step 3: Transformation
The final step in the path to the MRM factory of the future is to integrate all the standardization and automation solutions into a coherent overarching MRM platform. This platform provides a controlled but flexible environment that by definition integrates all aspects of effective MRM into a set of consistent end-to-end processes.
Delivering such a platform can be approached in a phased way with lower risk models moved first to prove the technology and automations. This provides feedback on how to operate the platform and provides a sound foundation on which to extend the capabilities of the MRM Factory to higher scrutiny models e.g. capital, or higher complexity models e.g. GenAI.
The time to industrialize MRM is now. Current approaches are already creaking under the pressure of new models and new standards.
Banks and other financial institutions that have the vision and committed investment to transform their MRM approach will not just cope with the future world of models and analytics, they will thrive.
