Industry
Banking
Banking Services Transformation: Unlocking Exponential Value
Highlights
- Banks and financial institutions have so far used commercial analytics platforms to obtain the business intelligence necessary to make quick decisions and grab new opportunities.
- However, commercial platforms pose challenges that are impeding banks in accessing business intelligence at critical moments.
- Banks must consider moving to an open-source analytics platform to overcome the challenges associated with commercial products and reap the benefits of lower costs and faster access to business intelligence.
On this page
Traditional data analytics platforms may not work now
Banks and financial institutions require business intelligence to capitalize on new opportunities as they emerge.
Business intelligence is also critical for quick, informed decisions aimed at maintaining a lead in the market and delivering delightful experience to their most important stakeholder—customers.
Business analytics had a modest beginning with intelligence reports generated from enterprise data accumulated by financial institutions over a period of time. Banks then graduated to running analytics on information stored in enterprise-wide data-lakes that included big data, external data, and real-time data. The function has now evolved into analyzing humongous data stored in data lakehouses that span the physical and logical boundaries of banks and offer the advantages of both the warehouse and the lake.
Banks have traditionally used commercial big data platforms for their analytics needs that till recently offered favorable licensing terms and conditions, a single point of contact for issue resolution, and a well-defined upgrade plan to meet future needs. However, commercial data analytics platforms now pose several difficulties including:
- Higher investment to buy, install, and operate.
- Need for frequent upgrades to newer versions that come with higher licensing and support costs.
- Delays in urgently needed feature enhancements or patches or support when there are issues related to performance and security.
- Inability of vendors to keep pace with the latest technology advances forcing users to rely on outdated components which means that banks are not able to embrace emerging technologies and access insights in real time.
- Compulsion to use bundled products that come with irrelevant features (bloatware) leading to higher costs.
These drawbacks make it difficult for financial institutions to access business intelligence at critical moments, which adversely impacts their ability to make the right decision at the right time, compromising their competitive edge in the longer run. Overcoming these challenges has become a business imperative and we believe that banks must consider a shift to open-source data analytics platforms to address these issues.
The journey to open-source data analytics platforms
Open-source data analytics platforms offer several advantages to financial institutions.
They include zero license costs, better software security, flexibility to tailor the platform to specific needs, and quick access to innovations that result from collaboration among global open-source contributors. In addition, the fact that there is no vendor lock-in allows banks the freedom to experiment with multiple options available in the open-source community.
Financial institutions, however, have traditionally been reluctant to adopt open-source software due to concerns around privacy and cybersecurity. Open-source platforms can come with cyber vulnerabilities that expand the attack surface. Banks believe that the onus of identifying cyber threats and risks and taking action to fix them lies with them as the open-source community is not accountable to provide such support in a timely manner. This is in contrast to commercial vendors that take up this responsibility.
Additionally, maintenance costs can increase due to higher technical debt when or if the open-source provider decides to abandon the platform. Furthermore, banks will need to spend on building a team to operate and maintain the open-source platform and train their staff in using it.
Another concern with open-source data analytics platforms is the need for constant monitoring and fine-tuning to manage the volume, velocity, and changing nature of data that floods in incessantly. In the absence of such monitoring, performance issues arise—for example, report generation may be delayed, resulting in violation of regulatory compliance timelines.
In addition, banks labor under some erroneous perceptions with regard to open-source data analytics platforms:
Absence of support: On the contrary, members of the open-source community do provide support. Contributors from across the globe collaborate to discuss and fix issues. This is corroborated by Apache Software Foundation’s prompt action in releasing an upgrade within a month of a security engineer in Alibaba discovering the Log4j vulnerability. For immediate troubleshooting, however, banks must establish an in-house team.
Difficulties in identifying the right software version: Using the version of the software that is stable at a given point in time and meets all the bank’s requirements is the right way forward.
Crude user interface and vague documentation: A dedicated community of world-wide contributors collaborate to maintain clear and comprehensive documentation, ensuring it is accurate and up to date at all times. In addition, members with design expertise focus on creating a user-centric, intuitive interface that is functionally efficient.
But the tide is changing—banks are now undergoing a tremendous mindset shift and are more open to adopting open-source platforms for their business analytics needs. This can be attributed to several reasons: from a data analytics perspective, open-source platforms are more modern and have the ability to rationalize heterogeneous ecosystems comprising varied cloud environments, diverse technologies, and analytics models allowing a higher degree of customization to organizational needs. Additionally, banks are now more receptive to building engineering capabilities required to facilitate a shift to open-source and quickly reap the benefits of technology advances and overcome interoperability pain points. Most importantly, a shift to open-source will save banks substantial costs associated with commercial systems without compromising the ability to customize the platform to accommodate new features.
An approach to making the move
Before shifting to open-source data analytics platforms, banks must implement a few preliminary initiatives to address the inherent challenges.
- Get buy-in from the leadership and relevant stakeholders to migrate to an open-source platform.
- Create an in-house team and equip with them with the necessary tools and expertise to manage the open-source data analytics platform. This could include in-house tech-savvy staff as well as new recruits with expertise in open-source platforms.
- Establish a robust security operations practice to oversee and mitigate the risks that arise or are likely to arise from the adoption of open-source data analytics platforms and arrive at the remedial action to be taken.
- Engage with service providers that train resources in the usage of the open-source platforms in a cost-effective manner.
Figure 1 shows the steps banks must traverse in order to successfully migrate from a commercial business analytics platform to an open-source one.
Figure 1: Steps involved in migrating to an open-source data analytics platform
Data: Identify and group related datasets along with related entities such as analytics code, data pipelines, analytics pipelines, orchestrations, and security processes. This helps in executing multiple agile workstreams in parallel, to migrate datasets and their related entities to the open-source platform. In our experience, using automated data comparison and reconciliation tools can reduce the effort required to certify the accuracy of migrated data by almost 60%.
Code: Move analytics code(s) to the target platform by using automated code conversion tools that accelerate the migration process and reduce time-to-deliver—in our experience, this can result in 25% time saves.
Governance: Set up robust control mechanisms to address security and audit concerns, maintain data catalogue and lineage, ensure efficient metadata management, preserve data quality and model standardisation.
Go live: Run both the existing platform and the open-source model concurrently to check for glitches, if any, and optimize, as appropriate. This will help confirm that the new platform meets the bank’s business requirements, service level agreements (SLAs), and standards.
Embracing this approach can facilitate hassle-free migration to the new open-source data analytics platform. Nevertheless, banks must be prepared to address unexpected issues that may arise during and/or after the implementation. In our experience, open-source platforms deliver exponential benefits in the initial phase of the migration. However, unexpected spikes in the volume of data to be processed can slow down the system and impede timely access to critical business intelligence or delay report generation, which can have serious consequences. For example, lack of access to intelligence at the right time may mean a missed business opportunity. Similarly, delay in report submission can erode regulators’ trust in the transparency of the financial institution, lower investor confidence and reputation, disrupt operations, and expose the bank to penalties and lawsuits.
A case in point
The UK division of a Southeast Asian bank with global operations across personal banking, lending, investment banking, loans, and mortgages, was operating with a commercial data analytics platform.
The bank was facing issues in getting prompt support from the vendor for issue resolution and platform upgrades, resulting in delays in regulatory submissions. In addition, the platform was fast becoming unviable as license and support costs were increasing with each upgrade.
The bank therefore decided to shift to Apache Spark, an open-source data analytics platform. This was easy to set up and manage, required lower capital investment, and resulted in quick wins in the form of simpler platform management. In addition, the bank realized the following benefits:
- 75% reduction in the time needed to generate regulatory reports
- 100% adherence with regulatory reporting timelines
- 70% reduction in maintenance and support costs
- Reuse of infrastructure across different lines of business
- Drastic reduction in time taken to run complex analytics
- Efficient utilization of platform resources
Looking ahead
Financial institutions have traditionally been hesitant to use open-source software.
But the industry is now seeing a gradual shift in mindset where banks are more open to adopting open-source platforms. However, success will demand a cultural shift and robust change management strategies.
In our view, open-source data analytics platforms are the future in the banking and financial services industry given their benefits of lower costs, potential for innovation, and the flexibility to tailor solutions to specific requirements. In addition, open-source platforms eliminate the compulsion to adopt bundled solutions from commercial vendors which often come with features that the bank does not need, in turn increasing costs.
The benefits are clear, but the move will not be without pitfalls. We believe that the advantages outweigh the challenges and banks must seriously consider migrating to open-source platforms for their analytics needs. A hassle-free migration may necessitate partnering with a service provider with the requisite domain expertise and implementation experience after a well-rounded market analysis.
