AI and Operational Resilience

Mathieu Hemery
October 2026
•
Mosaic

In an earlier piece we discussed how expanding AI usage in the enterprise introduced the need for relevant governance, to ensure that sufficient control is retained as accountabilities for AI usage remain with the enterprise.

The famous quote from the IBM manual back in 1979 says that “a computer can never be held accountable, therefore a computer must never make a management decision”.

In a recent note from the RBNZ, they highlighted expectations on other sources of AI risks, namely increased cyber risk, third-party and concentration risk. “The core message today is simple: advances in AI are changing the operating environment for financial institutions. Entities are responsible for ensuring that they have the systems and controls in place to manage the risks arising from AI”, RBNZ Assistant Governor Financial Stability Mr McGregor concluded.

How AI Is amplifying cyber risk for financial institutions

From an enterprise perspective, cyber risk is largely about how the organisation’s perimeter is defended. The underlying risk is not new, but AI is amplifying it:

  • Frontier models can now discover vulnerabilities in legacy infrastructure. For example, AI tools were used to identify critical remote code execution vulnerabilities in PHP in October 2026.1
  • With significant improvements in image and voice generation, it is becoming harder to detect a malicious actor using AI to simulate an individual’s voice, appearance and demeanour. Deepfakes are not new, but the tools to create them are increasingly accessible.2

The response to these heightened risks is not new either. It requires enterprises to apply their existing defences rigorously, including:

  • timely patching and maintenance of internet-facing servers, which is more important than ever for protecting data and customer assets
  • robust operational processes, such as identity verification and multi-factor authentication (MFA), to protect against deepfakes and other AI-powered impersonation attacks.

Click to enlarge

Managing third-party and concentration risk in AI systems

Not all of the new risks are AI-augmented cyber-attacks. Some arise from the enterprise’s increasing reliance on third-party AI models and systems, which can change in ways the enterprise does not control. Changes to underlying models, model updates or retraining, advances in model capabilities, or changes to third-party configurations and settings may cause AI-enabled or AI-supported business applications to behave differently or produce inconsistent outputs over time. As a result, the testing and system controls put in place for an AI-enabled business application on ‘day one’ may no longer be relevant or robust enough on ‘day two’.

Change is not always a problem; it is to be expected. Comparing a basic interaction with generative AI today with one from 2023, for example, shows clear progress: responses are of better quality, with fewer discernible patterns of machine-generated text. However, when AI is used for decision-making, automated classification or core enterprise processes, it should be subject to the same fundamental change-management principles as traditional software. If a software component is changed or reconfigured, good practice requires appropriate testing to confirm the application continues to meet business requirements and remains fit for purpose.

The same principle should apply where an underlying AI model, service or configuration changes. Customers and regulators would expect the enterprise, not the third-party provider, to remain accountable for ensuring that its AI-enabled or AI-supported business applications continue to operate as intended, remain fit for purpose and do not expose the enterprise or its customers to undue risk over time.

Concentration risk adds a further challenge. Where several business applications rely on the same third-party model or provider, a single change, outage or withdrawal of service can affect them all at once. Knowing (and documenting) which applications depend on which models, and having a credible fallback where an application supports a critical process or service, is as important as testing each application individually.

Ultimately, resilience against third-party and concentration risk starts with sound change and project management disciplines. Consistent with our views in our previous article, an AI-enabled or AI-supported business application should be implemented to meet a defined business requirement, just as traditional software would be. That requirement should remain the reference point over time: as the underlying AI changes, the enterprise needs to be satisfied that the application continues to meet the requirement for which it was implemented. Because changes to underlying AI models and services can occur rapidly and outside the enterprise’s direct control, this may require more frequent testing and revalidation than traditional software. Assurance obtained at implementation may not remain sufficient over the life of the application.

AI may be novel technology, but many of the disciplines needed to manage it are not. The real challenge for financial institutions is understanding where AI changes the assumptions on which their existing controls and resilience arrangements were built and ensuring that, when it does, accountability remains firmly within the enterprise.

If you would like to discuss how your governance, resilience and control frameworks can better support the management of AI-driven or AI-enhanced risks, please get in touch.

Sources

  1. https://www.bbc.com/news/articles/c3qjnwye0ep5o
  2. https://dailyhodl.com/2026/10/26/anthropic-ai-model-used-fake-identities-to-pressure-real-developers-into-accepting-malware/

AI and Operational Resilience

Published
October 2026
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

In an earlier piece we discussed how expanding AI usage in the enterprise introduced the need for relevant governance, to ensure that sufficient control is retained as accountabilities for AI usage remain with the enterprise.

The famous quote from the IBM manual back in 1979 says that “a computer can never be held accountable, therefore a computer must never make a management decision”.

In a recent note from the RBNZ, they highlighted expectations on other sources of AI risks, namely increased cyber risk, third-party and concentration risk. “The core message today is simple: advances in AI are changing the operating environment for financial institutions. Entities are responsible for ensuring that they have the systems and controls in place to manage the risks arising from AI”, RBNZ Assistant Governor Financial Stability Mr McGregor concluded.

How AI Is amplifying cyber risk for financial institutions

From an enterprise perspective, cyber risk is largely about how the organisation’s perimeter is defended. The underlying risk is not new, but AI is amplifying it:

  • Frontier models can now discover vulnerabilities in legacy infrastructure. For example, AI tools were used to identify critical remote code execution vulnerabilities in PHP in October 2026.1
  • With significant improvements in image and voice generation, it is becoming harder to detect a malicious actor using AI to simulate an individual’s voice, appearance and demeanour. Deepfakes are not new, but the tools to create them are increasingly accessible.2

The response to these heightened risks is not new either. It requires enterprises to apply their existing defences rigorously, including:

  • timely patching and maintenance of internet-facing servers, which is more important than ever for protecting data and customer assets
  • robust operational processes, such as identity verification and multi-factor authentication (MFA), to protect against deepfakes and other AI-powered impersonation attacks.
Contributors
No items found.