AI agents in banking must earn authority before they act

Banks are moving AI agents from answering queries to performing operational tasks, handing them access to customer data and core systems. That shift brings risks traditional contact centres never faced: misread instructions acted upon, errors cascading across connected systems, and fraudsters exploiting support channels through impersonation and social engineering. In this conversation with CISO Forum, VijayPal Singh, Co-Founder and CEO of Karta AI, explains why understanding a request does not confer authority to act, and why identity must be verified at every turn. He also outlines the authorisation boundaries banks should insist on, how accountability should be settled before deployment, which cases must always reach a human, and what it will take to earn customer trust over the next two years.

VijayPal Singh, Co-Founder and CEO, Karta AI

CISO Forum: Banks are giving AI agents access to customer data and core systems. What new risks do they create that traditional contact centres never faced?

VijayPal Singh: The key difference is the extent to which an AI agent can interact with banking systems. In a traditional contact centre, employees operate within defined processes and can exercise judgment based on the situation. An AI agent, depending on how it is configured, may be able to retrieve information, initiate workflows or carry out actions across systems at a much greater scale and speed.

This creates risks such as incorrect instructions being acted upon, unintended actions being triggered through a misunderstanding, or vulnerabilities in one connected system affecting others. The challenge is not simply to prevent unauthorised access, but also to ensure that an agent interprets a request correctly before acting on it. Banks will need to account for these risks as they move from using AI to answer questions towards using it to perform operational tasks.

CISO Forum: The Prime Minister has flagged the need to protect India’s banking system from cybercriminals. In light of that, how should banks rethink customer support security?

VijayPal Singh: Customer support needs to be considered an important part of a bank’s security environment. It is often the point at which customers report suspicious transactions, seek help with account access, or request changes to their banking services. Fraudsters can exploit these interactions through impersonation and social engineering, particularly when customers are under pressure to act quickly.

Banks should therefore review the security of the entire customer support journey, including the channels through which customers communicate, the information shared during interactions and the processes followed when a customer reports a suspected fraud. They should also ensure that customer-facing teams are equipped to identify common fraud tactics and guide customers towards verified channels. Security should not depend solely on customers recognising a threat; the bank’s support processes must be designed to reduce the opportunity for such threats to succeed.

CISO Forum: Fraud queries rarely sit in one system. So how does an AI agent retrieve information from multiple platforms while keeping the customer’s data secure?

VijayPal Singh: The first requirement is for the agent to work through well-established connections to the bank’s systems rather than relying on information gathered from disconnected sources. When a customer reports a suspicious transaction, for instance, the agent may need to refer to transaction records, card information and the bank’s complaint-handling system to understand the issue. At Karta AI, our Context Engine pulls this information in real time for each customer from sources such as core banking, cards, loans, payments, CRM, and the helpdesk. The underlying banking systems remain the authoritative source for customer and transaction records.

Security has to be built into how the agent retrieves data, not added afterwards. The agent should access account information only after the customer is verified and pull only what the case needs. The bank’s data and PII rules should be built into the agent itself, defining what it can see, what it must mask and what it is allowed to say. Every system call should be role-based and logged. With us, customer data stays in India, personal information is redacted in logs, and we do not train models on customer data.

CISO Forum: What separates L2 and L3 queries from routine ones, and why have banks struggled to automate them?

VijayPal Singh: Routine queries typically have a defined response or involve retrieving a specific piece of information, such as an account balance or loan status. L2 and L3 queries differ because they often involve exceptions, multiple related requests, or actions that depend on information from different systems.

Traditional automation has largely been built to handle individual, predefined tasks. It becomes less effective when a conversation moves beyond a standard query and requires the system to interpret the situation, determine the appropriate next step and complete a sequence of actions. At Karta AI, we are focused on this more complex layer of inbound customer operations.

Our Predictive Context Engine brings together what the bank knows about the customer, across voice, WhatsApp, chat, email, the app, core banking, cards, loans, CRM and past calls, with what the bank requires: its SOPs, processes and compliance rules. Often the agent knows why a customer is calling before they explain, for example because an EMI bounced yesterday or a ticket was raised on Monday. Most bots load context once, at the start of the call. Ours reloads it on every turn. A customer might say their EMI was debited twice, then mention their card was blocked, then ask to move their EMI date. That is three systems and three sets of rules, and the agent can resolve all three in one call without a transfer.

CISO Forum: How does an AI agent tell a genuine customer concern from a suspicious interaction, especially in voice conversations?

VijayPal Singh: An AI agent should not treat the way a person speaks or the details they provide during a conversation as proof of identity. A fraudster may know certain details about a customer or imitate the way a legitimate customer communicates. This is particularly important in voice interactions, where familiarity or confidence can sometimes be mistaken for authenticity.

The customer’s identity must be established through the bank’s approved authentication process. The AI agent should follow the same verification requirements applicable to the specific service or request and should not bypass them based on the conversation. If verification is incomplete, the agent should not disclose protected information or proceed with a sensitive request. The distinction is important: an AI agent can understand what a customer is asking for, but that does not automatically establish that the person making the request is authorised to do so.

Beyond identity, the agent should monitor what is being asked and the order in which it is asked. A request to change a registered mobile number followed by a request to move money, a customer mentioning that someone called asking for an OTP, or unusual urgency around a sensitive change should trigger stricter verification or an immediate handoff to the fraud team. This is why context must be checked on every turn, not just at the start. If a conversation shifts from a routine refund to unblocking a card with a fraud flag, the agent should re-verify before acting, regardless of how smoothly the call has gone so far.

CISO Forum: How do you decide what an AI agent can see and do? And what authorisation boundaries should every bank insist on before deployment?

VijayPal Singh: Banks should begin by mapping the agent’s responsibilities to the actual tasks it is expected to perform. For example, an agent handling loan-related enquiries may need access to repayment information, but that does not mean it should also have the authority to modify a repayment schedule or approve a change to the loan account.

The distinction between viewing information, initiating a process and approving or completing an action needs to be explicit. Each capability should be enabled only where there is a clear business requirement, with additional approval requirements for actions that carry financial or account-related consequences. These boundaries should also be reviewed whenever the agent’s role changes. The objective is to ensure that the agent’s technical capabilities do not exceed the responsibilities assigned to it by the bank.

At Karta AI, these boundaries are configured as the bank’s own compliance and data rules, covering what the agent can and cannot do and what it can see. Our Adherence Engine makes sure the bank’s SOP is followed step by step on every call.

CISO Forum: If an AI agent makes a wrong call on a sensitive account, who is accountable: the bank, the vendor or the model? And how should that be settled upfront?

VijayPal Singh: The responsibility should be defined according to the role each party plays. The bank owns the customer relationship and the banking service being delivered. The technology provider is responsible for the solution it supplies and for meeting the obligations agreed upon with the bank. The model itself is a component of the technology and cannot be held responsible for an outcome on its own.

This division needs to be addressed before the solution goes live. Contracts should specify the scope of the technology provider’s responsibilities, the bank’s oversight requirements, procedures for reporting incidents and the process for investigating errors. It is equally important to establish who has the authority to suspend or modify the agent’s operations if a problem is identified. These arrangements should be clear before deployment, rather than worked out after an incident occurs.

None of this works unless every decision can be traced. Banks should require that, for any conversation, the vendor can show what the agent saw, what it concluded and which action it took. At Karta AI, every step an agent takes is observable and quality-audited, which turns an accountability dispute into a fact-finding exercise.

CISO Forum: Which cases should always go to a human, and how do you ensure the handoff happens quickly enough to prevent losses?

VijayPal Singh: A human should take over when a case requires discretion, negotiation or a decision that falls outside the agent’s defined role. This could include a customer disputing the outcome of a fraud complaint, a complex grievance that remains unresolved, a request involving exceptional circumstances, or a situation in which the customer explicitly asks to speak to a person.

The handoff process should be built into the customer journey rather than treated as an exception that needs to be arranged separately. The agent should be able to identify that it cannot take the matter further and transfer the customer to the appropriate team. For urgent matters, such as a suspected account compromise, the bank’s existing emergency response process should determine the priority and escalation route. Customers should not have to complete an automated interaction before accessing urgent assistance.

Speed also depends on what travels with the handoff. The agent should pass along a summary, the transcript, and the information already gathered so the human expert can act immediately rather than asking the customer to repeat everything.

CISO Forum: Where is the line between automation that speeds up service and automation that creates risk? As the technology matures, how does that line shift?

VijayPal Singh: The suitability of a process for automation should be judged by the consequences of an error, not just by how frequently the task occurs. A high-volume activity may be relatively straightforward to automate if the steps and expected outcomes are clearly defined. On the other hand, even an infrequent request may require human judgment if it involves a disputed decision, an unusual customer circumstance or significant financial consequences.

As AI capabilities improve, banks may be able to automate a wider range of activities. However, each new use case should be assessed on its own merits. Banks should examine how consistently the agent performs, how it handles unexpected inputs and whether errors can be identified and corrected before expanding its role. The objective should not be to maximise the number of automated interactions, but to identify where automation can deliver a dependable outcome for both the customer and the bank.

CISO Forum: Customers are wary after years of rising digital fraud. So what will it take for them to trust an AI agent with their money-related concerns, and what should banks do in the next two years to earn that trust?

VijayPal Singh: Trust will depend on whether customers feel informed and in control of the interaction. Banks should clearly communicate when a customer is speaking to an AI agent and explain the kinds of assistance it can provide. Customers should also have an accessible way to reach a human representative, to question an outcome, or to raise a complaint if the interaction does not address their concern.

Over the next two years, banks should focus on making AI adoption visible and understandable to customers, rather than introducing it only as a behind-the-scenes technology upgrade. This includes explaining how AI is being used in customer service, collecting feedback on customer experiences and addressing issues that arise during deployment. Customers are more likely to accept AI when they can see that it makes banking support easier to access without taking away their ability to seek clarification or assistance from a person.