Deepfakes have broken the trust signal; redesigning authentication and approvals when voice and video can be faked.
Beyond Seeing and Hearing: The New Reality of Digital Trust. For generations, we have relied on sight and sound as primary factors of trust. Seeing someone’s face or hearing a familiar voice was often enough to establish authenticity (remember those landline phones…). This assumption largely held until the emergence of deepfake in 2017, when we actually saw an instance of how the face of a popular celebrity got misused and superimposed on a pornographic video. That’s where deepfake technology evolved, enabling the creation of manipulated content using deep learning techniques. Since then, the rapid advancement of AI has created a world in which faces, voices, gestures, emotions, and even live interactions can be convincingly fabricated. Such AI-generated content can impersonate officials or military leaders, causing confusion, disrupting decision-making, and creating national security risks. What was once seen as a risk for celebrities and politicians has entered the corporate world with conviction and use, thanks to cloud-based AI tools, which have lowered the cost of producing high-quality deepfake content. According to reports, the proliferation is staggering, with projections that over 900,000 will be generated monthly by 2026. As these techniques become more available, the risk of harm will increase among politically, socially, and economically vulnerable actors, leading to significant financial fraud and disinformation campaigns operated at scale by well-funded, state-sponsored threat actors.
Evolution of Deepfake: A journey from entertainment to a sophisticated fraud engine:
- Phase I: Experimentation (Pre-2017): The initial applications of deepfakes were largely experimental and academic, not particularly harmful to celebrities or corporate bodies. Researchers and developers explored the potential of GANs to create realistic images and videos.
- Phase II: Manipulation at Scale (2017-2019): This is where the evolution of technologies like face-swapping and technology misuse began, with a primary focus on celebrity impersonation and entertainment videos. Open-source tools like FaceSwap Pipeline or DeepFaceLab became widely known. However, the artefacts were obvious, making detection easy.
- Phase III: Synthetic Identity (2020-2022): The underlying technology moved beyond face swapping. This is where we started seeing lip-syncing synthesis. Now, voice cloning systems could be trained on samples of a person’s speech and then used to generate new audio in that person’s voice, saying words they never said, extending synthetic media concerns from visual content into audio as well. This is where the barrier to entry collapsed, and anyone with moderate laptop skills can actually create a convincing impersonation.
- Phase IV: Attack Weaponisation (2023-present): This is where cybercriminals began exploiting the technology as a social engineering technique. Instead of sending phishing or spear-phishing emails, attackers began engaging in financial fraud recruitment scams and conducting interactive conversations about mergers,itions, etc.
What does this mean for Information Security? A deepfake is not a new category of attack but rather represents the evolution of social engineering, which not only targets what people read but also what they see, hear, and believe. We keep hearing many examples, such as a notable case where an employee in Hong Kong made a US$25 million transaction during what appeared to be a legitimate video call with a senior executive. The attacker didn’t breach the system or credentials, but the trust. Every organisation spends a considerable amount of budget on ensuring “the user is the same he/she claiming to be”, but with deepfakes, there is a new challenge: “Is this a real interaction?” Let’s look at some of the possible scenarios for the body corporates:
- Executive Impersonation: Many of us have heard of, or even experienced, cases where fake CEO video calls or cloned voices are used to make urgent payment requests, disclose confidential information, or even attempt to override existing approval flows.
- Financial Frauds: Some prominent examples include fake acquisition deals, account number changes for transfers,nsfers, etc.
- Corporate Sabotage: Use of deepfake technology to spread misinformation about company products, executives or overall brand.
- Unauthorised access: Exploiting voice authentication to gain access to financial accounts.
The Deepfake Lifecycle and Defence Opportunities: For a successful deepfake attack, the fundamentals of the kill chain remain similar to those of a cyber-attack, and every stage of the lifecycle also provides an opportunity for intervention. The lifecycle typically starts with the collection of publicly available images, videos, and voice samples, progresses through AI-driven content generation, and culminates in the distribution of convincing synthetic media via channels such as email, WhatsApp, and social media, to monetise the fraud. With a layered defence strategy, organisations can disrupt this lifecycle. Limiting executive exposure, monitoring for impersonation attempts, strengthening executive identity protection, using digital signatures, and defining escalation and crisis management procedures can be highly effective techniques for risk mitigation.
Why are Traditional Authentication Failing?
Our authentication systems have evolved from a simple user ID/password combination to multi-factor authentication, where we challenge the user on two of the three factors,, i.e., something the user knows, something the user has, and a biometric factor. Now, deepfakes have directly challenged the third piece with fake voice recognition, facial verification, and video presence. Accordingly, our security models also need to evolve. Rather than just appearance, our trust should be established through verifiable evidence. Let’s look at it in more detail.
- Strong identity protection: Organisations need controls such as phishing-resistant MFA using passkeys or hardware security keys. Enrolling the user in the official authenticator app is an important control. These controls shall also be applied to use cases such as account recovery, password reset, or even device replacement.
- Linking device and session trust: Continuous session evaluation with conditional access becomes an important aspect. Adopting risk-based authentication when privilege or risk changes, e.g., when a user tries to log in from an unfamiliar location. Having device-based certificates and posture checks becomes important in an era when our digital ecosystem is accessed from across the globe.
- Trusted Communication: Protection of corporate emails via SPF, DKIM, DMARC, BIMI with a clear notification for incoming emails either in the subject line or the email body where employees can easily identify the communication is originating from outside the corporate network.
- Make Approvals System-Driven: Financial approvals should occur through governed systems with defined approval levels, rather than through informal communication.
- Four-Eye Principle: Critical activities like vendor creation, change in bank account, high-value fund transfer, and sensitive data extraction-type scenarios shall be designed with multi-level verifications and SoD’s.
- Intent and Transaction Binding: Any change in financial transactions shall require explicit confirmation of beneficiary, amount and purpose.
- Build Deepfake-Aware Culture: Organisations shall have a platform to make users aware of the evolving tactics and techniques of fraudsters.
- Monitoring & Response: Unusual combinations of identity, device, session, and transaction signals need to be monitored and flagged by the SOC team. Based on these alerts, organisations shall have the capabilities for rapid account suspension, session revocation, and payment hold.
- Risk assessment of synthetic media: Based on the risk quantification, organisations may take a further call on investing in technologies for detecting possible face swaps, voice cloning, injected video, user liveness tests, etc.
Final Thought: As AI continues to evolve and blur the line between genuine and synthetic interactions, organisations must shift from traditional perception-based trust to evidence-based trust. The future of authentication needs cryptographic verification, governance and continuous validation.
Authored by Ranjan Revandkar, Chief Information Security Officer, PI Industries
