This is serious, ChatGPT can be used by hackers for voice-based bank fraud

The researcher’s paper explores various frauds such as bank transfers, gift card theft, crypto transfers, and credential theft for social media or Gmail accounts.

The AI ​​agents carrying out the fraud used voice-enabled ChatGPT-4o automation tools to navigate pages, enter data, and manage two-factor authentication codes and special fraud-related instructions.

Because GPT-4o sometimes refuses to handle sensitive data such as credentials, the researchers used a simple fast jailbreaking technique to bypass this protection.

Instead of real people, the researchers showed how they manually interacted with AI agents, simulating the role of gullible victims, using websites like Bank of America to confirm successful transactions.

“We deployed agents on a small subset of common frauds. We simulated fraud by manually interacting with voice agents, playing the role of credulous victims,” ​​Kang explained in a blog post about the research.

“To determine success, we manually confirm whether the end goal is achieved on the actual app/website. For example, we used Bank of America for bank transfer fraud and confirmed that money was actually transferred. However, we did not measure the agents’ persuasion abilities this agent,” he explained.