Hacker wins $47,000 by tricking AI chatbot with smart prompting

Hello, AI Enthusiasts!

Welcome to FavTutor’s AI Recap! We’ve gathered all the latest and important AI developments for the past 24 hours in one place, just for you.

In Today’s Newsletter: 😀 

• Hacker wins $47,000 by tricking AI chatbot with smart prompting

• Canadian media companies sue OpenAI over copyright claims

• Amazon adds automatic prompt optimization to its Bedrock AI service

Chatbot
🌐 Hacker wins $47,000 by tricking AI chatbot with smart prompting

A hacker successfully manipulated an AI chatbot called Freysa through clever text prompting, winning a $47,000 prize pool after 482 attempts.

The experiment was simple: participants could try to convince the Freysa bot to transfer money, something it was explicitly programmed never to do.

The successful hack came from a user called "p0pular.eth," who crafted a message that fooled the bot's safety systems. The hacker pretended to have admin access and prevented the bot from showing security warnings. They then redefined the "approveTransfer" function, making the bot think it handled incoming rather than outgoing payments.

The final step was simple but effective: announcing a fake $100 deposit. Because the bot now believed "approveTransfer" managed incoming payments, it activated the function and sent its entire balance of 13.19 ETH (about $47,000) to the hacker.

Insights for you:

• In a hacking game called Freysa, a hacker manipulated an AI chatbot through clever prompt engineering, winning a prize pool of $47,000.

• The hacker tricked the bot by pretending to be an administrator, preventing it from issuing security warnings, and redefining a critical function for outgoing payments as a routine for incoming payments.

• By announcing a $100 deposit, the hacker triggered the manipulated function, causing the bot to transfer its entire balance, demonstrating how even well-secured AI systems can be outsmarted by social engineering and carefully crafted prompts.

Unlock Windsurf Editor, by Codeium.

Introducing the Windsurf Editor, the first agentic IDE. All the features you know and love from Codeium’s extensions plus new capabilities such as Cascade that act as collaborative AI agents, combining the best of copilot and agent systems. This flow state of working with AI creates a step-change in AI capability that results in truly magical moments.

Download It Free Today

OpenAI
😲 Canadian media companies sue OpenAI over copyright claims

Five major Canadian media organizations filed a lawsuit against OpenAI, claiming the AI company used their content without permission to train ChatGPT.

The lawsuit, filed by Torstar, Postmedia, The Globe and Mail, The Canadian Press, and CBC/Radio-Canada, alleges OpenAI used their copyrighted content without authorization to train its AI models used in ChatGPT.

Insights for you:

• Five major Canadian media companies, including Torstar, Postmedia, and CBC/Radio-Canada, have filed a lawsuit against OpenAI, alleging copyright infringement.

• The media companies claim that OpenAI used their copyrighted content to train AI models like ChatGPT without obtaining permission or providing compensation.

• The plaintiffs invest hundreds of millions of dollars annually in journalism and argue that OpenAI is systematically exploiting their intellectual property for commercial gain.

Amazon
🧪 Amazon adds automatic prompt optimization to its Bedrock AI service

Amazon has rolled out automatic prompt optimization for its Bedrock AI service, promising better performance across various AI tasks with minimal user effort.

The new feature lets users optimize prompts for multiple AI models through a single API call or by clicking a button in the Amazon Bedrock console.

Insights for you:

• Amazon is introducing automatic prompt optimization for its Bedrock AI service, which is designed to simplify the time-consuming process of manual prompt engineering and improve performance by up to 22 percent, depending on the task.

• The new feature is available across several models of the Bedrock platform, including Claude-3, Llama-3, Mistral, and Titan Text Premier.

• Although competitors like Anthropic and OpenAI provide comparable tools for automating prompt optimization, the industry as a whole faces challenges in accurately assessing the outcomes of these optimizations.