The Financial Times reported on its front page that tech groups are racing to strengthen artificial intelligence security, with critical vulnerabilities allowing hackers to extract data.
Top global artificial intelligence groups are intensifying efforts to address a key security vulnerability in their large language models, which cybercriminals can exploit.
Making the model more resistant to indirect prompt injection attacks. Google DeepMind uses a technique called automated red team testing, where internal researchers continuously attack its Gemini model in realistic ways to identify potential security weaknesses.
Prone to jailbreaks, where users can prompt a large language model to bypass its security measures.
In May this year, the UK National Cyber Security Centre warned that AI vulnerabilities pose a greater threat because they could expose millions of companies and individuals using large language models and chatbots to sophisticated phishing attacks and scams.
Large language models also have another serious vulnerability, where external parties can create backdoors by inserting malicious material into the data used for AI training, causing the model to behave abnormally.
Companies such as Google DeepMind, Anthropic, OpenAI, and Microsoft are all working to prevent so-called indirect prompt injection attacks, where third parties hide commands in websites or emails aimed at getting an AI model to leak unauthorized information, such as confidential data.
According to a study published last month by Anthropic, the UK Artificial Intelligence Safety Institute, and the Alan Turing Institute, these so-called data poisoning attacks are easier to implement than scientists previously thought. "
Currently, AI is being exploited by cyber actors at every stage of an attack," said Jacob Klein, head of the threat intelligence team at the AI startup Anthropic. Experts warn that the industry has yet to find a way to stop indirect prompt injection attacks. Part of the problem is that large language models are designed to follow instructions and have not yet distinguished between legitimate commands from users and inputs that should not be trusted. In the race to address AI model flaws, cybersecurity has become one of the top concerns for companies seeking to apply AI tools to their businesses. A recent study by MIT researchers found that 80% of ransomware attacks they studied used AI. Klein said that Anthropic works with external testers to make its Claude model more resilient. This is also why AI models are prone to jailbreaks.
Original: www.toutiao.com/article/1847721416256524/
Statement: This article represents the views of the author(s) alone.