Rewterz Threat Update – ChatGPT Plugins Provided by Third-Party Can Lead to Data Theft and Account Takeovers

Severity

High

Analysis Summary

Cybersecurity experts have found a new attack surface for threat actors to gain unauthorized access to sensitive information, as Open AI ChatGPT has included a concept of Generative AI ecosystem, ChatGPT plugins, which allows the sharing of data and enables communication between Generative AI and third-party service providers.

ChatGPT plugins are the tools designed to operate the large language model (LLM) to access current up-to-date information, do computations, and utilize third-party services. By using such plugins, the user permits ChatGPT to send private information to third-party websites. Depending on the plugin, it also grants those plugins access to private accounts on GitHub, Google Drive, and other platforms.

One of the vulnerabilities found by renowned research labs and cybersecurity firms is that Open Authorization (OAuth) can deceive users into downloading random plugins without initiating installations. As a result, it intercepts and exfiltrates victims’ online data. They also discovered PluginLab vulnerability that threat actors could use as a weapon to launch zero-click account takeover assaults, which would give them access to an organization’s account on third-party websites.

The information was released about ChatGPT’s cross-site scripting (XSS) vulnerabilities that could be used to take over any account and the researchers also showed threat actors can create unique GPTs that can phish users for credentials and gain unauthorized access.

New research showed an LLM side-channel attack that utilizes token length as a covert way to get encrypted responses from the AI assistant over the web. Despite the encryption of packets, the attacker infers sensitive and confidential information by analyzing packet length and timing information shared in private AI assistant conversations.

This attack is achieved by employing a token interference attack, the fundamental concept involves utilizing an LLM provider to intercept real-time chat answers, extract and parse text segments, infer token lengths from network packet headers, and utilize a customized LLM to determine the response. This attack takes place when an AI client is running in streaming mode and if the threat actor can capture the traffic between the client and the AI chat service.

To mitigate the side-channel effectiveness, it is recommended that companies creating AI assistants use random padding to secure the actual length of the token and send tokens in larger groups with complete responses at once rather than token-by-token.

Open AI has also introduced customized variants of ChatGPT which no longer will install new or already existing plugins. The researchers concluded that balancing and integrating security, usability, and performance is a complex task that needs thorough consideration.

Impact

• Unauthorized Access
• Sensitive Data theft
• Credential Theft

Remediation

• Always be suspicious about emails sent by unknown senders and never click on the links/attachments in those emails.
• Implement ongoing phishing awareness training for partners and staff.
• Implement a web application firewall to filter out malicious traffic and protect against common web-based threats. 
• Implement network segmentation to contain and isolate potential threats to limit their impact on critical systems.
• Regularly monitor network activity for any unusual behavior, as this may indicate that a cyberattack is underway.
• Organizations need to stay vigilant and follow best practices for cybersecurity to protect their systems and data from potential threats. This includes regularly updating software and implementing strong access controls and monitoring tools.
• Develop a comprehensive incident response plan to respond effectively in case of a security breach or data leakage.
• Maintain regular backups of critical data and systems to ensure data recovery in case of a security incident.
• Adhere to security best practices, including the principle of least privilege, and ensure that users and applications have only the necessary permissions.
• Establish a robust patch management process to ensure that security patches are evaluated, tested, and applied promptly.
• Conduct security audits and assessments to evaluate the overall security posture of your systems and networks.