LLM02: Sensitive Information Disclosure

In 2025, AI security is a relevant issue. With the landscape changing so rapidly and new risks emerging every day, it is difficult for developers and security teams to stay on top of AI security. 

The OWASP Top 10 Risks for LLM attempts to break down the most prevalent vulnerabilities we are seeing in cyberspace, in order to better understand where the gaps are. In the last post in this series, we explored Prompt Injection, the number one issue on the OWASP list. 

Today, we’ll be talking about another key issue: Sensitive Information Disclosure.

What is Sensitive Information Disclosure?

As the name suggests, Sensitive Information Disclosure stems from information that was not intended to be public becoming available to other parties, including malicious parties. The information in question can include Personally Identifiable Information (PII), health records, financial data, and more.

LLMs may inadvertently expose this sensitive information because of issues such as poor configuration, data leaks, or even other types of attacks including prompt injection to the LLM.

Mitigation Techniques for Sensitive Information Disclosure

There are a variety of strategies that can be used to mitigate the risk of sensitive information disclosure. The OWASP Top 10 for LLM gives us a brief checklist of the most important methods, but these alone may not be enough to prevent the possibility of SID.

• Data sanitization: Data sanitization involves altering the data to make it difficult for attackers to get access to it, or even removing sensitive data altogether, effectively, cleaning it out.

• Input validation: Require and enforce strict formats for inputs, ensuring that the model detects and filters out malicious requests and does not compromise the information.

• Access controls: Limit access using the principle of least privilege, essentially only giving access to those who absolutely need it

• Restrict data sources: In addition to requiring specific formats, also requiring specific data sources helps to limit inputs, further narrowing the chances of sensitive information disclosure.

• Differential Privacy: Apply noises to the data, making it difficult for attackers to reverse-engineer the data points.

• Educate users: Ensure users are up-to-date on best practices and offer regular training on safe LLM usage to stay current.

• Transparency: Be upfront about data usage. Allow users to opt out of having their data used in the training process.

LLM02: Sensitive Information Disclosure is a critical issue for LLMs and a contributing cause of some recent AI breaches. There are many ways an LLM’s sensitive information can be disclosed, whether from poor configuration of the model itself, standard data leaks, and other types of attacks including Prompt Injection. 

When sensitive information is disclosed to bad actors, they can use it for malicious purposes and to launch further attacks. However, there are a variety of steps and measures users can implement to mitigate the risk of an SID, including data sanitization, input validation, access controls and more.

If you’re new to AI security, or struggling to keep up, the OWASP Top 10 for LLM is a great resource on the biggest risks in today’s landscape. If you’re looking for more in-depth information, check out FireTail’s recent report on the State of AI & API Security. We’ll see you next week for the third installment in this blog series on LLM03: Supply Chain.

In the meantime, if you want to see how FireTail can simplify your AI security posture, schedule a demo here, or start trying it out for free, today!