This isn’t your parents’ software security – Why AI Models are Different

AI software is different from traditional software. In the past, we wrote instructions called code; if our instructions were off, we would encounter a bug. We then adjusted the instructions to fix the bug. However, AI software has a brain called the model, which comprises both stored knowledge and neural pathways to access that knowledge. There is no way to see which pathways were used when using AI software. You cannot “step-by-step debug” an AI model as you could with macros or other instruction-based(code) software.

Protecting Your Information

Your Knowledge is Valuable – Don’t Give it Away.

The value of an AI model is in its knowledge. A model is “trained” on data to answer more questions and be a more powerful intelligence. Models are hungry for more information so they can become more intelligent. Ready sources of information are users’ conversations with the LLM. Just like we learn when we converse with each other, AI learns when we converse with it. When you ask the model a question and converse on a topic, the AI model can remember that it can converse and use it to inform its answer to the next person it talks with. This means your questions and responses to the AI model could be shared with others. Here are a few things to consider:

  • Nothing is free: Models need data to grow their intelligence. Check if your AI provider commits to not using your interaction with the model to train it. Most free AI is being paid for not in currency but with your data. Anything you do not want to share with the entire world on a billboard, you would not want to enter into an AI that incorporates it into its model.
  • How to address Data Protection: The version of Copilot that comes with most Microsoft core subscriptions, such as Microsoft 365 Basic, Standard, Premium, and their Enterprise counterparts, includes data protection, so your data is not used to train the AI model.
  • Sensitive Information: Copilot uses the security measures you already have to protect your internal content from unauthorized access. This means that your existing security protocols, such as restricting access to specific SharePoint libraries, remain in place to keep your data safe.

Accuracy and Validation

No One is Perfect, and Accuracy is not guaranteed.

Like the human brain, an AI model provides the most likely accurate response, but accuracy is not guaranteed. Think of AI as a thought partner, an intern, advisor, or consultant. In those scenarios, you would know to converse with them with the expectation that some of what they say will be wrong or inaccurate due to misunderstanding, ineffective sources of information, bias, etc. If you met that consultant, advisor, or intern for the first time, you would probably reference check their background and inspect what you expect – delegate but don’t abdicate.

How to Address Accuracy

It’s essential to test the validity of your AI model and continue to validate each response. First, for the model itself, look for documentation and reliable sources that support the model’s accuracy. Working with known or trusted vendors whose business model is based on something other than buying and selling data is another good step.

In our daily work, our best defense against AI’s inevitable errors is to use a tool that includes citations. By providing links to the source documents that the AI model is using to derive its answers, you can judge whether or not that source is reliable and evaluate the source itself for capability.

Poisoning an AI Model

Even if the model was created with care and there are excellent citations, a model can be poisoned. Data poisoning happens when threat actors target the training data the model uses. Here are some examples:

  • Malware: Corrupting the model by inserting malware into it. This happened when 100 poisoned models were uploaded to the Hugging Face AI platform. Each one could deliver malicious code onto a user’s machine.
  • Phishing Attacks: For example, having a bot that uses the AI model to provide a phishing link to users rather than the correct link.
  • Inserting Bad Data: A threat actor trains the model to provide inaccurate results or conspiracy theories or to create backdoors into the model for more damage.

Next Steps:

How do we stay safe in an AI world? Opt for paid subscriptions to protect your data, leverage your existing security protocols, understand the AI models you’re using, be aware of model poisoning, and test the validity of your models with citations. If this sounds familiar, our team at TechHouse is happy to help.