LLM03: Supply Chain Vulnerabilities
The hidden risks in the AI tools, models, and data your teams are plugging into your systems.
In December 2020, the world learned that attackers had compromised SolarWinds — a widely used IT management software — and used it to deliver malware to 18,000 organizations, including US government agencies. The victims had done nothing wrong. They were using legitimate, trusted software. The attack happened upstream, before the software ever reached them. This is the essence of a supply chain attack: the threat enters your environment through something you trust.
The AI ecosystem has created an entirely new supply chain — and it is largely unsecured. When your organization adopts an AI tool, you are not just trusting the AI vendor. You are trusting the dataset the model was trained on, the third-party plugins the platform uses, the open-source libraries in the software stack, the vector database storing your documents, and the cloud infrastructure running it all. Each link in this chain is a trust assumption. Any one of them can be compromised — before the product ever reaches you.
The AI Supply Chain: Every Link Is a Risk
Think of an AI deployment like a building. The AI model is the structure — but it sits on a foundation of training data, is connected by pipes (APIs and plugins), is wired with electrical components (SDKs and libraries), and stores its contents in rooms (vector databases and document stores). A compromised foundation, a faulty pipe, or a tampered wire can undermine the entire structure — regardless of how solid the main structure appears.
Open-source model repositories like Hugging Face host tens of thousands of AI models — including many that have been uploaded by anonymous contributors with unknown intentions. A model with a convincing name and a few positive reviews can attract thousands of enterprise downloads. Security researchers have found models on public repositories that were "fine-tuned" in ways that introduce backdoor behaviors — hidden instructions that trigger under specific conditions, invisibly to users and administrators.
Unlike traditional software supply chain attacks, AI supply chain attacks may not be detectable through code scanning. A poisoned model looks identical to a clean model in its file structure — the threat is in the billions of parameters that encode its behavior, which cannot be read or audited like source code.
Click the card to reveal the explanation
A Sensible Cost-Cutting Decision
The IT team identifies a highly-rated open-source AI model that costs 60% less than the current vendor. It has good benchmarks, positive user reviews on the repository, and an active community. Leadership approves the switch.
The new model was fine-tuned on public datasets that contained poisoned data. It subtly misclassifies certain legal documents, causing your legal team's AI assistant to provide incorrect analysis on a specific contract type. This isn't random — an attacker intentionally included the poisoned data knowing companies would use this model for legal work.
- The attack happened before you bought the model — your vendor assessment process cannot catch it
- Standard security testing (vulnerability scans, penetration tests) does not evaluate behavioral manipulation
- Supply chain attacks are rated High because they're hard to detect and can affect many organizations simultaneously
Mapping Your AI Supply Chain
1 / 5🏭 Model Source and Provenance
Where did the base model come from? Who trained it, on what data, with what objectives? Commercial models from reputable providers offer more traceability than open-source alternatives, but neither is inherently safe. Demand model cards and training data documentation.
📊 Training Data Provenance
What data was the model trained on — and who had write access to that data? Public web crawls include content from adversarially controlled websites. Fine-tuning on internal documents means those documents' integrity is now a security concern.
🔌 Plugins and Integrations
Every plugin added to an AI platform extends the attack surface. A plugin that connects to email, calendar, or CRM systems introduces both its own vulnerabilities and potentially its developer's access to your data flows. Vet each integration as you would a new vendor.
📦 SDK and Library Dependencies
The software frameworks used to build and deploy AI applications carry their own dependency trees. A compromised Python library used by your AI application could give attackers access to the AI's inputs, outputs, and actions — a traditional software supply chain attack disguised as an AI product.
☁️ Deployment Infrastructure
Where does the model run? Who has access to the compute environment? Cloud AI services introduce shared infrastructure risks. On-premises deployments shift responsibility to your team. Either way, the infrastructure layer must be hardened and monitored.
Why supply chain attacks are attractive to sophisticated adversaries:
- One compromised model or plugin reaches thousands of enterprises — a massively leveraged attack
- The attack happens before deployment, when victims have no visibility or monitoring
- Behavioral manipulation is nearly impossible to detect without specialized AI security tooling
- Nation-state actors and organized criminal groups specifically target AI supply chains to gain persistent access to enterprise environments
- The AI ecosystem's rapid growth means security practices have not kept pace — there are many soft targets
Supply Chain Defense Strategies
Defense requires applying proven software supply chain practices to the novel AI context:
- Vendor security assessments: Treat AI model providers like any other critical vendor — require SOC 2 reports, penetration test results, and training data documentation
- Prefer commercial over anonymous open-source: Open-source AI models from anonymous contributors offer no accountability. If cost is the driver, consider the risk-adjusted cost of a supply chain compromise
- AI SBOMs: Software Bill of Materials for AI — demand documentation of what data and components went into a model, just as you would demand a software component inventory
- Sandbox integrations: New plugins and integrations should be deployed in isolated environments before production, with behavioral monitoring to detect anomalies
- Behavioral regression testing: Establish behavioral baselines for AI systems and test for unexpected changes after any update or vendor change
Executive Action Items
1. Create an AI Vendor Register
Know every AI model, platform, plugin, and service your organization uses — including tools adopted by individual teams without formal IT approval (shadow AI). You cannot secure what you cannot see. Mandate a formal register and make self-reporting the path of least resistance.
2. Apply Vendor Security Standards to AI Providers
If your organization requires vendors to meet security standards (ISO 27001, SOC 2, penetration testing), apply those same requirements to AI model providers. Add specific AI requirements: training data documentation, model card publication, incident notification for behavioral anomalies.
3. Prohibit Unapproved Open-Source Model Downloads
Establish a policy that AI models may only be sourced from approved providers or pass a security review before use. This does not mean no open-source AI — it means open-source AI goes through the same vetting process as any other third-party software component.
4. Treat Every Plugin as a New Vendor
AI platform plugins often receive the same permissions as the platform itself — including access to connected data sources. Each plugin should be treated as a new vendor relationship: assessed for security posture, granted minimum necessary permissions, and monitored for anomalous behavior.
5. Establish Behavioral Baselines and Monitor for Drift
Define what "normal" AI behavior looks like for your deployed systems — typical outputs, data access patterns, error rates. After any vendor update or model change, run behavioral comparison tests. Unexpected changes in output patterns can signal supply chain compromise.
Board-level perspective: Your organization's AI security posture is only as strong as its weakest supplier. The SolarWinds attack took years to discover. AI supply chain attacks may be even harder to detect. Supply chain risk management must be a first-class concern in every AI procurement decision.