Third-Party and Embedded AI Risk: The Risk You Do Not Own

I sat in a governance review last month where the CISO opened a spreadsheet of every AI system in the company. Twelve rows. Neat owners, neat tiers, neat controls. Then someone from sales said, almost as an aside, that the CRM had just rolled out an auto-summary feature on every customer note. The room went quiet. Nobody had asked for it. Nobody had reviewed it. It had been processing the company’s most sensitive pipeline data for six weeks.

That was the moment everyone realised the inventory was a fiction. The twelve models the team had built were not the AI footprint. The AI footprint was every SaaS contract on the books, and most of those vendors had quietly become AI companies in the last eighteen months. The risk you do not own is, by 2026, the majority of the risk you carry.

Why the perimeter moved

For a long time the third-party risk conversation was about data: where it goes, who processes it, how it comes back. That was a recognisable shape. You wrote a DPA, you asked about subprocessors, you checked the certifications, you moved on. The shape worked because the vendor was running deterministic software on your data.

That assumption has quietly broken. The same vendor is now running a model on your data, and that model was trained on someone else’s data, and is monitored by a team you have never met, and is updated on a release cadence you do not control. The contract you signed in 2023 did not anticipate any of that. The control you put in place was a perimeter; the perimeter just moved inside the building.

The companion piece to this article on model risk covers the models you build. This one is about the much larger surface: the models you bought without realising it.

The three patterns I keep walking into

There are only three. They show up in different combinations, but the shapes do not change.

SaaS feature creep. The vendor adds an AI feature in a quarterly release. The CRM ships a smart-summary capability; the support platform ships auto-triage; the HR system ships a screening score; the design tool ships a generative assistant. You did not buy the feature, you cannot turn it off cleanly, and the release notes mentioned it on page four. Your data is now passing through a model in a jurisdiction you did not approve, and the contract you signed two years ago neither permits nor forbids it because nobody was thinking about this yet. Procurement does not catch it because procurement is not in the release-notes loop.

Embedded models you cannot audit. A piece of infrastructure (a fraud screen, a content moderator, an anomaly detector, a resume parser, a credit-decision engine) runs a model the vendor will not document. They will tell you it works. They will not tell you how it was trained, on what data, with what bias controls, or how it is monitored in production. The EU AI Act and most sector rules place obligations on the deployer, not the developer. The vendor sells you a black box; the regulator hands you the bill.

Open-source weights pulled from a hub. An engineer downloads a model from a public registry, points it at production data, and ships. The model card on the hub claims one thing; the weights may have been tampered with, fine-tuned in ways the README does not mention, or trained on data the original authors never disclosed. Provenance matters here in the same way it matters for any open-source dependency: if you do not pin the version and verify the checksum, you do not actually know what you deployed. This is also where the broader AI supply chain security story bleeds into model risk.

Where the three patterns meet (one picture)

Three different doors, the same room. The control that matches is not technical, it is contractual and inventory-shaped. You cannot patch what you did not write. You can demand to know it exists, demand the right to refuse it, and keep a list of every model that touches your data, no matter whose servers it runs on.

The procurement clause that earns its place

Most of the leverage you will ever have over a vendor is at renewal. After that the contract is locked, the integration is paid for, and your negotiating position is “we love you, please be careful.” Use the renewal window. Add a short AI clause to every standard contract template and make it non-negotiable, the way data-processing clauses became non-negotiable after GDPR. Four points are enough:

• Notification of new AI features, at least sixty days before they go live in your tenant. No silent rollouts; if the vendor adds a model that processes your data, you hear about it before it ships.
• Opt-out by default for AI features that process customer or personal data. The feature can be there; it stays off in your tenant unless you turn it on after review.
• Data-use commitments in writing. Your data is not used to train the vendor’s models, not shared with subprocessors you have not approved, not retained past the service window. If they cannot agree, you know exactly what you are signing.
• Disclosure of AI subprocessors, including the model providers behind the scenes. If the vendor runs your data through a third-party foundation model, that provider goes on the subprocessor list with the same DPA obligations as any other.

I have watched procurement teams add this clause and discover, in the first round of renewals, that three of their top ten vendors could not commit to any of it. That is not a failed negotiation. That is the most valuable signal you will get all year about which vendors take this seriously.

AI Bill of Materials: the inventory that survives the audit

The software world adopted the Software Bill of Materials (SBoM) after Log4j made the cost of not knowing what was in your stack obvious. The AI world is heading to the same place, faster, and the same logic applies. An AI Bill of Materials is a per-system record of every model that touches your data: the base model, the fine-tunes on top, the training-data lineage at whatever depth the vendor will share, the version pin, the host, the jurisdiction, the named human accountable for it.

Treat the AI BoM as a living document, owned by one person, updated when a vendor ships a new feature or you swap a model. Pair it with the inventory work in the parent hub on AI risk management and the discovery practice in Shadow AI: same muscle, applied to a different surface. The NIST third-party supply chain guidance and ENISA’s AI cybersecurity work both point in this direction; the standards are catching up to what operators are already doing.

When the regulator asks “show me every AI system processing personal data in your environment,” the company with an AI BoM hands over one PDF. The company without one starts a six-week exercise and discovers, in week three, that the answer is much worse than they thought.

The honest leadership framing

Third-party and embedded AI risk is not a vendor-management problem. It is a structural recognition that most of your AI no longer comes from your AI program. It comes from your procurement function and your engineering team’s import statements. The work is to put governance on the doors those models walk through, not on the models themselves. You cannot audit what is in the vendor’s data centre. You can absolutely choose to stop signing contracts that leave you holding their liability.