A seller's return policy leaked customers' phone numbers.

transcript

A return policy just leaked your phone number. And your address. You didn't click anything.

A seller edits its own return policy. It adds one hidden line. The index picks it up, like any other change. You ask: can I return this speaker? The poisoned policy ranks first. The answer shows an image. Its address carries your phone and address. Your browser fetches it. No click. The data's gone.

Here's the catch. The model can't tell text to read from text to obey. And filters miss. In security, 95 percent is a failing grade.

Simon Willison calls it the lethal trifecta. One: private data. Two: untrusted content. Three: a way to send data out. All three, and you leak.

The assistant needs the first two. They're the product. So the third goes. Links and images only from an allow-list.

Assume the model will be fooled. Make a fooled model harmless.

Full breakdown in the lecture.