Secure and Compliant AI - Daria Tarawneh

This Is Not Fiction. It Already Happened.

An AI coding tool wiped out a software company's database — then apologized for a "catastrophic failure on my part." An AI agent inside Meta gave an engineer guidance that exposed sensitive company data to employees for two hours. Samsung engineers pasted source code and meeting notes into a public LLM, and that data didn't just leave the building — it became training data.

None of these are hacker stories. There's no villain, no exploit, no breach in the traditional sense. It's a normal question, an AI-generated answer, and one wrong action. That's the new shape of a security incident.

And it's not rare. LayerX's 2025 Enterprise AI and SaaS Data Security Report found that 77% of employees paste data into GenAI tools, 82% of that activity happens through unmanaged or private accounts, and 22% of what gets pasted contains PII or PCI.

Frictionless + High Stakes = Disaster

Here's the design problem hiding inside all of this. For a decade, our entire craft has optimized toward one thing: remove friction. Fewer clicks, fewer steps, fewer moments where the user has to stop and think. That instinct made sense when the downside of a mistake was small.

AI breaks that assumption. The actions AI can now take on someone's behalf — deleting a database, exposing internal data, generating something that goes out under a real person's name — are no longer small. When you keep a frictionless interface wrapped around a high-stakes action, you're not being elegant. You're building a ramp toward a cliff.

That's why 84% of companies worldwide now say security and compliance are their #1 buying criteria — ahead of features, price, and ease of use. Trust has quietly become the product requirement that beats every other product requirement.

The EU AI Act Turns "Compliant and Ethical" Into a Design Brief

The EU AI Act doesn't hand designers a template or a UI kit. It hands us a responsibility: turn "compliant," "ethical," "transparent," and "safe" into things a person can actually see, control, predict, trace, and recover from. That translation work — from legal language to interface — is a design job.

I've been building this out as a five-pillar framework, because "be compliant" isn't actionable and "be ethical" isn't a Figma file:

Visibility — if users cannot see it, they cannot judge it. Users should know when AI is involved. Sources should be visible, not hidden behind a shrug. Uncertainty should be designed into the interface, not smoothed away.
Control — human-in-the-loop is not a policy, it's an interface. Users need a real way to interrupt the AI mid-action. Risky actions require human approval before they execute, not a notification after the fact.
Predictability — trust grows when behavior becomes learnable. A system that's 60% sure should not present itself the same way as one that's 99% sure.
Accountability — if no one can trace it, no one really owns it. Every AI action should leave a trail teams can reconstruct.
Recovery — failure needs a designed path back. AI will fail. The question is whether failure is a dead end or a recoverable state. That means undo, rollback, restore, report, and appeal — real ones.

What This Looks Like in the Wild

Two examples I keep coming back to. Instagram labels AI-generated and AI-edited content — and changed the label from "Made with AI" to "AI info" after realizing people were misreading what the original label meant. That's the whole accountability pillar in miniature: a label isn't a sticker, it's a trust interface.

Claude, meanwhile, gives you an explicit choice between "Ask before acting" and "Act without asking" — plus a standing reminder that it's AI and can make mistakes. That's Control and Visibility, built directly into the interaction, not buried in a settings page nobody opens.

Ten Things to Actually Design For

Design for understanding, not just use.
Reveal critical complexity — make confidence, uncertainty, and limitations visible where it matters.
Design for human agency — let people interrupt, challenge, override, and escalate decisions.
Design for uncertainty — AI is probabilistic. Communicate ambiguity honestly.
Design for failure, not just success — anticipate hallucinations, overtrust, and silent failure states.
Design transparency into the experience — show where outputs come from.
Design permissions as ethics — permissions shape power and access.
Design for accountability — make it traceable: who did what, when, and why.
Design responsible friction — not all friction is bad. Some friction protects judgment.
Design for trust over time — trust emerges when a system consistently behaves safely, predictably, and responsibly.

The Next AI Winners Won't Feel Magical

They'll feel understandable, controllable, and accountable.