AI Solution Architect Decisions That Survive Security Review
Security review killed a pilot I was proud of for one boring reason: the model path could reach a write API with a service identity that was “temporary.” Temporary lasted six weeks. The demo still worked. The architecture did not.
That is the gap I care about when people ask what an AI solution architect actually decides. Not which logo sits on the slide. Which identities, data paths, tool permissions, and proof of control survive a security, privacy, or risk review without a panic rewrite. This post is for engineers building the seams and for engineering managers or business owners who need a plain scoreboard before go-live.
The short version
An AI solution architect owns the shape of the system under real constraints: identity, data classification, network path, tool blast radius, observability, and a kill switch someone can operate at 2 a.m. Model choice is a detail inside that shape — important, rarely the first blocker.
If security cannot answer “who acts, on what data, with what proof, and how do we stop it,” you do not have a production design. You have a chat UI with a cost centre.
I treat the control plane as non-negotiable. Fancy prompts do not replace it.
Who this is for (two audiences, one scoreboard)
| Role | What you should care about | What “good” looks like |
|---|---|---|
| Business / risk | Customer impact, regulatory exposure, who is accountable when the agent is wrong | Named owner; clear “no auto” zones; residual risk accepted in writing |
| Engineering manager | Review readiness, scope of v1, on-call, cost envelope | Narrow intents; dashboard; kill switch tested, not theoretical |
| Engineers | Identity per hop, tool schemas, traces, data egress | Every action is attributable and replayable from logs |
Same system. Different questions. One shared checklist beats three separate decks.
What an AI solution architect decides under security pressure
Role blur is real. Teams use “AI architect,” “solutions architect,” and “platform engineer” interchangeably. In practice I draw the line like this:
| Decision | Why security cares | Default I defend |
|---|---|---|
| Identity model | Who is the principal on each tool call? | User-delegated or short-lived workload identity — never a shared god key in the agent host |
| Data path | Where prompts, embeddings, and logs land | Classified stores; retention caps; no “debug forever” copies of PII |
| Tool policy | What the model is allowed to invoke | Allow-list by intent; high-impact writes gated or dual-controlled |
| Network edge | Public model APIs vs private endpoints | Private link / egress control where the org already requires it for SaaS |
| Evidence | Can we prove what happened? | Trace IDs on plan → tool → result; store enough to audit, not enough to hoard secrets |
| Stop conditions | How we contain a bad loop | Rate limits, max steps, kill switch, human escalation paths |
If you want the role framing without the security deep dive, I wrote that separately as what an AI architect actually owns in Sydney enterprises. This piece is the review packet side of the same job.
Decisions that usually fail security review
These are patterns I have had to unwind more than once. None of them are exotic.
- One service principal for “the AI app.” It can read tickets, write CRM, and call refunds. Reviewers correctly treat that as a lateral-movement gift.
- RAG over everything, filters later. If the index already holds restricted docs, “the prompt says not to reveal them” is not a control.
- Tools that return raw secrets “just for the model.” Anything the model can see can leak via prompt injection or careless logging.
- Logging full prompts and tool payloads forever. Great for debugging week one; ugly under retention and privacy review month three.
- Shadow mode never instrumented. “We will watch it” without comparable human baselines is theatre. I covered the production scoreboard in evaluating AI agents in production.
The fix is almost always boring: split identities by capability, shrink the retrieval corpus, put policy before the side effect, and define what “good enough to go live” means in numbers.
A review packet I actually bring
When I walk into a security or architecture board as the AI solution architect on the design, I bring one short packet — not a 40-page novel.
| Packet item | One-sentence test |
|---|---|
| System context diagram | Can a stranger mark trust boundaries in under five minutes? |
| Data classification table | What leaves the org boundary (prompts, embeddings, vendor logs)? |
| Identity & auth matrix | Per hop: human, agent host, model provider, each tool |
| Tool catalogue | Read vs write; irreversible vs reversible; who approves live mode |
| Threat notes | Prompt injection → data exfil; confused deputy; poisoned retrieval |
| Ops controls | Kill switch owner, rate limits, on-call, rollback story |
| Eval gates | What metrics must be green before write tools open |
For agent and tool gateways specifically, the same instincts show up in how I design an enterprise MCP gateway in .NET: central policy, attributable calls, no silent superuser path.
Trade-offs I document before build
Security review is not only “blockers.” It is choosing which risks you accept. I write those down so product and risk are not surprised later.
| Choice | You gain | You pay |
|---|---|---|
| Hosted frontier model vs private deploy | Speed, quality ceiling | Data processing agreements, egress design, vendor dependency |
| Broad RAG corpus vs narrow curated sets | Coverage | Harder ACL fidelity; more injection surface |
| Autonomous writes vs human confirm | Throughput | Higher blast radius; stronger eval burden |
| Shared multi-tenant agent vs tenant-isolated stacks | Lower ops cost | Stricter isolation proof for regulated tenants |
I do not pretend there is one correct column. I pretend we are adults about the column we picked.
Go / no-go for “security will not hate this”
Five gates. Fail one → fix root cause → re-check. No multi-week ceremony required.
- Principal clarity — every tool call maps to a real identity with least privilege.
- Data map signed off — classification, retention, and vendor processing are explicit.
- High-impact actions gated — payments, access changes, bulk comms are not free-fire.
- Observable loop — traces show plan, tools, results; unsupported claims are measurable.
- Stoppable system — kill switch and rate limits tested in a drill, not only in a wiki.
Pass those and you can argue residual model risk like adults. Fail them and the model brand on the architecture diagram is a distraction.
FAQ
What does an AI solution architect do that a normal solution architect does not?
Same integration craft — plus non-deterministic components, retrieval corpora, tool-calling agents, and evaluation gates. The extra work is making probabilistic behaviour fit enterprise controls without pretending the model is a trusted microservice.
Is “AI solution architect” different from “AI architect”?
In hiring language, often not. In delivery, I use “solution” when the job is a concrete system boundary (this product, these systems of record). “AI architect” sometimes means platform standards across many products. Both still need identity, data, and blast-radius decisions.
How early should security join?
Before the first production data path. After a public demo with customer content is the expensive version of the same conversation.
Do we need an AI gateway?
Not always as a product. You need a control point: auth, policy, logging, and budgets in one place. That can be an API gateway plus policy service, a dedicated LLM gateway, or a locked-down MCP edge. Scatter those controls across five repos and review becomes archaeology.
What is the fastest way to fail a review?
A shared superuser identity, unrestricted tools, and “we will add logging later.”
Closing
The AI solution architect job, when it is real, is not picking a model family under fluorescent lights. It is designing a system security can reason about: who acts, what they may touch, what evidence we keep, and how we stop a bad loop. Do that first and the interesting model work has somewhere safe to live. Skip it and every pilot eventually meets the same review wall — usually the week someone wants to put it on a real customer path.
