How to Make Sure Your Software Is Protected Against AI Security Threats

The previous piece laid out how AI is opening new holes in banking cores, CRM platforms, and ERP suites — prompt injection, over-permissioned agents, shadow AI, and poisoned supply chains. This one answers the obvious follow-up: what do you actually do about it?

The short version is that you cannot "buy AI security" as a single product, and you cannot treat it as a one-time audit. Protecting software against AI threats means layering governance, engineering controls, and continuous testing on top of the systems you already run — and treating every AI agent as a privileged identity rather than a convenient feature. Here's how to build that, step by step, grounded in the frameworks the industry has converged on in 2026.

Start With a Framework, Not a Tool

The mistake most teams make is buying a firewall or a scanner first and figuring out governance later. Do it the other way around. The industry has settled on a stack of complementary frameworks, and each one answers a different question:

NIST AI Risk Management Framework (AI RMF) tells you what to govern. It organizes AI risk into four functions — Govern, Map, Measure, Manage — and has become the de facto enterprise standard, with over 70% of US federal agencies and a growing share of Fortune 500 companies formally aligned to it. Its companion, NIST AI 600-1, adds 200+ concrete risk-management actions for generative AI.

OWASP Top 10 for LLM Applications (2026 edition) tells you what to defend against at the engineering level — prompt injection, sensitive data disclosure, insecure output handling, data poisoning, supply-chain compromise, and excessive agency. It's the technical vocabulary that turns governance policy into code.

OWASP Top 10 for Agentic Applications (2026) extends that list to autonomous agents specifically, covering goal hijacking, insecure tool use, excessive privilege, agent-to-agent trust violations, and cascading multi-agent failure — the risks that don't exist until your AI can take actions on its own.

MITRE ATLAS tells you how attacks actually happen, giving your red team a threat-modeling map of real AI attack techniques, in the same way ATT&CK does for traditional security.

ISO/IEC 42001 gives you external, certifiable proof that your AI management system meets a recognized standard — increasingly what auditors, regulators, and enterprise buyers ask for.

The key insight: NIST AI RMF describes what accountability structures should exist, but it does not tell you how to stop a prompt injection during execution. Governance frameworks and technical controls are not interchangeable — you need both. A team that treats NIST compliance as "done" will have excellent policies and zero enforcement when an agent is actually attacked. Stack the frameworks; don't pick one.

The Technical Controls That Actually Matter

Frameworks point you at the risks. These are the concrete engineering controls that close them, organized by the threat they address.

1. Enforce a hard boundary between instructions and data

Prompt injection — direct and indirect — remains the number one AI application risk, and it's the root cause behind almost every CRM and ERP agent breach of the past year. The fix is architectural, not cosmetic:

Validate and sanitize every input an agent reads, and treat anything coming from a form field, email, uploaded document, database record, or external website as untrusted data — never as a command.

Define explicit context boundaries so the model cannot be talked out of its role by text embedded in the data it's processing.

Filter and constrain outputs, especially any output that can trigger an action, call a URL, or reach another system. The Salesforce Agentforce fix, for example, came down to preventing agent output from being sent to untrusted URLs.

Red-team continuously for injection. Prompt-injection defenses degrade; patched systems have been shown to leak again through a slightly different path. Assume this control needs ongoing testing, not a one-time sign-off.

2. Give agents the least privilege — and the least agency — possible

Most organizations already struggle with over-permissioned human users. AI agents inherit those same excessive permissions and exercise them at machine speed, so an over-permissioned agent turns "will sensitive data leak?" into "how fast?" To contain this:

Scope every agent narrowly. Grant access to the specific data and tools a task requires, nothing more.

Enforce least agency, not just least privilege — restrict what actions an agent can take, not only what it can see. Manual access reviews can't keep up with how fast agent permissions expand, so this needs continuous, automated enforcement.

Treat each agent as a distinct identity with its own credentials, logging, and audit trail — not as an extension of the human who launched it.

Keep a human in the loop for consequential actions. An autonomous action can be fully policy-compliant and still be operationally damaging; human review on high-impact decisions is a safety net that policy alone can't provide.

3. Prevent data leakage and lock down the supply chain

Set a data-classification policy for AI use that defines exactly what data can and cannot be sent to which AI systems.

Use enterprise AI plans with no-training and data-processing guarantees, or self-host models entirely for your most sensitive workloads — for regulated data, keeping inference on your own infrastructure removes a whole category of third-party exposure.

Deploy DLP that inspects AI inputs and outputs, redacting sensitive data before it reaches a model or leaves it.

Never put credentials, API endpoints, or internal architecture in system prompts — system-prompt leakage has been one of the most consistently exploitable bugs, and an exposed prompt hands an attacker a map of your internals.

Pin and vet AI dependencies. Models, connectors, and integration middleware are now high-value supply-chain targets; treat the connectors linking AI to your core systems as critical infrastructure, and monitor vendor access, secrets, and OAuth tokens continuously rather than annually.

4. Bring shadow AI into the light

Governance policies with formal controls reduce data-leakage incidents by up to 46% versus having none — but only if usage is visible. A large share of employees paste company data into unmanaged personal AI accounts that bypass SSO and monitoring entirely. The fix is rarely a blanket ban (which just drives it further underground):

Provide good, approved AI tools so employees don't need to reach for unsanctioned ones.

Monitor for unapproved AI usage the way security teams track shadow IT.

Train staff on what data is and isn't acceptable to route through AI.

5. Runtime defense and continuous monitoring

Legacy security tools were built for files, domains, and structured patterns; AI risk is largely behavioral, so it has to be governed at the behavioral layer, not just the traffic layer. That means:

Runtime AI defenses (AI firewalls / guardrails) that detect and block prompt injections, jailbreaks, and exfiltration attempts as they happen, in both directions.

Anomaly detection on agent behavior — flagging an agent that suddenly queries far more records than usual, or reaches for a tool outside its normal scope.

Intent-based classification that can tell routine AI use apart from risky use, which binary allow/block controls can't do on their own.

Ground Every High-Stakes Output

For regulated or business-critical workflows, one property matters above almost all others: can you trace an AI output back to a verifiable source?

Ungrounded generation — an answer or action produced with no auditable link to source data — is hard to explain to a regulator, hard to trust when something breaks, and hard to catch when it's been manipulated. Retrieval-grounded, cited AI flips that: every answer points back to the document or record it came from, which makes it far easier to verify a result before it triggers an action, to defend it in an audit, and to spot when an injected instruction has pulled the output off course. For high-risk systems, the EU AI Act's Article 15 (effective August 2026) explicitly requires accuracy, robustness, and cybersecurity controls that map directly to these AI-security risk categories — so grounding and auditability are moving from "good practice" to legal obligation.

A Practical Rollout Sequence

If you're standing this up from scratch, order the work so you're patching the biggest holes first rather than boiling the ocean:

Map your AI surface. Inventory every place AI touches your software — embedded features, agents, connectors, and shadow usage. You can't protect what you haven't found.

Adopt NIST AI RMF as your governance backbone and define ownership, risk categories, and escalation paths.

Run the OWASP LLM + Agentic Top 10 as a technical assessment against each deployment, and prioritize by severity.

Fix the top three first: instruction/data separation, least privilege and least agency, and data-leakage controls. These cover the majority of real incidents.

Add runtime defense and monitoring so you can detect what your preventive controls miss.

Red-team continuously, using MITRE ATLAS as your playbook, because AI attacks evolve faster than static controls.

Get ISO 42001 certification when you need external proof for auditors, regulators, or enterprise buyers.

The Bottom Line

There is no single switch that makes software "AI-threat protected." What you can do is combine a governance framework (NIST AI RMF), an engineering baseline (OWASP LLM and Agentic Top 10), a threat-modeling playbook (MITRE ATLAS), and external assurance (ISO 42001) — then enforce the handful of controls that stop the attacks actually happening in production: separate instructions from data, give agents the least privilege and least agency possible, prevent data leakage, surface shadow AI, defend at runtime, and ground high-stakes outputs in verifiable sources.

The vendors building AI into banking, CRM, and ERP systems are still catching up on their own security. The organizations that stay safe won't be the ones waiting for the platforms to fix it — they'll be the ones who assume the platform will be breached and build defense-in-depth on top of it today.