AI Security Checklist for Production Teams
Use this AI security checklist to assess models, data, vendors, agents, and monitoring before a release becomes an incident in production. For teams.
A model can pass every benchmark, impress a pilot group, and still create a production security problem on its first day. The gap is usually not model quality. It is the surrounding system: who can reach it, what data it can retrieve, which actions it can take, and whether anyone will notice abuse quickly enough. This AI security checklist is built for teams moving from experimentation to systems that handle real users, proprietary data, and business workflows.
The objective is not to eliminate every model error. That is not realistic. The objective is to reduce the blast radius, make risky behavior difficult to trigger, and give security and product teams evidence that controls work under pressure.
Start With the AI System, Not the Model
An AI feature is rarely just an API call. It is a chain of components that may include a foundation model, system prompts, retrieval, vector databases, identity providers, plugins, agent tools, evaluation services, logging pipelines, and third-party infrastructure. Security reviews fail when they assess only the model endpoint.
Before approving a release, map the full path from user input to model output and downstream action. Identify every trust boundary. A public chatbot that can answer questions from approved documents carries one risk profile. An internal agent that can search customer records, create support tickets, and issue refunds carries another.
The key question is direct: what can an attacker cause this system to reveal, change, spend, or do? Treat prompt text as untrusted input, even when it appears inside a document, email, calendar event, webpage, or retrieved knowledge-base result.
Define ownership and acceptable failure
Assign a named owner for the AI system, plus owners for application security, data governance, incident response, and vendor management. Shared responsibility without clear decision rights becomes a delay during an incident.
Set explicit boundaries for acceptable behavior. Specify which data categories the system may access, which outputs require review, and which tool actions are prohibited without a human approval step. These decisions should be versioned alongside the product requirements, not buried in a slide deck.
AI Security Checklist: Data, Access, and Isolation
The highest-value controls tend to be ordinary security disciplines applied carefully to AI-specific workflows. Teams should verify the following before a production deployment:
- Classify every data source. Separate public information from internal, confidential, regulated, and customer data. Do not assume a model provider's enterprise terms resolve your obligations for data minimization or retention.
- Apply least-privilege retrieval. Retrieval-augmented generation should enforce the requesting user's permissions at query time. A vector index that returns documents a user could not otherwise open is an access-control failure, not a helpful model feature.
- Protect secrets outside prompts and documents. API keys, credentials, tokens, connection strings, and private keys should never be available to the model context. Use a dedicated secrets manager and scoped service identities for tools.
- Separate environments. Development prompts, test datasets, production logs, and production credentials need distinct controls. Real customer data in a convenient test environment is a recurring and avoidable exposure.
Data retention deserves a specific review. Confirm what is stored by the application, model provider, observability platform, and any annotation or evaluation vendor. Check whether prompts and outputs are used for training, how deletion requests work, and whether logs can be searched by unauthorized staff.
Redaction can help, but it is not a substitute for authorization. A redacted output may still expose sensitive meaning through context, while an authorized user may need the original record to complete legitimate work. Design around data classification and access policy first.
Test for Prompt Injection and Unsafe Tool Use
Prompt injection is not a one-time filter problem. It is an attempt to manipulate a model through untrusted content, often to override instructions, extract sensitive context, or trigger a tool call. It can arrive through direct user input or indirectly through a file, website, support ticket, or document retrieved by the system.
Test realistic attack paths, not just obvious phrases such as “ignore previous instructions.” Ask whether a malicious document can instruct the agent to disclose other retrieved content. Test whether a user can coerce the model into calling an administrative tool, changing a transaction, or sending data to an external destination.
Put policy enforcement outside the model
A model can recommend an action, but it should not be the final authority for sensitive decisions. Build deterministic checks around tool use: allowed parameters, destination allowlists, transaction limits, tenant boundaries, and approval requirements. If an agent can send email, for example, validate recipients and require confirmation for external domains or high-volume sends.
Use constrained interfaces where possible. A tool that accepts a structured request with a narrow schema is safer than unrestricted shell access or a generic database query interface. The trade-off is flexibility. For early internal workflows, narrow tools may feel slower to build. For high-impact functions, that friction is cheaper than recovering from an unauthorized action.
Run adversarial evaluations before release and after material changes to prompts, models, tools, or retrieval sources. Include multilingual attacks, role-play attempts, encoded instructions, poisoned documents, data exfiltration requests, and multi-step tasks that exploit an agent's persistence.
Verify Model and Vendor Exposure
Most production AI stacks depend on external model APIs, open-weight models, hosting platforms, embedding services, and data-processing vendors. Each dependency expands the attack surface and complicates accountability.
Review vendor terms, security documentation, incident notification commitments, data processing practices, geographic handling, access controls, and subcontractors. For open-weight models, validate the source, version, license, checksum, and artifact provenance. A model file is software supply chain input, not a mysterious asset that should bypass normal intake procedures.
Track model versions in production. Providers can retire models, alter behavior, change safety systems, or update pricing with operational consequences. Re-run critical security tests when a provider changes a model version or when your team swaps embeddings, rerankers, orchestration frameworks, or agent libraries.
There is no universal answer on whether to use hosted or self-managed models. Hosted services can reduce operational burden and offer mature security features, while self-hosting may improve control over data location and isolation. The right choice depends on data sensitivity, compliance requirements, internal capacity, latency needs, and the actions the system can perform.
Monitor Behavior That Traditional Logs Miss
Standard application telemetry is necessary, but AI systems need additional signals. You need to know not only that a request succeeded, but what data sources were retrieved, whether tools were invoked, which policy checks fired, and whether behavior drifted after a change.
Log enough context to investigate incidents while minimizing sensitive prompt and output retention. Record model and prompt versions, retrieval document identifiers, tool names, authorization decisions, latency, error states, and user or service identities. Limit who can view raw content, and set retention periods that match the risk.
Create alerting for patterns that merit human review: repeated denied tool calls, sudden spikes in retrieval volume, unusual output lengths, cross-tenant access attempts, external data transfers, prompt-injection indicators, and agents retrying actions after failures. A single alert may be noise. A cluster around one account, document collection, or workflow may be the first signal of active abuse.
Prepare an AI-specific response path
Incident response should include controls that can be applied quickly without redeploying the entire application. That might mean disabling a tool, revoking a service identity, removing a retrieval source, rolling back a prompt, pinning a model version, or routing outputs to human review.
Practice these moves. A tabletop exercise should test more than a generic data breach scenario. Simulate a poisoned document that causes an agent to attempt external exfiltration, or a permission bug that exposes another customer's knowledge base. Measure how quickly the team can identify the affected requests, contain the behavior, notify stakeholders, and restore service safely.
Make Security a Release Gate, Not a Cleanup Task
The fastest teams do not treat this checklist as a quarterly compliance artifact. They convert it into release criteria. New tools, permissions, data sources, models, and agent capabilities should trigger proportionate review before they reach users.
Risk should scale with capability. A summarization feature over public content may need basic input handling, vendor review, and monitoring. An autonomous workflow with access to payroll, source code, financial systems, or customer communications needs stronger isolation, approval controls, adversarial testing, and an operational owner on call.
The useful standard is not whether a system sounds safe in a demo. It is whether your team can explain, test, and control what happens when the model encounters hostile input, sensitive data, or a request it should never be allowed to execute. That is the point where an AI feature becomes ready for production scrutiny.




Comments
Keep comments relevant and respectful. No links, advertising, self-promotion, or spam.