Generative AI Security Fundamentals

AI applications introduce familiar application-security risks plus model-specific failure modes.

Protect the system at multiple boundaries

• Identity: authenticate users and services and enforce least privilege.
• Data: classify, encrypt, and authorize access to source documents and retrieved context.
• Prompt boundary: treat user-provided instructions and retrieved text as untrusted input.
• Tool boundary: allow agents to call only explicitly authorized tools and validate arguments.
• Model boundary: apply safety policies and test harmful or adversarial inputs.
• Output boundary: validate structured output before downstream execution.
• Observability: record security-relevant events without unnecessarily logging secrets or sensitive data.

RAG-specific risk

Filtering access only at the application endpoint is insufficient if retrieval can return documents the user is not authorized to see. Authorization must be enforced at the retrieval/data layer as appropriate for the system.

Agent-specific risk

An agent that can execute actions should be treated as a privileged application component. Minimize tool permissions, validate inputs and outputs, require approval for high-impact actions, and define safe failure behavior.

Connected knowledge

RAG → S3 access troubleshooting → secure file architecture → production agent architecture.