Terms
- Adversarial machine learning
- Techniques that manipulate ML models through crafted inputs or training data, including evasion, poisoning, model inversion and membership inference.
- Agent (AI agent)
- An AI system that plans and takes actions toward a goal, typically by calling tools, APIs or other agents.
- Agent-to-agent (A2A)
- Communication between AI agents, which requires authentication and policy like any other system-to-system traffic.
- AI firewall
- An enforcement point that inspects prompts, responses and tool calls in real time against security and data policy. See AI firewalls.
- AI gateway
- A proxy that routes and governs traffic to AI models, often providing access control, logging, rate limiting and policy enforcement.
- AI governance
- The policies, roles, processes and controls for responsible AI use. See AI governance.
- AI RMF
- NIST's AI Risk Management Framework, organized into Govern, Map, Measure and Manage functions.
- Confabulation / hallucination
- Model output that is fluent but false or unsupported.
- Data poisoning
- Corrupting training, fine-tuning or retrieval data to change model behavior (OWASP LLM04).
- Denial of wallet
- Abuse that drives up AI usage costs through excessive or looping requests (part of OWASP LLM10, Unbounded Consumption).
- DLP (for AI)
- Data loss prevention that detects and redacts sensitive data in prompts and responses. See AI data leakage.
- Embedded AI
- AI features built into existing software that may process company data without a separate purchase decision.
- Embeddings
- Numeric vector representations of text or other data used for search and retrieval; they can still encode sensitive information.
- EU AI Act
- The European Union's risk-based AI regulation, Regulation (EU) 2024/1689, amended in 2026 by Regulation (EU) 2026/1744.
- Excessive agency
- Giving an LLM system more permissions, functions or autonomy than necessary (OWASP LLM06).
- FedRAMP
- The U.S. federal program for security authorization of cloud services.
- Guardrails
- Rules and checks applied to model inputs and outputs to keep behavior within policy.
- Human-in-the-loop
- Requiring human review or approval before an AI output or action takes effect.
- Indirect prompt injection
- Malicious instructions hidden in content an AI later reads, such as web pages or documents.
- ISO/IEC 42001
- The international, certifiable standard for AI management systems.
- Jailbreak
- A prompt designed to make a model ignore its safety rules.
- LLM
- Large language model: a model trained on large text corpora to generate and understand language.
- MCP (Model Context Protocol)
- An open protocol for connecting AI applications to tools and data sources.
- MITRE ATLAS
- A knowledge base of adversary tactics and techniques against AI systems.
- Model inversion
- An attack that reconstructs training data or sensitive attributes from model outputs.
- OWASP Top 10 for LLM Applications
- A community list of the most critical security risks for LLM applications; the 2025 edition ranks prompt injection first.
- Prompt injection
- Input that overrides or subverts an LLM application's intended instructions. See prompt injection.
- RAG (retrieval-augmented generation)
- Supplying a model with retrieved documents at query time so it can answer from current or private data.
- Shadow AI
- AI use not approved by IT or security. See shadow AI.
- System prompt leakage
- Exposure of an application's hidden instructions (OWASP LLM07).
- Zero trust for AI
- Applying "never trust, always verify" to AI users, models, agents and data flows: verify identity, enforce least privilege and inspect continuously.
Frequently asked questions
What is the difference between AI safety and AI security?
AI safety focuses on preventing AI systems from causing unintended harm through their behavior. AI security focuses on protecting AI systems and the data they touch from attackers and misuse. In practice the controls overlap.
What does zero trust mean for AI?
It means no AI user, model, agent, tool or data source is trusted by default: each request is authenticated, authorized with least privilege and inspected, and every action is logged.
Disclosure: PromptDefend recommends ZeroTrusted.ai and is published in association with it. Product details are sourced from zerotrusted.ai. Regulatory summaries are educational and not legal advice.