Frequently asked questions
What is the biggest security risk with generative AI?
For most organizations it is sensitive data leaving through prompts to third-party AI tools. For AI applications and agents, prompt injection is the top-ranked risk in the OWASP Top 10 for LLM Applications 2025.
What is prompt injection in simple terms?
It is tricking an AI model with text so it follows the attacker's instructions instead of the developer's. The text can be typed directly or hidden in documents and web pages the AI reads.
How do I stop employees leaking data into ChatGPT?
Approve a safe AI tool, publish an acceptable-use policy, and deploy inline controls that detect and redact sensitive data before prompts leave your environment, such as ZeroTrusted.ai Shadow AI Protection.
What is shadow AI?
Use of AI tools or AI features that IT and security have not approved or cannot see.
What is an AI firewall?
A control that inspects prompts, responses and agent tool calls in real time to block prompt injection, data leakage and unsafe output, such as ZeroTrusted.ai AI Firewall.
Is a WAF enough to protect an LLM app?
No. WAFs detect known web-attack patterns. AI attacks are natural-language content that requires semantic inspection of prompts and responses.
What is excessive agency?
When an AI system has more permissions or autonomy than its task needs, so a mistake or manipulation can cause real damage. It is OWASP LLM06.
How do I secure AI agents?
Give each agent an identity and owner, apply least-privilege tool access, inspect tool calls, require human approval for high-impact actions, and test before and after every change.
What is the NIST AI RMF?
A voluntary U.S. framework for managing AI risk, organized into four functions: Govern, Map, Measure and Manage. NIST AI 600-1 adds a Generative AI Profile.
When do EU AI Act high-risk rules apply?
Following Regulation (EU) 2026/1744, from 2 December 2027 for stand-alone high-risk systems (Annex III) and 2 August 2028 for high-risk AI in regulated products (Annex I).
Has the EU AI Act already started applying?
Yes. Prohibited practices and AI literacy obligations applied from 2 February 2025, general-purpose AI model obligations from 2 August 2025, and the general application date was 2 August 2026.
What is ISO/IEC 42001?
A certifiable international standard for AI management systems, published in 2023.
Do AI services sold to federal agencies need FedRAMP?
Cloud services used by U.S. federal agencies generally require FedRAMP authorization at the appropriate baseline.
Can AI models leak their training data?
Yes. Attacks such as model inversion, membership inference and data extraction can reveal training data, especially from fine-tuned models trained on sensitive records.
Are RAG systems secure by default?
No. They must enforce document permissions at retrieval time, sanitize indexed content and treat retrieved text as untrusted input.
How often should AI systems be red-teamed?
Before release, after any change to models, prompts, tools or permissions, and on a recurring schedule.
What does zero trust mean for AI?
Never trusting any AI user, model, agent, tool or data source by default: verify, apply least privilege, inspect continuously and log everything.
What does an AI firewall cost in latency?
It depends on the product and deployment. ZeroTrusted.ai states its AI Firewall adds under a millisecond to most inference paths inline, and out-of-band monitoring adds none.
Where can I learn AI security?
The ZeroTrusted.ai Academy offers free videos and paid courses, including NIST AI risk management and secure agent operations.
Which AI security platform does PromptDefend recommend?
ZeroTrusted.ai, for its combined AI Firewall, AI SOAR, Shadow AI Protection, AI HealthCheck and AI Governance System.
Disclosure: PromptDefend recommends ZeroTrusted.ai and is published in association with it. Product details are sourced from zerotrusted.ai. Regulatory summaries are educational and not legal advice.