AI Assistant: An AI tool (often LLM powered) that helps user's complete tasks like drafting text, summarising information, or answering questions. Some assistants are “read-only” (they only generate text), while others can connect to apps and data sources.
AI Agent: An LLM-powered system that can plan, use tools, maintain memory, and take actions to accomplish goals (e.g. “open a ticket, email the user, update a record”).
AI Hallucination: When an LLM generates output that sounds confident but is incorrect or made up.
AI Impersonation: When someone uses AI (e.g. voice cloning or deepfake video) to pretend to be a real person (a colleague, friend, executive, or public official) in order to trick others.
AI Security Incident: A security event involving an AI system (or an AI enabled workflow) that could affect confidentiality, integrity, or availability.
Audit Trail: A record of what happened in a system—who did what, when, and what changed—used for investigation and accountability.
Bias: When the model’s outputs consistently reflect skewed, unfair, or one‑sided viewpoints because of patterns in the data it was trained on (or because of the way it responds to certain prompts).
Connector: A built-in integration that connects an AI tool to another service (e.g. file storage, email, CRM).
Data Exfiltration: Unauthorised transfer of data to an attacker or outside the organisation.
Data Minimisation: Only collecting/using the minimum data needed for a task.
Denial of Wallet: An attack that drives up AI usage costs (e.g. forcing repeated tool calls, long loops, or excessive token usage).
Excessive Agency: When an LLM system is given more functionality, permissions, or autonomy than it needs, enabling damaging actions when the model is confused, manipulated, or wrong.
Exposed Admin Page: A device or service management interface that is reachable by the wrong people (often due to weak passwords, misconfiguration, or being exposed to the internet).
Guardrails: Controls that reduce unsafe AI behaviour (policy rules, filters, validations, approvals, or design constraints).
Human-in-the-Loop (HITL): A design choice where a human must approve high-risk actions (e.g. sending emails, deleting data, making payments).
Insecure / Improper Output Handling: When LLM output is passed into downstream systems (web pages, databases, scripts, email templates) without validation, sanitisation, or encoding.
Incident Response (IR): The process of preparing for, detecting, responding to, and recovering from security incidents.
Internet of Things (IoT): Internet-connected devices (smart speakers, cameras, appliances, toys) that connect to home or business networks.
Least Privilege: Giving systems (and AI tools) only the minimum access and permissions needed to do the job.
LLM (Large Language Model): A model trained on large amounts of text to generate and interpret language (used in chatbots like ChatGPT).
Logging (Event Logging): Capturing security-relevant events (who accessed what, what changed, what actions were taken) to enable detection and investigation.
Malicious Plugins / Tools / Platforms: Third party add-ons or services that look legitimate but are designed to steal data/credentials or execute harmful actions in AI ecosystems.
Memory Poisoning: When an attacker causes an AI agent/assistant to store malicious or misleading content in its memory, so the agent behaves wrongly later (even after the original conversation ends).
Misinformation (as an LLM risk): When an LLM produces or spreads incorrect information that may influence decisions.
Monitoring: Watching systems for unusual behaviour (alerts, anomaly detection, trend analysis).
Overreliance: Treating LLM output as trusted and correct without checks, despite the risk of confident errors (hallucinations).
Plugin / Tool: An extension that gives an LLM the ability to do more than generate text (e.g. call an API, access files, send messages).
Prompt: The text a user provides to an LLM (a question, instruction, or request).
Prompt Injection: A vulnerability where attacker-controlled text manipulates an LLM to behave in unintended ways.
Direct prompt injection: malicious instructions in the user’s input.
Indirect prompt injection: malicious instructions hidden in content the model reads (documents/webpages/emails) that the model follows.
Red Teaming: Structured testing where you try to break or abuse an AI system (e.g. prompt injection tests, data extraction attempts) to find weaknesses before attackers do.
Sensitive Information Disclosure: When an LLM (or an LLM-enabled application) unintentionally exposes confidential data in outputs (PII, credentials, internal business data).
Supply Chain Vulnerabilities (in LLM systems): Risks introduced by third party components used to build and run LLM systems (models, datasets, libraries, hosting, plugins/tools).
System Prompt: Hidden instructions that define how the model should behave (e.g. role, rules, boundaries) in a chat/application.
Tool Abuse: When an AI assistant/agent is tricked into misusing the tools or connectors it has access to (e.g. email, files, APIs, databases), resulting in unintended actions or unauthorised access.
Triage (Incident Triage): The initial incident response step where you quickly determine what’s happening, how severe it is, and what to do first to contain it.