Cybersecurity of high-risk AI: the attacks and measures in sandbox Guide 11
Article 15 of the AI Regulation requires high-risk systems to resist attempts to alter their operation. Guide 11 of the Spanish sandbox describes the attacks specific to AI and the measures for the provider and for whoever deploys the system.
An artificial intelligence system can fail because someone attacks it. And attacks on AI are not only the classic ones on any computer system: there are specific techniques that manipulate the data it learns from or the inputs it receives. Guide 11, "Cybersecurity", of the Spanish regulatory sandbox pilot, explains them.
Its version 2.0 dates from 17 July 2026, adapted to the Digital Omnibus; version 1.0 is from 10 December 2025. It is credited to the Ministry for Digital Transformation and the Civil Service and, like the rest of the series, is not binding.
The basis: Article 15 of the AI Regulation
The guide focuses on two paragraphs of Article 15 of Regulation (EU) 2024/1689, on accuracy, robustness and cybersecurity of high-risk systems:
- Paragraph 1: systems must reach an appropriate level of cybersecurity and perform consistently throughout their lifecycle.
- Paragraph 5: they must be resilient against attempts by unauthorised third parties to alter their use, outputs or performance.
Accuracy and robustness are dealt with in other guides in the series.
Attacks specific to AI
| Attack | What it consists of, according to the guide |
|---|---|
| Data poisoning | Manipulating the training data set |
| Model poisoning | Manipulating pre-trained components |
| Adversarial examples | Inputs designed to make the model err |
| Attacks on confidentiality | Expressly mentioned in Article 15.5, according to the guide |
| Model flaws | Also mentioned in Article 15.5, according to the guide |
Who must do what
The provider must adopt organisational and technical measures, plan cybersecurity from the design stage, include specific recommendations in the instructions for use and define indicators and periodic audits.
The deployer must read and apply the provider's instructions, distribute installation and configuration information within its organisation and, if the system is on its premises or it manages it, treat it as an asset to protect within its cybersecurity policy. If it retrains the system, it takes on responsibilities similar to the provider's on data.
Some of the measures it proposes
- Inventory of data, models, software and actors, with assigned responsibilities.
- Threat model aligned with the risk analysis, especially for rights and freedoms.
- Access policies, with separation of duties where possible.
- Specific training in AI cybersecurity.
- Version control of data and models, and encryption in transit and at rest.
- Detection of anomalous inputs and query limiting.
- Security testing, including penetration tests oriented to machine learning.
- Multi-factor authentication, detailed logs and continuous monitoring.
The guide also recalls that the Cyber Resilience Act will apply, from 11 December 2027, to products with digital elements classified as high-risk AI systems.
Usefulness for lawyers
In procurement. The distinction between what falls to the provider and what falls to the deployer is useful for drafting supply contracts for high-risk systems: who audits, who updates and who answers for each layer of security.
In incident management. If a system is attacked, knowing which measures should have been in place helps assess responsibility.
In-house. Even if a firm does not use high-risk systems, the basic ideas, such as access control, training or multi-factor authentication, apply equally to its generative AI tools. The AEPD's guidance goes in the same direction, as we explain in the article on agentic AI and data protection.
The series of guides
This is the fourth sandbox guide we have summarised. The earlier ones deal with data governance, transparency and high-risk practical cases.
More analysis in Legal AI.
Fuentes
- Ministerio para la Transformación Digital y de la Función Pública. Guía 11. Ciberseguridad, versión 2.0, 17 de julio de 2026 (PDF). Accessed 18 October 2026.
The specific data in this article come from the official sources cited.
