KI-Flash: EDPB Publishes Guidelines on Web Scraping in the Context of Generative AI

Web scraping is practically indispensable for training large AI models – and, from a data protection perspective, one of the biggest open questions: who is liable if personal data ends up in a training dataset through the automated harvesting of the open internet? On 7 July 2026, the European Data Protection Board (EDPB) addressed this question in Guidelines 03/2026, presenting a concrete assessment framework for the first time. Having already reported on the EDPB's Opinion 28/2024 on AI models in an earlier KI-Flash, we now turn to this second major development from the same plenary session. We reported separately on the Guidelines on the Anonymization of Personal Data adopted at the same time. The new web scraping guidelines are likewise open for public consultation until 30 October 2026.

Web Scraping for AI Training Purposes

More precisely, web scraping refers to the automated extraction of large volumes of data from publicly accessible internet sources – one of the central methods for sourcing training data for generative AI models. Until now, there was no specific, EU-wide guidance on how this practice can be reconciled with the requirements of the GDPR. The new guidelines close this gap and build on the Opinion 28/2024 mentioned above, as well as on Guidelines 1/2024 on Article 6(1)(f) GDPR. They are addressed to private entities that scrape data themselves, engage third parties to do so, or use already-scraped datasets for training or fine-tuning.

Controllership: Who Is Responsible for the Scraping Process?

A key question in practice concerns the allocation of roles under data protection law: the EDPB clarifies that the entity carrying out the scraping is not automatically a controller within the meaning of the GDPR. What matters instead is who determines the purposes and means of the processing. If an AI developer engages a service provider to carry out scraping under documented instructions, that provider will generally qualify as a processor, while the developer is treated as the controller. Where an already-scraped dataset is reused by a third party, the scraper and the reusing AI developer are, in principle, separately responsible for their own respective processing. Only where both parties jointly determine the purposes and means does joint controllership come into consideration.

Transparency: When Does the Individual Duty to Inform Not Apply?

With controllership clarified, this also raises the question of adequate transparency: the information obligations under Articles 13 and 14 GDPR pose practical difficulties for controllers engaged in web scraping, since data subjects are often not individually identifiable where data is collected indirectly. The EDPB acknowledges that individual information may be dispensed with where it proves impossible or would involve disproportionate effort (Article 14(5)(b) GDPR). This exception, however, does not apply across the board; it requires weighing the effort involved against the impact on the data subjects concerned, considering the volume and age of the data and the safeguards already in place. As a minimum measure, the EDPB requires controllers in such cases to make the information publicly available, for instance through a privacy notice specifying the categories of data, the sources and, where possible, the characteristics of the crawler used.

Data Minimisation: Measures Before, During and After Collection

The principle does not rule out training on large volumes of data as such, but it does require that personal data not needed for the purpose should not be collected in the first place. The EDPB proposes a multi-layered set of measures to this end. Before collection, controllers should, among other things, consider using synthetic data, define precise selection criteria, and exclude websites that structurally contain particularly sensitive data or that technically oppose scraping, for example through robots.txt, ai.txt or CAPTCHA. During and after collection, syntax-based filtering, pseudonymization and anonymization come into consideration as well. In addition, the EDPB requires controllers to ensure data quality by relying on reliable sources, timestamping the data and carrying out sample checks, to meet the principle of accuracy.

Legitimate Interest as the Key Legal Basis

The question of which legal basis could justify any of this in the first place usually leads, in practice, to Article 6(1)(f) GDPR: consent is practically impossible to obtain in the case of indirect, large-scale collection, which is why web scraping for generative AI is regularly based on legitimate interest instead. The EDPB applies the familiar three-step test: the existence of a legitimate interest, the necessity of the processing, and a balancing of interests. As examples of legitimate interests, it cites the development of chatbots or improvements to threat detection. In the balancing exercise, particular weight is given to data subjects' ability to control their own data, possible chilling effects arising from a sense of being under surveillance, and data subjects' reasonable expectations, for example whether a website technically excludes scraping or whether the data was made recognizably and publicly available.

Where the balancing test comes out against the data subjects, mitigating measures such as opt-out lists, shortened retention periods or enhanced transparency measures can restore the lawfulness of the processing.

Special Categories of Personal Data

Handling sensitive data also poses a particular challenge: special categories of personal data under Article 9 GDPR are, in principle, subject to a prohibition on processing that can only be lifted where one of the exceptions under Article 9(2) GDPR applies. Because it is difficult to reliably rule out in advance that sensitive data will also be captured when scraping large volumes of data, the EDPB transposes the CJEU's reasoning in GC and Others (C-136/17), concerning the responsibility of search engine operators, to the web scraping context: the prohibition under Article 9(1) GDPR then applies only within the framework of the controller's responsibilities, powers and capabilities, provided the controller takes appropriate measures to prevent and delete such data before, during and after AI development. This transposition is subject to narrow conditions: it applies only where the activity is structurally comparable to that of a search engine, and only to the incidental, unintended capture of sensitive data.

Practical Note

Even though the guidelines have not yet been finally adopted, they already provide clear guidance that national supervisory authorities are likely to apply when reviewing existing and future training data pipelines. Companies that scrape data themselves, commission scraping, or purchase already-scraped datasets should promptly review their own documentation on the balancing of interests, data minimization measures and the handling of special categories of data against the criteria set out in the guidelines. The ongoing consultation also offers an opportunity to feed practical experience and concerns directly into the final text.

We would be glad to assist you in reviewing your training data pipelines for compliance with the new EDPB guidelines, as well as in preparing or updating your data protection documentation for AI training processes.