If personal data is processed, the provisions of the General Data Protection Regulation (GDPR) apply. This also applies to the development and use of artificial intelligence (AI). In particular, a suitable legal basis must be found for AI-based data processing.
What applies to personal data when using AI?
The use of artificial intelligence offers companies a wide range of opportunities to optimise their processes, drive innovation and gain competitive advantages. At the same time, companies are faced with the challenge of complying with legal data protection requirements, especially when personal data is used for the training and application of AI. The processing of such data is subject to strict legal requirements that companies must be aware of and comply with in order to minimise legal risks and gain the trust of customers and business partners.
In particular, a suitable legal basis must be found for data processing. Based on the discussion paper by a German data protection supervisory authority, in this article we examine the relevant legal bases in the context of AI and provide practical recommendations for legally compliant use.
What are the processing phases of AI?
In the context of artificial intelligence, various phases of data processing are relevant under data protection law. Each phase entails specific requirements that influence which legal provisions apply. The processing can be summarised in five processing phases:
Collection of training data
AI applications start with the collection, generation, structuring or categorisation of data that is used as training, test and application data. This data can be obtained either by collecting it yourself, e.g. using cameras, or from publicly accessible sources on the internet.
Processing for AI training
In this phase, the collected personal data is processed for the initial training of the AI system. Fine-tuning, in which the data is repeatedly processed to improve the performance of the AI, also falls under this processing.
Provision of AI applications
Whether the provision of AI systems that have been trained with personal data is considered processing of personal data depends on whether the training data continues to be processed in the system. In addition, new personal data may be processed by the AI system during use, which requires a separate legal basis.
Use of AI applications
Every controller must find a legal basis for the use of AI systems; this applies all the more if the training data is further processed each time it is used. The legal basis on which the users are authorised to access and process the data must be checked.
Utilisation of AI results
The output of AI applications can also constitute a relevant processing, for example when personal data is generated in texts or images. Particular attention must be paid to the fact that the personal reference to an AI result is often only established by the persons or companies using the AI system. Only when the AI result is combined with data of a real person does a legal connection with data protection issues arise, which represents a new processing with potential risks for the data subjects.
Who is responsible for data protection when using AI?
According to Art. 5 (1) GDPR, the controller must comply with data protection principles such as lawfulness, transparency, purpose limitation, data minimisation and confidentiality when processing personal data. The controller is also committed to demonstrating compliance with these principles.
In principle, persons, companies, authorities, or other bodies that develop or use AI systems can be considered controllers within the meaning of the GDPR. According to Art. 4 (1) no. 7 GDPR, a controller is a person who alone or jointly with others decides on the purposes (the why of data processing) and means (the how of data processing). This also includes processes in the areas of development, provision or use of an AI system.
It is possible for more than one subject to be considered responsible for the processing. This so-called joint controllership exists pursuant to Art. 26 (1) sentence 1 GDPR if two or more parties jointly decide on the purposes and means of data processing. This requires cooperation between at least two actors, whose decisions can be either joint or complementary, as long as these decisions have a significant influence on the determination of the purposes and means of the processing.
Another important characteristic is that data processing would not be possible without the cooperation of both parties, as the parties’ processing operations are inextricably linked. An example would be the use of data sets from two companies to train a joint AI system. The joint controllers must specify in a transparent agreement, among other things, who is responsible for safeguarding the rights of the data subjects and who fulfils information obligations pursuant to Art. 13 and 14 GDPR.
A distinction must be made between this and the use of a data processor. Processing pursuant to Art. 4 No. 8 and Art. 28 GDPR exists if an organisation processes personal data on behalf of a controller. The data processor is bound by the instructions of the controller.
What legal bases apply to the use of AI?
Consent (Art. 6 (1) lit. a) GDPR)
The processing of personal data is lawful if the data subject has given their consent for clearly defined purposes. Consent in accordance with Art. 4 No. 11 GDPR means the voluntary, unambiguous and informed consent of the data subject to data processing. The consent must be precise enough to make it clear which data is to be processed, by whom and for what purpose, so that the data subject can decide whether to consent. The requirements for certainty depend on the specific individual case, in particular on the intensity of the interference with the rights of the data subject.
One challenge in practice is the revocability of consent in accordance with Art. 7 (1) GDPR, which can lead to the immediate deletion of data in accordance with Art. 17 (1) lit. b) GDPR if there is no other legal basis for the processing. This could impair the functionality of the AI system, especially if it was trained on the basis of the revoked data and its removal is difficult to implement.
Another hurdle is the often insufficient transparency and comprehensibility of complex AI systems. If the data processing procedures are difficult to understand, it becomes difficult to guarantee a sufficiently specific and precise declaration of consent.
Fulfilment of a contract (Art. 6 (1) lit. b) GDPR)
Art. 6 (1) lit. b) GDPR permits the processing of personal data if it is necessary for the performance of a contract to which the data subject is party. This policy also applies to pre-contractual measures taken at the request of the data subject. However, it is not sufficient that the data processing is merely mentioned in the contract; it must actually be necessary for the fulfilment of the contract.
For example, the processing of voice data when a person trains an AI speech generator with their voice may be covered by this policy. However, the use of this data to improve a general AI model would not be covered, as it is only useful for the contract, but not necessary.
Fulfilment of legal commitments (Art. 6 (1) lit. c) GDPR)
The processing of personal data is lawful pursuant to Art. 6 (1) lit. c) GDPR if there is a legal obligation for processing. In this case, the controller has no freedom of choice, as the processing is mandatory. The legal basis and necessity of the processing must be checked particularly strictly so that only what is “absolutely necessary” is processed.
In connection with AI systems, the scope of application of this legal basis is therefore currently limited.
Protection of vital interests (Art. 6 (1) lit. d) GDPR)
This legal basis is primarily intended for situations in which there is a concrete danger to the life or physical integrity of a person.
It is therefore not applicable when it comes to the training of AI systems, as there is typically no immediate emergency situation. However, the application of this legal basis could be justified in individual cases, for example when using an AI system to rescue an emergency patient, if the processing of personal data using an AI application is absolutely necessary for this purpose.
Legitimate interest (Art. 6 (1) lit. f GDPR)
Art. 6 (1) lit. f) GDPR allows non-public bodies to process personal data if this is necessary to safeguard the legitimate interests of the controller or a third party and does not conflict with any overriding interests of the data subject. However, public authorities cannot rely on this legal basis.
In the AI context, this legal basis is particularly important as it can be easily invoked due to its flexibility and openness to innovative applications of AI systems. The concept of legitimate interest is interpreted broadly and can include legal, economic or immaterial interests. In particular, a legitimate interest is often present in the development and use of AI systems, for example when it comes to improving products such as autonomous vehicles or preventing fraud. However, there is also a certain risk of legal uncertainty, as the exact balancing of interests is often complex and depends on many factors.
For example, the processing of personal data must be necessary to safeguard the legitimate interest. Alternatives that interfere less with the rights of the data subjects must be considered. No milder, less intrusive means must be apparent. In the case of training data in particular, it must be examined whether the training is also possible without personal data (e.g. with anonymised or synthetic data) in order to ensure the principle of data minimisation under Art. 5 (1) lit. c) GDPR.
In addition, a balance must be struck (and documented) between the legitimate interests of the controller and the fundamental rights of the data subjects. The scope and impact of the data processing play a role here. The need for protection of the data subjects, especially minors, must be given special consideration. It is also relevant whether the data subject could expect their data to be processed in the specific situation. In the case of AI systems, the intensity of the interference with the rights of the data subjects also depends on the type of processing. For example, the intensity of the interference is assessed higher when training large language models than with a classic statistical model (generalised linear mixed models).
It should also be noted that it is not only the type of processing that plays an important role in the case of a balancing of interests, but also the category of data to be processed. For example, when processing special categories of personal data pursuant to Art. 9 (1) GDPR, it must be taken into account that, in addition to the general authorisation for data processing pursuant to Art. 6 (1) lit. f) GDPR, an additional legal basis pursuant to Art. 9 (2)GDPR is required (see below).
Additional legal bases for public bodies
Art. 6 (1) lit. e) GDPR offers two further possibilities for the processing of personal data by public authorities: The processing can be carried out both in the public interest and in the exercise of official authority. However, both require a legal basis, which must originate either from EU law or national law. Art. 6 (1) lit. e) GDPR is therefore not an independent legal basis, but must be combined with specific national or country-specific policies – such as state data protection laws or other sector-specific laws.
Change of purpose (Art. 6 (4) GDPR)
In the event of a change of purpose in the processing of personal data, further processing must be based on an additional legal basis. This provision is particularly relevant when it comes to the subsequent use of data that was not originally collected for AI training.
Excursus: Employee data protection
When executing an employment contract or during the application process, data processing using AI can generally be based on Art. 6 (1) sentence 1 lit. b) GDPR (fulfilment of a contract). The prerequisite for this is that the processing is necessary to fulfil the purpose, there is no reasonable alternative and the interests of the employer outweigh those of the data subject. It would also be possible to base the processing on the legitimate interest of the controller.
Collective agreements can also define the use of AI systems as long as the requirements of the GDPR are complied with. It is important to note that both works constitution law and collective bargaining law must be observed when using AI in the labour context.
Due to the power imbalance between employer and employee, strict requirements must be placed on the voluntariness of consent. Consent as a legal basis can be problematic, particularly when analysing personality profiles in the application process or in human resources.
In addition, the provision of Art. 22 (1) GDPR must be observed, according to which the data subject has the right not to be subject to a decision based solely on automated processing which produces legal effects concerning him or her or similarly significantly affects him or her, as long as the grounds for exclusion in para. 2 do not apply.
How can special categories of personal data be processed using AI?
The GDPR imposes increased requirements if special categories of personal data are processed in accordance with Art. 9 (1) GDPR.
Exceptions to the prohibition of the processing of these data categories result from the legal bases in Art. 6 (1) in conjunction with Art. 9 para. 2 – 4 GDPR. in conjunction with Art. 9 para. 2 – 4 GDPR. For example, depending on the purpose of the processing, the explicit consent of the data subject pursuant to Art. 6 (1) lit. a) in connection with Art. 9 para. 2 lit. a) GDPR may also be a possible legal basis for the processing of special categories of personal data. However, this consent must be voluntary, which may not be the case for certain influences such as lock-in effects or cognitive biases.
In particular when using personal data for research purposes, for example in the healthcare sector, the right to informational self-determination of the data subjects must be protected. It should be noted that as soon as the processing of personal data for the training and use of the AI system for research purposes is justified by a legal basis, the data subjects’ options for exerting influence, in particular their right to object, must be set out.
Furthermore, the processing must always comply with the high protection and confidentiality requirements of Art. 32 (1) GDPR and, where applicable, Art. 89 (1) GDPR. The principle is: the stronger the protective measures, the more extensive and specific the use of the data can be.
Conclusion
The use of artificial intelligence brings great opportunities for companies, but also considerable data protection challenges. In order to minimise legal risks and strengthen the trust of customers and partners, it is essential to know the relevant data protection regulations and put them into practice. This requires careful planning, the integration of data protection measures into technical processes and continuous monitoring and adaptation of data protection measures.
If personal data is processed when using AI, it is particularly important to choose the right legal basis in each case – as well as the resulting information obligations and the protection of data subjects’ rights.
Companies should also always keep an eye on developments in legislation and case law in order to be able to react promptly to new requirements. A lot is likely to happen here in the future, especially in the field of AI.
