Federated Learning: What Lawyers Need to Know

This article analyses some of the legal issues raised by Federated Learning (FL) – a method to train algorithms for artificial intelligence systems, on data held across different data sets, without sharing the data sets. This article will explain what FL is and will highlight some of the relevant legal issues. In particular this article will analyse how the Personal Data Protection Act (PDPA) applies to FL, arguing that FL is not a panacea for compliance with the PDPA but it can be a useful tool for organisations to meet their requirements pursuant the PDPA. As many of the legal questions raised by FL are, as yet, untested, the goal of this article is to educate the legal community about these questions and to invite a discourse on the legal treatment of FL in its practical application.

1. Introduction

The use of Artificial Intelligence (AI) has become customary, and all professionals, including lawyers, are familiar with AI in their daily work, and are often asked to provide advice on legal issues raised by AI. However, among non-specialists, the discussion on AI often remains on the surface, without a detailed understanding of the underlying concepts. One of the concepts which has escaped attention, outside a small circle of specialists, is Federated Learning (FL) – a method to train an algorithm on data across different data sets, without sharing the data sets. This article will fill this lacuna by explaining what FL is and highlight some of the relevant legal issues. As many of the legal questions raised by FL are, as yet, untested, the goal of this article is to educate the legal community about these questions and to invite a discourse on the legal treatment of FL.

2. What is Federated Learning?

The basic idea behind FL is simple. The current iteration of AI relies on algorithms being trained on vast amounts of data. In standard training of AI, if data is held across data sets, the relevant data is pooled together by transmitting (or copying) the data to a central repository, where the algorithm is then trained on the pooled data. In contrast, in FL the data is not transmitted to a repository and remains in the original data sets. Instead, an algorithm is transmitted to the different data sets, and the algorithms are then trained locally on the data sets. Once the training is completed, the trained algorithm (or the relevant parameters of the algorithm) is transmitted back and combined with the trained algorithms from the other data sets, into a single unified algorithm. The result is that the algorithm was effectively trained on all the data sets, but without the data being transferred out of these data sets.

The problem FL is intended to solve is that data is often held in “data silos”. A data silo can be anything that holds data, from smartphones to research laboratories or hospitals. Some of these data silos are created to meet regulatory requirements (e.g. IP and privacy laws, like the Personal Date Protection Act (PDPA)). Others are created because there is simply an unwillingness to share data (perhaps due to concerns such as authenticity, reliability or trade secrets). FL is intended to “break open” these data silos, by enabling data sharing whilst preserving privacy and protecting other concerns, thereby potentially making significantly more data available for the training of AI. If FL can fulfil this expectation, the benefits are potentially vast. For example, healthcare providers could train algorithms to improve treatments and develop new drugs based on patient data, while maintaining privacy and patient confidentiality; or researchers in different countries could train algorithms without transmitting data across jurisdictions. The potential significance of FL has not gone unnoticed in Singapore either, with AI Singapore launching Synergos, an open-source FL platform, in February 2024.¹AI Singapore, Synergos, https://aisingapore.org/aiproducts/synergos/ (accessed 1 March 2024)

3. Federated Learning and the Personal Data Protection Act

One of the most significant advantages FL is thought to have is to ensure privacy preservation for the purposes of data protection legislation. However, as far as the PDPA is concerned, this claim seems doubtful.

To answer the question of how the PDPA applies to FL, it will be assumed that the organisations engaged in the FL project are subject to the PDPA and that the data contained in the data silos is personal data within the meaning of the PDPA. The PDPA Part 4 sets out various obligations for organisations engaged in the “collection, use or disclosure” of personal data.²Section 13 PDPA Thus, the first question is whether FL involves the collection, use or disclosure of personal data.

Different cases need to be distinguished. If personal data is specifically collected for the purpose of using it for FL, then this will constitute collection, use or disclosure. Another case is where the data sets already exist, and the question arises whether using this data for FL constitutes collection, use or disclosure of personal data. In relation to the EU’s General Data Protection Regulation (GDPR) it was argued that “the processing operation performed [by FL] on … data (e.g. data alignment, data processing etc.) are likely to qualify as processing”.³S. Rossello, R. Díaz Morales, L. Muñoz-González, “Data Protection by design in AI? The case of federated learning”, 2021, Computerrecht: tijdschrift voor informatica en recht, page 9 One may argue that a similar conclusion holds in relation to the PDPA. Although in FL data is not transmitted from one location to another, the training of the algorithm locally will usually involve some form of data alignment or data labelling, i.e. activities which can be considered “use” of personal data for the purpose of the PDPA. If this is the correct interpretation of the PDPA, it follows that if FL is used to train an AI system on personal data, either actual or deemed consent is required, unless an exemption, e.g. the Business Improvement or Research exception, can be relied on.⁴See part III of PDPC, “Advisory Guidelines on use of Personal Data in AI Recommendation and Decision Systems”, 2024, https://www.pdpc.gov.sg/-/media/files/pdpc/pdf-files/advisory-guidelines/advisory-guidelines-on-the-use-of-personal-data-in-ai-recommendation-and-decision-systems.pdf (accessed 7 May 2024)

Yet, it would be wrong to conclude that FL makes no difference to the analysis of how the PDPA applies. The PDPA does not make reference to AI (or similar concepts like the training of algorithms) but the Personal Data Protection Commission (PDPC) issued the very helpful “Advisory Guidelines on use of Personal Data in AI Recommendation and Decision Systems” (the AI Guidelines).⁵AI Guidelines , note 5 The AI Guidelines make it clear that when developing AI systems, organisations should adhere to the principles of data minimisation⁶Para 7.1 AI Guidelines note 5 and pay special attention to data security.⁷Para 7.3 AI Guidelines note 5 In FL only the trained algorithms or the relevant parameters of the trained algorithm are transmitted. In contrast, in standard training of AI systems, the data is transmitted or copied to a central repository. Therefore, FL minimises data as compared to the standard training of AI systems.⁸The same conclusion is reached in relation to the GDPR by S. Rossello, R. Díaz Morales, L. Muñoz-González, note 4, page 8 Furthermore, in FL, the data is not held in one location but remains dispersed over the data silos and this should lower the risk of data breaches.⁹S. Rossello, R. Díaz Morales, L. Muñoz-González, note 4, page 8

There is also a further difference. It was assumed for the purposes of the above analysis that the data in the data silos is personal data; however, this does not necessarily mean that the AI system is trained on personal data. Assume the data silos contain x-ray images and each image is linked to patients’ IC numbers and names. Yet, assume that the algorithm is only trained on the x-ray images, and is incapable of detecting the link between the x-ray images and the IC numbers and names. An argument can be made that the relevant data which is collected, used or disclosed is not the entire data in the data silo but only the data which the algorithm can be trained on (i.e. the x-ray images). If this is correct, then in relation to FL, the relevant question for whether the PDPA applies is whether the data on which the algorithm is trained is personal data and not whether the data in the data silo is personal data. Thus, although FL does not guarantee compliance with the PDPA, FL can be a powerful tool to assist organisations to comply with the PDPA.

4. Alternative Analysis

It is submitted that an alternative analysis to the one given above is also possible. The AI Guidelines make a distinction between the development stage and the deployment stage of AI systems.¹⁰See parts III and IV AI Guidelines note 5 This distinction is to be welcomed because the data protection issues in the two stages are very different. The goal of training AI systems is not so much to draw inferences about the data on which the algorithm was trained but rather to apply the algorithm to data outside the training data set. Concerns about personal data are relevant at the stage when AI systems are deployed because at this stage inferences may be drawn about identifiable individuals but not during the training stage. ¹¹A similar conclusion was reached by the EU Parliament research report. See European Parliament, “The impact of the General Data Protection Regulation (GDPR) on artificial intelligence” (2020) STUDY Panel for the Future of Science and Technology EPRS | European Parliamentary Research Service, pages 46-47 Further, one could point to the stated aim of the PDPA to strike a balance between the legitimate data protection needs of individuals and the needs of organisations to collect, use or disclose data.¹²Section 3 PDPA Thus, an argument could be made that, provided it is not reasonably possible for individuals to be identified, then the training of the AI system in itself does not interfere with an individual’s legitimate data protection interests, even if the training is on personal data and involves data alignment, data labelling etc. For FL this would mean that the relevant question in relation to PDPA is whether FL prevents individuals from being identified, and not whether the algorithm is trained on personal data.¹³However, note that research suggests that individuals may be identified even if FL is used. See Lyu, H. Yu and Q. Yang, “Threats to Federated learning: A Survey”, 2020, section 4, available at https://arxiv.org/pdf/2003.02133 (accessed 2 May 2024). See also S. Rossello, R. Díaz Morales, L. Muñoz-González, note 4, page 4

However, the PDPC seems to take a different view. The AI Guidelines state that: ¹⁴Paragraphs 2.1 AI Guidelines note 8

“The PDPA is broad-based legislation that applies to all collection and use of personal data by an organisation, including the collection and/or processing of personal data to develop, test and monitor AI Systems, or as part of their deployment process.”

The AI Guidelines do not specifically address whether the training of AI systems per se constitutes collection, use or disclosure of personal data, nevertheless, the statement above seems to suggest that it does. This is a reasonable and defensible interpretation of the PDPA. However, for the reasons given above, a different interpretation of the PDPA such that the training of AI systems on personal data per se does not constitute collecting, using or disclosing of personal data, also seems possible.

It is submitted that this touches on a fundamental question, namely what interests the PDPA is meant to protect. In standard training of AI the training data can be accessed by the developers of the AI system. If the data is personal data, it is important to have legislation in place to prevent developers from misusing this data. For instance, by drawing inferences about individuals, whose personal data is in the data set. However, to the extent FL effectively prevents developers from accessing the training data or draw inferences about individuals, whose personal data is in the training set, an argument could be made that there is no need to protect the data from misuse by the developers because the developers cannot access the data.¹⁵Note that FL does not guarantee that no inferences can be drawn about the local data sets. See, See Lyu, H. Yu and Q. Yang, note 14 and S. Rossello, R. Díaz Morales, L. Muñoz-González, note 4, page 4

The fundamental question seems to be the following: Does the training of AI systems on personal data per se constitute a potential infringement of an individual’s legitimate interest to protect their personal data? Or does the training of AI systems only infringe an individual’s legitimate interest to protect their personal data if inferences can be drawn about said individual or there is a risk thereof. In standard AI training this question is unlikely to arise because the training is likely to expose individuals to risk of inferences being drawn. Yet, in FL this is not the case, and the question about what interests the PDPA seeks to protect needs to be answered.

5. Transferring Data Overseas

Another aspect where FL may impact how the PDPA applies is in relation to the Transfer Limitation Obligation pursuant to section 26 of the PDPA. For the standard training of AI systems the issue is relatively straightforward. If personal data is held in Singapore and the data is copied or transmitted to another country for the training of an AI system, then this will constitute transmitting data overseas and section 26 PDPA will apply.

For FL the analysis is more complex because the word transfer is not defined in the PDPA, the AI Guidelines or the PDPC “Guidelines on Key Concepts in the Personal Data Protection Act” (the PDPA Guidelines).¹⁶Advisory Guidelines on Key Concepts in the Personal Data Protection Act, issued 23 September 2013, revised 16 May 2022, https://www.pdpc.gov.sg/-/media/files/pdpc/pdf-files/advisory-guidelines/ag-on-key-concepts/advisory-guidelines-on-key-concepts-in-the-pdpa-17-may-2022.pdf (accessed 7 May 2024) Assume a scenario where personal data is held in a data silo in Singapore, and an algorithm is trained on this data which was transmitted from a server located overseas and transmitted back to that server. An argument could be made that if no individual can be identified from the algorithm which is transmitted back to the server, then no personal data is transferred abroad. On the other hand, one could point to the PDPA Guidelines, which states that term “disclosing” includes “making available”¹⁷Para 7.2. (c) page 31 PDPA Guidelines note 16, and argue that the term “transfer” should also be read to include “making data available”. Thus, even if the trained algorithm itself is not personal data, FL may make personal data held in Singapore available to an overseas organisation, and therefore the personal data may be considered as transferred abroad.

It is submitted that the latter approach seems more reasonable. The AI Guidelines state that the “Transfer Limitation Obligation is a manifestation of the Accountability Obligation”¹⁸Para 7.2. (c) page 31 PDPA Guidelines note 16 thus, the onus is on the organisation that holds personal data to ensure that the personal data is treated in accordance with the PDPA. If a Singaporean organisation allows personal data it holds to be used by an overseas organisation to train an algorithm then the Singaporean organisation needs to ensure that the training complies with the PDPA. The Singaporean organisation should not be allowed to avoid liability as this may leave individuals without an effective remedy. However, similar to the conclusion in relation to the applicability of the PDPA, this does not mean that FL is not relevant to the legal assessment of the Transfer Limitation Obligation. In contrast to standard training of AI, where data is transmitted or copied to an overseas location, FL only makes data available to an overseas entity by allowing the overseas entity to transmit an algorithm to train on that data, but the data remains in Singapore. This should mean that the data is more secure from abuse than in standard training of AI.

6. Other Legal Issues and Fairness in AI

It is important to note that, apart from data protection issues, FL raises a host of other legal questions, not least in relation to IP law. Issues of IP law and AI are complex and questions of whether the training of AI systems may infringe copyright are currently subject to litigation.¹⁹See for instance, the case of New York Times v Microsoft and OpenAI, Case 1:23-cv-11195. In relation to Singapore note Division 8 Copyright Act 2021 Other IP law questions in relation to AI relate to whether AI can be an inventor for patent law purposes,²⁰See for instance, the Australian case Thaler v Commissioner of Patents (2021) FCA 879, contrast with the UK case Thaler v Comptroller-General of Patents, Designs and Trade Marks (2023) UKSC 49 or whether an AI system can be patented.²¹For an overview see for instance, IPOS, IP and Artificial Intelligence, Information Note, https://www.ipos.gov.sg/docs/default-source/default-document-library/ip-and-ai-info-note.pdf (accessed 28 March 2024) Due to size constrains it is not possible to explore these issues in this article. However, it is important to note that FL may change the legal analysis of these questions. For instance, FL may be relevant for the question of whether the training of an AI system involves making a “copy of an authorial work”²²Section 41(1) Copyright Act 2021 (speech marks omitted) for copyright purposes.

Another important issue is AI fairness. Currently, Singapore does not have specific legislation for AI but it does have the voluntary “Model Artificial Intelligence Governance Framework”,²³PDPC, Model Artificial Intelligence Governance Framework, Second Edition, Jan 2020, https://www.pdpc.gov.sg/-/media/files/pdpc/pdf-files/resource-for-organisation/ai/sgmodelaigovframework2.pdf, accessed 13 Mar 2024, (the Governance Framework) which states that “[o]rganisations using AI in decision-making should ensure that the decision-making process is explainable, transparent and fair.”²⁴Governance Framework, note 24, page 15 In FL the training data is not accessible to the developers of the AI system, which makes it more difficult than in standard training of AI to check for biases in the training data and to ensure transparency and explainability.²⁵Nguyen Truong, Kai Sun, Siyao Wang, Florian Guitton, Yi Ke Guo, “Privacy Preservation in Federated Learning: An Insightful Survey from the GDPR perspective”, (2021) Computer & Security, volume, 110, page 15

Good governance structures and good record keeping may help to ameliorate some of these problems. For instance, in a FL project where different organisations contribute data (e.g. different hospitals contributing data to an FL project to improve patient care) the governance structure needs to clearly set out which entity is responsible for the overall FL project, including explainability, transparency and fairness, and which entity is responsible for which local data set. There also needs to be a requirement that the data in the local data sets meet certain quality and fairness standards. Further, there needs to be a mechanism which allows the entity in charge of the FL project to check compliance with the required standards. Participating entities in the FL project should keep accurate records detailing how the data was collected and provide key statistical parameters of the data set (e.g. size of date sets, distribution, if applicable key demographic information (e.g. race, gender) etc. to the entity in charge of the FL project. The entity in charge should also check that the algorithms which are transmitted back from the data silos are within a statistically reasonable range. In addition, the entity in charge may transmit algorithms to the data silos to confirm the key statistical properties of the data set.²⁶See also Annie Abay, Yi Zhou, Nathalie Baracaldo, Shashank Rajamoni, Ebube Chuba, Heiko Ludwig “Mitigating Bias in Federated Learning”, (2020) available at https://arxiv.org/abs/2012.02447 (accessed 5 May 2024) Nevertheless, even with the appropriate measures in place, explainability, transparency and fairness will be a difficult issue for FL. As there seems to be a privacy – fairness trade off because the more the entity in charge of the FL project knows about the local data sets the easier it is to monitor for bias and the ensure fairness but the less private the data sets will be.²⁷Peter Kairouz et al. , Advances and Open Problems in Federated Learning, 2021, https://arxiv.org/abs/1912.04977 (accessed 07 May 2024), page 77

7. Organisational Issues

It is important to note that FL requires an element of trust. For instance, there needs to be trust that the algorithm which is transmitted to the data silos does not contain spyware, or that the entity which controls the central server does not try to draw inferences about the data silos from the individually trained algorithms after they are transmitted back.²⁸See S. Rossello, R. Díaz Morales, L. Muñoz-González note 4, page 6 Conversely, the controller of the central server, needs to trust that the participants in the FL project do not engage in ‘data poisoning’ (i.e. deliberating submitting wrong data to corrupt the final model).²⁹See S. Rossello, R. Díaz Morales, L. Muñoz-González note 4, pages 5-6 Technological solutions are available to ameliorate some of these issues, but lawyers can add value at this stage, by creating an appropriate governance structure and contractual provisions to deal with the trust issues.

8. What is FL’s Potential?

It is important to recognise that legislation like the PDPA or IP law are not the only reason that makes data sharing difficult. Another important reason is that organisations may be unwilling to share data due to concerns that competitors may gain a commercial advantage through this data. For instance, competitor companies may be reluctant share customer data because making customer data available to competitors carries the risk that competitors may “poach” clients. FL may be a way to overcome this problem because in FL the data does not leave the company’s servers, which means that there is less risk that the data is used to harm the commercial interests of the company. Thus, FL may enable companies to train algorithms jointly on their data, even if the companies are competitors. Yet, as of time of writing, it seems that this has not happened to a significant degree.

The term FL was coined in 2016,³⁰Peter Kairouz et al., note 28,page 4 and since then FL was applied to multiple use cases. For instance, Google uses FL for its Gboard mobile keyboard and in Android Messages, and Apple uses it in iOS13.³¹Peter Kairouz et al., note 28, page 5 Yet, these are all examples of intra-organisational use of FL, but thus far there has not been a significant utilisation of FL between competitor organisations to jointly train models, e.g. Google and Apple training algorithms jointly.³²However, note that Swiss Re and WeBank signed a MoU to explore the use of FL in the reinsurance sector. See, Business Insider, WeBank and Swiss Re Signed Cooperation MOU, 2019, https://markets.businessinsider.com/news/stocks/webank-and-swiss-re-signed-cooperation-mou-1028228738 (accessed 7 May 2024) The analysis as to why this is the case is outside the scope of article, however, the trust issues mentioned above may play a part in this. Often the models used in the relevant computer science literature assume the existence of a trusted third party, but in the real world such parties may not exist. This is an area where regulation may be helpful. If private actors are unable to create a regime to establish the trust necessary for FL, the state by providing an appropriate regulatory or licensing framework may be able to address some of these issues. Alternatively, a public body could be appointed as data intermediary and fulfil the role of trusted third party.

9. Conclusion

This article highlights some of the legal issues raised by FL, in particular, the question of how PDPA applies to FL. As this article made clear many of the legal issues are, as of yet, untested and reasonable legal opinion may differ on the proper treatment of some these issues. Therefore, it will be fruitful to have a debate among the Singaporean legal community how these issues are to be resolved, especially because some of them touch on some fundamental questions as to how data protection applies not only to FL but to AI generally. FL is an area where lawyers and AI practitioners need to work together to achieve data sharing while protecting personal and commercially sensitive data.

Acknowledgement

This research/project is supported by the National Research Foundation Singapore and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-RP-2020-018).

The post Federated Learning: What Lawyers Need to Know appeared first on The Singapore Law Gazette.