Ethical Real-world Experimentation – A collaboration between the Province of South-Holland (PZH) & the TU Delft

Public and private institutions routinely test new digital technologies within society, such as generative AI, predictive policing algorithms, and self-driving cars under real-world conditions. These experimental practices raise questions about research ethics and the moral responsibility of investigators to those people they involve and affect. However, clear moral norms or protection protocols are often lacking or insufficient. Joost Mollen’s research project - in collaboration with the Province of South Holland - focusses on addressing this gap.

We take the opportunity to interview both Ivonne and Joost on this fascinating topic, and on how this research on the ethics of real-world experimentation with emerging digital technologies could help the province of South Holland to govern such experiments.

Joost: Can you tell us more about your research?

The target of my research concerns the ethics of real-world experimentation with emerging digital technologies. These are, for example, experiments conducted with the use of sensors to measure and manage crowds, or the use of satellite images to monitor buildings of businesses. I am interested in the ethical issues in researching and developing new digital technologies under real-world conditions, specifically concerning the people affected by and involved in such experimentation. Real-world experimentation is an important strategy for developing robust and responsible emerging technologies such as AI, robotics, and smart city applications. However, the ethics of these experimentations are often unaddressed and unregulated. While real-world experimentation might benefit the development of responsible digital technologies or help solve ‘grand social challenges,’ attention should be paid to conducting such experimentation responsibly.

Ivonne: where do we see such experiments or tests in the province of South-Holland?

We conduct such experiments in our 'Fieldlabs'. For example, there is the Living Lab Scheveningen (LLS). This is a lab and learning area on the boulevard of Scheveningen. Here small-scale experiments are conducted with digital innovations in practice. These include crowd measurements for crowd management, a registration system of incoming and outgoing ships for the Port of Scheveningen, a smart energy grid in which sustainably generated energy is better distributed and smart waste bins.

Ivonne: Why did PZH want this research collaboration?

Governments are becoming increasingly dependent on the extent to which technology plays a role in shaping society and the way in which they make policy. PZH is a knowledge-intensive organisation, where a lot of policy is being made. Within the province, we noticed that more and more attention is needed for the ethical issues in the application of technology. We believe it is important for policymakers to build knowledge about ethics so we can embed ethics by design in our policy, processes and the choices we make.

Ivonne: How was this research project started?

I joined PZH in 2018 as a strategic advisor on ethics and digital information. It is quite unique that the PZH was already working on this subject, or public values as the theme is now called. PZH is a frontrunner here. I was living in Amsterdam at the time and moved to The Hague. I was new to the region and, based on my interest and new position, I immediately started looking for parties in the region that are working on this theme. That's when I quickly came across Professor Jeroen van den Hoven. Together with Henk Burgering, project leader of digital Zuid-Holland, we decided to start a collaboration with the TU Delft on the theme ethics by design. We wanted to know more about how we, as an organization, could apply ethics by design and how we could better integrate ethics. This lead to the start of a PhD project. Joost was hired and further shaped his research. He wanted to focus on the ethical aspects of smart city experiments.

Ivonne: why is ethics by design so important for the province of South-Holland?

For the General Data Protection Act (GDPR), many processes have been set up to ensure that we comply with the law. We have an entire privacy unit within the organization. A formal operational structure has been created. We don't have that for ethics. This makes it more difficult to embed ethics in procedures. However, steps are being made. We carry out risk analyses on an increasingly structural basis. Data ethics is a formal part of data management and is embedded in processes to a certain extent. At the same time, ethics is not only about whether you do things right, but also about whether you do the right things. It's also about dialogue and an open culture.

Joost: Back to your research: what ethical problems might arise when experimenting in a real-world environment?

Well, since real-world experiments are conducted close to people's daily lives or environment and can actively intervene within them, they potentially cause undue influence, impact, or harm. People can become (un)knowingly or (un)desirably involved in these experiments since they sometimes become indiscernible from daily life. However, these practices are often not covered by existing research ethics protocols, as is the case with scientific research on human subjects.

Additionally, when real-world experiments with emerging technology are conducted by public and private parties in transdisciplinary collaborations, for example, in so-called living labs or real-world laboratories, research ethics responsibilities are often distributed unevenly. The scope of research ethics regulations often stops at the scientific side of these collaborations.

Furthermore, even if current regulations would be extended, a question arises regarding their applicability. For example, in current research ethics regulation, research participation is understood as a free and voluntary activity, meaning participation is a choice. However, as soon as you move from a controlled research environment to an uncontrolled environment, it becomes difficult to maintain this since researchers often do not have a clear idea of who becomes involved or affected by their test.

Take a test with self-driving vehicles on public roads or an experiment with predictive policing in a public nightlife street. In these examples, it becomes difficult to uphold established research ethical obligations, such as acquiring informed consent, offering additional protections when involving vulnerable groups, such as elderly people or children, or ensuring some just distribution of benefits and risks. Additionally, it becomes difficult for people to control whether or not they are participating in an experiment.

Ivonne: To what extent are employees open to exploring such ethical questions?

They are very open to this. Policymakers already are critical and focused on dialogue. The fact that there is now explicit attention for Ethics by design helps to formulate questions that were already there, but for which there previously was no room to ask them. For example, a question that I personally have is whether more data leads to better policy. Certain skills and domain knowledge are very important for policymaking. In policymaking processes, you learn that it requires bringing in the right knowledge at the exact right time. A multitude of considerations and interests need to come together. I admire the care, prudence and modesty that this requires. It is sometimes thought that policymakers will no longer be needed as much because of AI. That we find answers and can steer by combining as much data as possible. I believe - after having worked as a civil servant for a while - that that is impossible. 

Ivonne: What does the province of South Holland already do when it comes to ethics by design?

A lot has already been achieved during the research project, also in collaboration with TU Delft. For example, an ethics committee has been set up within the Interprovincial association (IPO) with Jeroen van den Hoven as chairman and researcher Roel Dobbe. They advise on ethics by design issues. Another example is a structure that we have set up internally: the ethics thinktank. From the thinktank, we identify issues within the organization and look at interventions, such as moral deliberation, guidance ethics, or an Impact Assessment Human Rights and Algorithms (IAMA). We invite the stakeholders to the session. Examples of topics covered are ‘critical cartography’, 'open government' and 'guidelines for models'. Joost's research on using the public space for experimentation will be discussed in February, so we can further safeguard the impact of the results of his research. 

Joost: How do you think your research can further contribute and help institutions?

A first step is awareness that certain ethical issues are at stake. A discussion is necessary about how and for what reasons this experimentation is conducted and not whether. This awareness, in turn, can help public institutions have conversations about what specific ethical questions might arise and how they can address or mitigate them in the experiments with technology that they either participate in or allow in their public spaces. Real-world experimentation with emerging technologies should be thought of and organized as a formalized learning process, subject to epistemic and ethical standards.

Alternatively, public institutions could look at how research ethics is organized in other domains and adopt analogous regulatory protocols. Scientific research has a thorough process for assessing research proposals when human subjects are involved. Similarly, internal or independent ethical commissions can promote ethical discussion, set out ethical guidelines for best practices, or make real-world experimentation conditional to certain requirements. This way, ethics can be incorporated ‘by-design’ into the policies that encourage and enable real-world experimentation.

The EU’s new AI Act is a recent example of this. It allows for testing high-risk AI in so-called regulatory sandboxes, in which risky artificial intelligence can be tested under real-world conditions under the supervision of a regulatory agency. Additionally, it stipulates certain criteria for when public and private developers want to experiment under real-world conditions outside this regulatory framework, such as consent and the exclusion of vulnerable groups. This regulation is not perfect, but it is a first step.

Ivonne: How do you believe PZH could benefit from the research outcome?

I am pleasantly surprised by Joost's approach: public space as an experimental space. What kind of voice does the citizen have? Is informed consent even possible? What we hear a lot about now -when it comes to digital technologies- is privacy, autonomy, sovereignty and control over AI. Joost's perspective is an important addition to the discussion: these experiments have an effect on people, but can they give their consent? From this research, we can learn a lot about how we can responsibly set up the governance for field labs.

The short lines of communication and the expertise available at TU Delft are very useful as well. We are organising a work visit to TU Delft for the deputies Frederik Zevenburgen and Frank Rijkaart. There we will reflect on this research, but we can also utilize further expertise. The big questions are around the administrative action perspective. How can ethics be embedded in processes and what is the control mechanism? During the working visit, we hope to learn more about governance in a data-driven era.

Joost: how has it been for you to be working with the Province so far?

It has been a very interesting and informative process! Everyone is very welcoming, and it has been very educational to be involved in their projects and see how public officials in the Province discuss and deal with data ethical challenges in their work.

Ivonne: What was it like for you to have a researcher on board?

An academic perspective is different from a policy one, that's where the value lies. It was useful that Joost could be so close to our projects and it was nice to have a sounding board. I look forward to working together to see how the research results can be embedded!