Ethical Guidelines for Trustworthy Artificial Intelligence (AI)

Ethical Guidelines for Trustworthy AI

AID:

Category:

wdt_ID	AID	AIID	Category	Assessment	Guidance	Recommendation
1	1	1.1	Human Agency and Oversight	Fundamental Rights	Like many technologies, AI systems can equally enable and hamper fundamental rights. They can benefit people for instance by helping them track their personal data, or by increasing the accessibility of education, hence supporting their right to education. However, given the reach and capacity of AI systems, they can also negatively affect fundamental rights. In situations where such risks exist, a fundamental rights impact assessment should be undertaken. This should be done prior to the system’s development and include an evaluation of whether those risks can be reduced or justified as necessary in a democratic society in order to respect the rights and freedoms of others. Moreover, mechanisms should be put into place to receive external feedback regarding AI systems that potentially infringe on fundamental rights.	◾Did you carry out a fundamental rights impact assessment where there could be a negative impact on fundamental rights? Did you identify and document potential trade-offs made between the different principles and rights? ◾Does the AI system interact with decisions by human (end) users (e.g. recommended actions or decisions to take, presenting of options)?  Could the AI system affect human autonomy by interfering with the (end) user’s decision-making process in an unintended way?  Did you consider whether the AI system should communicate to (end) users that a decision, content, advice or outcome is the result of an algorithmic decision?  In case of a chat bot or other conversational system, are the human end users made aware that they are interacting with a non-human agent?
2	2	1.2	Human Agency and Oversight	Human Agency and Autonomy	Users should be able to make informed autonomous decisions regarding AI systems. They should be given the knowledge and tools to comprehend and interact with AI systems to a satisfactory degree and, where possible, be enabled to reasonably self-assess or challenge the system. AI systems should support individuals in making better, more informed choices in accordance with their goals. AI systems can sometimes be deployed to shape and influence human behaviour through mechanisms that may be difficult to detect, since they may harness sub-conscious processes, including various forms of unfair manipulation, deception, herding and conditioning, all of which may threaten individual autonomy. The overall principle of user autonomy must be central to the system’s functionality. Key to this is the right not to be subject to a decision based solely on automated processing when this produces legal effects on users or similarly significantly affects them.	Is the AI system implemented in work and labour process? If so, did you consider the task allocation between the AI system and humans for meaningful interactions and appropriate human oversight and control?  Does the AI system enhance or augment human capabilities?  Did you take safeguards to prevent overconfidence in or overreliance on the AI system for work processes?
3	3	1.3	Human Agency and Oversight	Human Oversight	Human oversight helps ensuring that an AI system does not undermine human autonomy or causes other adverse effects. Oversight may be achieved through governance mechanisms such as a human-in-the-loop (HITL), human-on-the-loop (HOTL), or human-in-command (HIC) approach. HITL refers to the capability for human intervention in every decision cycle of the system, which in many cases is neither possible nor desirable. HOTL refers to the capability for human intervention during the design cycle of the system and monitoring the system’s operation. HIC refers to the capability to oversee the overall activity of the AI system (including its broader economic, societal, legal and ethical impact) and the ability to decide when and how to use the system in any particular situation. This can include the decision not to use an AI system in a particular situation, to establish levels of human discretion during the use of the system, or to ensure the ability to override a decision made by a system. Moreover, it must be ensured that public enforcers have the ability to exercise oversight in line with their mandate. Oversight mechanisms can be required in varying degrees to support other safety and control measures, depending on the AI system’s application area and potential risk. All other things being equal, the less oversight a human can exercise over an AI system, the more extensive testing and stricter governance is required.	◾Did you consider the appropriate level of human control for the particular AI system and use case?  Can you describe the level of human control or involvement?  Who is the “human in control” and what are the moments or tools for human intervention?  Did you put in place mechanisms and measures to ensure human control or oversight?  Did you take any measures to enable audit and to remedy issues related to governing AI autonomy? ◾Is there is a self-learning or autonomous AI system or use case? If so, did you put in place more specific mechanisms of control and oversight?  Which detection and response mechanisms did you establish to assess whether something could go wrong?  Did you ensure a stop button or procedure to safely abort an operation where needed? Does this procedure abort the process entirely, in part, or delegate control to a human?
4	4	2.1	Technical Robustness and Safety	Resilience to Attack and Security	AI systems, like all software systems, should be protected against vulnerabilities that can allow them to be exploited by adversaries, e.g. hacking. Attacks may target the data (data poisoning), the model (model leakage) or the underlying infrastructure, both software and hardware. If an AI system is attacked, e.g. in adversarial attacks, the data as well as system behaviour can be changed, leading the system to make different decisions, or causing it to shut down altogether. Systems and data can also become corrupted by malicious intention or by exposure to unexpected situations. Insufficient security processes can also result in erroneous decisions or even physical harm. For AI systems to be considered secure, possible unintended applications of the AI system (e.g. dual-use applications) and potential abuse of the system by malicious actors should be taken into account, and steps should be taken to prevent and mitigate these.	◾Did you assess potential forms of attacks to which the AI system could be vulnerable?  Did you consider different types and natures of vulnerabilities, such as data pollution, physical infrastructure, cyber-attacks? ◾Did you put measures or systems in place to ensure the integrity and resilience of the AI system against potential attacks? ◾Did you verify how your system behaves in unexpected situations and environments? ◾Did you consider to what degree your system could be dual-use? If so, did you take suitable preventative measures against this case (including for instance not publishing the research or deploying the system)?
5	5	2.2	Technical Robustness and Safety	Fallback Plan and General Safety	AI systems should have safeguards that enable a fallback plan in case of problems. This can mean that AI systems switch from a statistical to rule-based procedure, or that they ask for a human operator before continuing their action. It must be ensured that the system will do what it is supposed to do without harming living beings or the environment. This includes the minimisation of unintended consequences and errors. In addition, processes to clarify and assess potential risks associated with the use of AI systems, across various application areas, should be established. The level of safety measures required depends on the magnitude of the risk posed by an AI system, which in turn depends on the system’s capabilities. Where it can be foreseen that the development process or the system itself will pose particularly high risks, it is crucial for safety measures to be developed and tested proactively.	◾Did you ensure that your system has a sufficient fallback plan if it encounters adversarial attacks or other unexpected situations (for example technical switching procedures or asking for a human operator before proceeding)? ◾Did you consider the level of risk raised by the AI system in this specific use case?  Did you put any process in place to measure and assess risks and safety?  Did you provide the necessary information in case of a risk for human physical integrity?  Did you consider an insurance policy to deal with potential damage from the AI system?  Did you identify potential safety risks of (other) foreseeable uses of the technology, including accidental or malicious misuse? Is there a plan to mitigate or manage these risks? ◾Did you assess whether there is a probable chance that the AI system may cause damage or harm to users or third parties? Did you assess the likelihood, potential damage, impacted audience and severity?  Did you consider the liability and consumer protection rules, and take them into account?  Did you consider the potential impact or safety risk to the environment or to animals?  Did your risk analysis include whether security or network problems such as cybersecurity hazards could pose safety risks or damage due to unintentional behaviour of the AI system? ◾Did you estimate the likely impact of a failure of your AI system when it provides wrong results, becomes unavailable, or provides societally unacceptable results (for example discrimination)?  Did you define thresholds and did you put governance procedures in place to trigger alternative/fallback plans?  Did you define and test fallback plans?
6	6	2.3	Technical Robustness and Safety	Accuracy	Accuracy pertains to an AI system’s ability to make correct judgements, for example to correctly classify information into the proper categories, or its ability to make correct predictions, recommendations, or decisions based on data or models. An explicit and well-formed development and evaluation process can support, mitigate and correct unintended risks from inaccurate predictions. When occasional inaccurate predictions cannot be avoided, it is important that the system can indicate how likely these errors are. A high level of accuracy is especially crucial in situations where the AI system directly affects human lives.	◾Did you assess what level and definition of accuracy would be required in the context of the AI system and use case?  Did you assess how accuracy is measured and assured?  Did you put in place measures to ensure that the data used is comprehensive and up to date?  Did you put in place measures in place to assess whether there is a need for additional data, for example to improve accuracy or to eliminate bias? ◾Did you verify what harm would be caused if the AI system makes inaccurate predictions? ◾Did you put in place ways to measure whether your system is making an unacceptable amount of inaccurate predictions? ◾Did you put in place a series of steps to increase the system's accuracy?
7	7	2.4	Technical Robustness and Safety	Reliability and Reproducibility	It is critical that the results of AI systems are reproducible, as well as reliable. A reliable AI system is one that works properly with a range of inputs and in a range of situations. This is needed to scrutinise an AI system and to prevent unintended harms. Reproducibility describes whether an AI experiment exhibits the same behaviour when repeated under the same conditions. This enables scientists and policy makers to accurately describe what AI systems do. Replication files40 can facilitate the process of testing and reproducing behaviours.	◾Did you put in place a strategy to monitor and test if the AI system is meeting the goals, purposes and intended applications?  Did you test whether specific contexts or particular conditions need to be taken into account to ensure reproducibility?  Did you put in place verification methods to measure and ensure different aspects of the system's reliability and reproducibility?  Did you put in place processes to describe when an AI system fails in certain types of settings?  Did you clearly document and operationalise these processes for the testing and verification of the reliability of AI systems?  Did you establish mechanisms of communication to assure (end-)users of the system’s reliability?
8	8	3.1	Privacy and Data Governance	Privacy and Data Protection	AI systems must guarantee privacy and data protection throughout a system’s entire lifecycle. This includes the information initially provided by the user, as well as the information generated about the user over the course of their interaction with the system (e.g. outputs that the AI system generated for specific users or how users responded to particular recommendations). Digital records of human behaviour may allow AI systems to infer not only individuals’ preferences, but also their sexual orientation, age, gender, religious or political views. To allow individuals to trust the data gathering process, it must be ensured that data collected about them will not be used to unlawfully or unfairly discriminate against them.	◾Depending on the use case, did you establish a mechanism allowing others to flag issues related to privacy or data protection in the AI system’s processes of data collection (for training and operation) and data processing? ◾Did you assess the type and scope of data in your data sets (for example whether they contain personal data)? ◾Did you consider ways to develop the AI system or train the model without or with minimal use of potentially sensitive or personal data? ◾Did you build in mechanisms for notice and control over personal data depending on the use case (such as valid consent and possibility to revoke, when applicable)? ◾Did you take measures to enhance privacy, such as via encryption, anonymisation and aggregation? ◾Where a Data Privacy Officer (DPO) exists, did you involve this person at an early stage in the process?
9	9	3.2	Privacy and Data Governance	Quality and integrity of data	The quality of the data sets used is paramount to the performance of AI systems. When data is gathered, it may contain socially constructed biases, inaccuracies, errors and mistakes. This needs to be addressed prior to training with any given data set. In addition, the integrity of the data must be ensured. Feeding malicious data into an AI system may change its behaviour, particularly with self-learning systems. Processes and data sets used must be tested and documented at each step such as planning, training, testing and deployment. This should also apply to AI systems that were not developed in-house but acquired elsewhere.	◾Did you align your system with relevant standards (for example ISO, IEEE) or widely adopted protocols for daily data management and governance? ◾Did you establish oversight mechanisms for data collection, storage, processing and use? ◾Did you assess the extent to which you are in control of the quality of the external data sources used? ◾Did you put in place processes to ensure the quality and integrity of your data? Did you consider other processes? How are you verifying that your data sets have not been compromised or hacked?
10	10	3.3	Privacy and Data Governance	Access to Data	In any given organisation that handles individuals’ data (whether someone is a user of the system or not), data protocols governing data access should be put in place. These protocols should outline who can access data and under which circumstances. Only duly qualified personnel with the competence and need to access individual’s data should be allowed to do so.	◾What protocols, processes and procedures did you follow to manage and ensure proper data governance?  Did you assess who can access users’ data, and under what circumstances?  Did you ensure that these persons are qualified and required to access the data, and that they have the necessary competences to understand the details of data protection policy?  Did you ensure an oversight mechanism to log when, where, how, by whom and for what purpose data was accessed?
11	11	4.1	Transparency	Traceability	The data sets and the processes that yield the AI system’s decision, including those of data gathering and data labelling as well as the algorithms used, should be documented to the best possible standard to allow for traceability and an increase in transparency. This also applies to the decisions made by the AI system. This enables identification of the reasons why an AI-decision was erroneous which, in turn, could help prevent future mistakes. Traceability facilitates auditability as well as explainability.	◾Did you establish measures that can ensure traceability? This could entail documenting the following methods:  Methods used for designing and developing the algorithmic system: o Rule-based AI systems: the method of programming or how the model was built; o Learning-based AI systems; the method of training the algorithm, including which input data was gathered and selected, and how this occurred.  Methods used to test and validate the algorithmic system: o Rule-based AI systems; the scenarios or cases used in order to test and validate; o Learning-based model: information about the data used to test and validate.  Outcomes of the algorithmic system: o The outcomes of or decisions taken by the algorithm, as well as potential other decisions that would result from different cases (for example, for other subgroups of users).
12	12	4.2	Transparency	Explainability	Explainability concerns the ability to explain both the technical processes of an AI system and the related human decisions (e.g. application areas of a system). Technical explainability requires that the decisions made by an AI system can be understood and traced by human beings. Moreover, trade-offs might have to be made between enhancing a system's explainability (which may reduce its accuracy) or increasing its accuracy (at the cost of explainability). Whenever an AI system has a significant impact on people’s lives, it should be possible to demand a suitable explanation of the AI system’s decision-making process. Such explanation should be timely and adapted to the expertise of the stakeholder concerned (e.g. layperson, regulator or researcher). In addition, explanations of the degree to which an AI system influences and shapes the organisational decision-making process, design choices of the system, and the rationale for deploying it, should be available (hence ensuring business model transparency).	◾Did you assess:  to what extent the decisions and hence the outcome made by the AI system can be understood?  to what degree the system’s decision influences the organisation’s decision-making processes?  why this particular system was deployed in this specific area?  what the system’s business model is (for example, how does it create value for the organisation)? ◾Did you ensure an explanation as to why the system took a certain choice resulting in a certain outcome that all users can understand? ◾Did you design the AI system with interpretability in mind from the start?  Did you research and try to use the simplest and most interpretable model possible for the application in question?  Did you assess whether you can analyse your training and testing data? Can you change and update this over time?  Did you assess whether you can examine interpretability after the model’s training and development, or whether you have access to the internal workflow of the model?
13	13	4.3	Transparency	Communication	AI systems should not represent themselves as humans to users; humans have the right to be informed that they are interacting with an AI system. This entails that AI systems must be identifiable as such. In addition, the option to decide against this interaction in favour of human interaction should be provided where needed to ensure compliance with fundamental rights. Beyond this, the AI system’s capabilities and limitations should be communicated to AI practitioners or end-users in a manner appropriate to the use case at hand. This could encompass communication of the AI system's level of accuracy, as well as its limitations.	◾Did you communicate to (end-)users – through a disclaimer or any other means – that they are interacting with an AI system and not with another human? Did you label your AI system as such? ◾Did you establish mechanisms to inform (end-)users on the reasons and criteria behind the AI system’s outcomes?  Did you communicate this clearly and intelligibly to the intended audience?  Did you establish processes that consider users’ feedback and use this to adapt the system?  Did you communicate around potential or perceived risks, such as bias?  Depending on the use case, did you consider communication and transparency towards other audiences, third parties or the general public? ◾Did you clarify the purpose of the AI system and who or what may benefit from the product/service?  Did you specify usage scenarios for the product and clearly communicate these to ensure that it is understandable and appropriate for the intended audience?  Depending on the use case, did you think about human psychology and potential limitations, such as risk of confusion, confirmation bias or cognitive fatigue? ◾Did you clearly communicate characteristics, limitations and potential shortcomings of the AI system?  In case of the system's development: to whoever is deploying it into a product or service?  In case of the system's deployment: to the (end-)user or consumer?
14	14	5.1	Diversity, Non-Discrimination and Fairness	Avoidance of Unfair Bias	Data sets used by AI systems (both for training and operation) may suffer from the inclusion of inadvertent historic bias, incompleteness and bad governance models. The continuation of such biases could lead to unintended (in)direct prejudice and discrimination against certain groups or people, potentially exacerbating prejudice and marginalisation. Harm can also result from the intentional exploitation of (consumer) biases or by engaging in unfair competition, such as the homogenisation of prices by means of collusion or a non-transparent market. Identifiable and discriminatory bias should be removed in the collection phase where possible. The way in which AI systems are developed (e.g. algorithms’ programming) may also suffer from unfair bias. This could be counteracted by putting in place oversight processes to analyse and address the system’s purpose, constraints, requirements and decisions in a clear and transparent manner. Moreover, hiring from diverse backgrounds, cultures and disciplines can ensure diversity of opinions and should be encouraged.	◾Did you establish a strategy or a set of procedures to avoid creating or reinforcing unfair bias in the AI system, both regarding the use of input data as well as for the algorithm design?  Did you assess and acknowledge the possible limitations stemming from the composition of the used data sets?  Did you consider diversity and representativeness of users in the data? Did you test for specific populations or problematic use cases?  Did you research and use available technical tools to improve your understanding of the data, model and performance?  Did you put in place processes to test and monitor for potential biases during the development, deployment and use phase of the system? ◾Depending on the use case, did you ensure a mechanism that allows others to flag issues related to bias, discrimination or poor performance of the AI system?  Did you establish clear steps and ways of communicating on how and to whom such issues can be raised?  Did you consider others, potentially indirectly affected by the AI system, in addition to the (end)-users? ◾Did you assess whether there is any possible decision variability that can occur under the same conditions?  If so, did you consider what the possible causes of this could be?  In case of variability, did you establish a measurement or assessment mechanism of the potential impact of such variability on fundamental rights? ◾Did you ensure an adequate working definition of “fairness” that you apply in designing AI systems?  Is your definition commonly used? Did you consider other definitions before choosing this one?  Did you ensure a quantitative analysis or metrics to measure and test the applied definition of fairness?  Did you establish mechanisms to ensure fairness in your AI systems? Did you consider other potential mechanisms?
15	15	5.2	Diversity, Non-Discrimination and Fairness	Accessibility and Universal Design	Particularly in business-to-consumer domains, systems should be user-centric and designed in a way that allows all people to use AI products or services, regardless of their age, gender, abilities or characteristics. Accessibility to this technology for persons with disabilities, which are present in all societal groups, is of particular importance. AI systems should not have a one-size-fits-all approach and should consider Universal Design principles addressing the widest possible range of users, following relevant accessibility standards. This will enable equitable access and active participation of all people in existing and emerging computer-mediated human activities and with regard to assistive technologies.	◾Did you ensure that the AI system accommodates a wide range of individual preferences and abilities?  Did you assess whether the AI system usable by those with special needs or disabilities or those at risk of exclusion? How was this designed into the system and how is it verified?  Did you ensure that information about the AI system is accessible also to users of assistive technologies?  Did you involve or consult this community during the development phase of the AI system? ◾Did you take the impact of your AI system on the potential user audience into account?  Did you assess whether the team involved in building the AI system is representative of your target user audience? Is it representative of the wider population, considering also of other groups who might tangentially be impacted?  Did you assess whether there could be persons or groups who might be disproportionately affected by negative implications?  Did you get feedback from other teams or groups that represent different backgrounds and experiences?
16	16	5.3	Diversity, Non-Discrimination and Fairness	Stakeholder Participation	In order to develop AI systems that are trustworthy, it is advisable to consult stakeholders who may directly or indirectly be affected by the system throughout its life cycle. It is beneficial to solicit regular feedback even after deployment and set up longer term mechanisms for stakeholder participation, for example by ensuring workers information, consultation and participation throughout the whole process of implementing AI systems at organisations.	◾Did you consider a mechanism to include the participation of different stakeholders in the AI system’s development and use? ◾Did you pave the way for the introduction of the AI system in your organisation by informing and involving impacted workers and their representatives in advance?
17	17	6.1	Societal and Environmental Well-being	Sustainable and Environmentally Friendly AI	AI systems promise to help tackling some of the most pressing societal concerns, yet it must be ensured that this occurs in the most environmentally friendly way possible. The system’s development, deployment and use process, as well as its entire supply chain, should be assessed in this regard, e.g. via a critical examination of the resource usage and energy consumption during training, opting for less harmful choices. Measures securing the environmental friendliness of AI systems’ entire supply chain should be encouraged.	◾Did you establish mechanisms to measure the environmental impact of the AI system’s development, deployment and use (for example the type of energy used by the data centres)? ◾Did you ensure measures to reduce the environmental impact of your AI system’s life cycle?
18	18	6.2	Societal and Environmental Well-being	Social Impact	Ubiquitous exposure to social AI systems in all areas of our lives (be it in education, work, care or entertainment) may alter our conception of social agency, or impact our social relationships and attachment. While AI systems can be used to enhance social skills, they can equally contribute to their deterioration. This could also affect people’s physical and mental wellbeing. The effects of these systems must therefore be carefully monitored and considered.	◾In case the AI system interacts directly with humans:  Did you assess whether the AI system encourages humans to develop attachment and empathy towards the system?  Did you ensure that the AI system clearly signals that its social interaction is simulated and that it has no capacities of “understanding” and “feeling”? ◾Did you ensure that the social impacts of the AI system are well understood? For example, did you assess whether there is a risk of job loss or de-skilling of the workforce? What steps have been taken to counteract such risks?
19	19	6.3	Societal and Environmental Well-being	Impact on Society and Democracy	Beyond assessing the impact of an AI system’s development, deployment and use on individuals, this impact should also be assessed from a societal perspective, taking into account its effect on institutions, democracy and society at large. The use of AI systems should be given careful consideration particularly in situations relating to the democratic process, including not only political decision-making but also electoral contexts.	Society and democracy: ◾Did you assess the broader societal impact of the AI system’s use beyond the individual (end-)user, such as potentially indirectly affected stakeholders?
20	20	7.1	Accountability	Auditability	Auditability entails the enablement of the assessment of algorithms, data and design processes. This does not necessarily imply that information about business models and intellectual property related to the AI system must always be openly available. Evaluation by internal and external auditors, and the availability of such evaluation reports, can contribute to the trustworthiness of the technology. In applications affecting fundamental rights, including safety-critical applications, AI systems should be able to be independently audited.	◾Did you establish mechanisms that facilitate the system’s auditability, such as ensuring traceability and logging of the AI system’s processes and outcomes? ◾Did you ensure, in applications affecting fundamental rights (including safety-critical applications) that the AI system can be audited independently?
21	21	7.2	Accountability	Risk Management	Both the ability to report on actions or decisions that contribute to a certain system outcome, and to respond to the consequences of such an outcome, must be ensured. Identifying, assessing, documenting and minimising the potential negative impacts of AI systems is especially crucial for those (in)directly affected. Due protection must be available for whistle-blowers, NGOs, trade unions or other entities when reporting legitimate concerns about an AI system. The use of impact assessments (e.g. red teaming or forms of Algorithmic Impact Assessment) both prior to and during the development, deployment and use of AI systems can be helpful to minimise negative impact. These assessments must be proportionate to the risk that the AI systems pose.	◾Did you carry out a risk or impact assessment of the AI system, which takes into account different stakeholders that are (in)directly affected? ◾Did you provide training and education to help developing accountability practices?  Which workers or branches of the team are involved? Does it go beyond the development phase?  Do these trainings also teach the potential legal framework applicable to the AI system?  Did you consider establishing an ‘ethical AI review board’ or a similar mechanism to discuss overall accountability and ethics practices, including potentially unclear grey areas? ◾Did you foresee any kind of external guidance or put in place auditing processes to oversee ethics and accountability, in addition to internal initiatives? ◾Did you establish processes for third parties (e.g. suppliers, consumers, distributors/vendors) or workers to report potential vulnerabilities, risks or biases in the AI system?
22	22	7.3	Accountability	Documenting Trade-offs	When implementing the above requirements, tensions may arise between them, which may lead to inevitable trade-offs. Such trade-offs should be addressed in a rational and methodological manner within the state of the art. This entails that relevant interests and values implicated by the AI system should be identified and that, if conflict arises, trade-offs should be explicitly acknowledged and evaluated in terms of their risk to ethical principles, including fundamental rights. In situations in which no ethically acceptable trade-offs can be identified, the development, deployment and use of the AI system should not proceed in that form. Any decision about which trade-off to make should be reasoned and properly documented. The decision-maker must be accountable for the manner in which the appropriate trade-off is being made, and should continually review the appropriateness of the resulting decision to ensure that necessary changes can be made to the system where needed.	◾Did you establish a mechanism to identify relevant interests and values implicated by the AI system and potential trade-offs between them? ◾How do you decide on such trade-offs? Did you ensure that the trade-off decision was documented?
23	23	7.4	Accountability	Ability to Redress	When unjust adverse impact occurs, accessible mechanisms should be foreseen that ensure adequate redress. Knowing that redress is possible when things go wrong is key to ensure trust. Particular attention should be paid to vulnerable persons or groups.	◾Did you establish an adequate set of mechanisms that allows for redress in case of the occurrence of any harm or adverse impact? ◾Did you put mechanisms in place both to provide information to (end-)users/third parties about opportunities for redress?
24	24	8.1	Technical Methods	Ethics and rule of law by design	Methods to ensure values-by-design provide precise and explicit links between the abstract principles which the system is required to respect and the specific implementation decisions. The idea that compliance with norms can be implemented into the design of the AI system is key to this method. Companies are responsible for identifying the impact of their AI systems from the very start, as well as the norms their AI system ought to comply with to avert negative impacts. Different “by-design” concepts are already widely used, e.g. privacy-by-design and security-by-design. As indicated above, to earn trust AI needs to be secure in its processes, data and outcomes, and should be designed to be robust to adversarial data and attacks. It should implement a mechanism for fail-safe shutdown and enable resumed operation after a forced shut-down (such as an attack).
25	25	8.2	Technical Methods	Explanation methods	For a system to be trustworthy, we must be able to understand why it behaved a certain way and why it provided a given interpretation. A whole field of research, Explainable AI (XAI) tries to address this issue to better understand the system’s underlying mechanisms and find solutions. Today, this is still an open challenge for AI systems based on neural networks. Training processes with neural nets can result in network parameters set to numerical values that are difficult to correlate with results. Moreover, sometimes small changes in data values might result in dramatic changes in interpretation, leading the system to e.g. confuse a school bus with an ostrich. This vulnerability can also be exploited during attacks on the system. Methods involving XAI research are vital not only to explain the system’s behaviour to users, but also to deploy reliable technology.
26	26	8.3	Technical Methods	Testing and validating	Due to the non-deterministic and context-specific nature of AI systems, traditional testing is not enough. Failures of the concepts and representations used by the system may only manifest when a programme is applied to sufficiently realistic data. Consequently, to verify and validate processing of data, the underlying model must be carefully monitored during both training and deployment for its stability, robustness and operation within well-understood and predictable bounds. It must be ensured that the outcome of the planning process is consistent with the input, and that the decisions are made in a way allowing validation of the underlying process. Testing and validation of the system should occur as early as possible, ensuring that the system behaves as intended throughout its entire life cycle and especially after deployment. It should include all components of an AI system, including data, pre-trained models, environments and the behaviour of the system as a whole. The testing processes should be designed and performed by an as diverse group of people as possible. Multiple metrics should be developed to cover the categories that are being tested for different perspectives. Adversarial testing by trusted and diverse “red teams” deliberately attempting to “break” the system to find vulnerabilities, and “bug bounties” that incentivise outsiders to detect and responsibly report system errors and weaknesses, can be considered. Finally, it must be ensured that the outputs or actions are consistent with the results of the preceding processes, comparing them to the previously defined policies to ensure that they are not violated.
27	27	8.4	Technical Methods	Quality of Service Indicators	Appropriate quality of service indicators can be defined for AI systems to ensure that there is a baseline understanding as to whether they have been tested and developed with security and safety considerations in mind. These indicators could include measures to evaluate the testing and training of algorithms as well as traditional software metrics of functionality, performance, usability, reliability, security and maintainability.
28	28	9.1	Non-Technical Methods	Codes of conduct	Organisations and stakeholders can sign up to the Guidelines and adapt their charter of corporate responsibility, Key Performance Indicators (“KPIs”), their codes of conduct or internal policy documents to add the striving towards Trustworthy AI. An organisation working on or with AI systems can, more generally, document its intentions, as well as underwrite them with standards of certain desirable values such as fundamental rights, transparency and the avoidance of harm.
29	29	9.2	Non-Technical Methods	Standardisation	Standards, for example for design, manufacturing and business practices, can function as a quality management system for AI users, consumers, organisations, research institutions and governments by offering the ability to recognise and encourage ethical conduct through their purchasing decisions. Beyond conventional standards, co-regulatory approaches exist: accreditation systems, professional codes of ethics or standards for fundamental rights compliant design. Current examples are e.g. ISO Standards or the IEEE P7000 standards series, but in the future a possible ‘Trustworthy AI' label might be suitable, confirming by reference to specific technical standards that the system, for instance, adheres to safety, technical robustness and transparency.
30	30	9.3	Non-Technical Methods	Certification	As it cannot be expected that everyone is able to fully understand the workings and effects of AI systems, consideration can be given to organisations that can attest to the broader public that an AI system is transparent, accountable and fair. These certifications would apply standards developed for different application domains and AI techniques, appropriately aligned with the industrial and societal standards of different contexts. Certification can however never replace responsibility. It should hence be complemented by accountability frameworks, including disclaimers as well as review and redress mechanisms.
31	31	9.4	Non-Technical Methods	Accountability via governance frameworks	Organisations should set up governance frameworks, both internal and external, ensuring accountability for the ethical dimensions of decisions associated with the development, deployment and use of AI systems. This can, for instance, include the appointment of a person in charge of ethics issues relating to AI systems, or an internal/external ethics panel or board. Amongst the possible roles of such a person, panel or board, is to provide oversight and advice. As set out above, certification specifications and bodies can also play a role to this end. Communication channels should be ensured with industry and/or public oversight groups, sharing best practices, discussing dilemmas or reporting emerging issues of ethical concerns. Such mechanisms can complement but cannot replace legal oversight (e.g. in the form of the appointment of a data protection officer or equivalent measures, legally required under data protection law).
32	32	9.5	Non-Technical Methods	Education and awareness to foster an ethical mind-set	Trustworthy AI encourages the informed participation of all stakeholders. Communication, education and training play an important role, both to ensure that knowledge of the potential impact of AI systems is widespread, and to make people aware that they can participate in shaping the societal development. This includes all stakeholders, e.g. those involved in making the products (the designers and developers), the users (companies or individuals) and other impacted groups (those who may not purchase or use an AI system but for whom decisions are made by an AI system, and society at large). Basic AI literacy should be fostered across society. A prerequisite for educating the public is to ensure the proper skills and training of ethicists in this space.
33	33	9.6	Non-Technical Methods	Stakeholder participation and social dialogue	The benefits of AI systems are many, and Europe needs to ensure that they are available to all. This requires an open discussion and the involvement of social partners and stakeholders, including the general public. Many organisations already rely on stakeholder panels to discuss the use of AI systems and data analytics. These panels include various members, such as legal experts, technical experts, ethicists, consumer representatives and workers. Actively seeking participation and dialogue on the use and impact of AI systems supports the evaluation of results and approaches, and can particularly be helpful in complex cases.
34	34	9.7	Non-Technical Methods	Diversity and inclusive design teams	Diversity and inclusion play an essential role when developing AI systems that will be employed in the real world. It is critical that, as AI systems perform more tasks on their own, the teams that design, develop, test and maintain, deploy and procure these systems reflect the diversity of users and of society in general. This contributes to objectivity and consideration of different perspectives, needs and objectives. Ideally, teams are not only diverse in terms of gender, culture, age, but also in terms of professional backgrounds and skill sets.
35	35	10.1	Governance	Management and Board	Top management discusses and evaluates the AI systems’ development, deployment or procurement and serves as an escalation board for evaluating all AI innovations and uses, when critical concerns are detected. It involves those impacted by the possible introduction of AI systems (e.g. workers) and their representatives throughout the process via information, consultation and participation procedures.
36	36	10.2	Governance	Compliance/Legal department/Corporate responsibility department	The responsibility department monitors the use of the assessment list and its necessary evolution to meet the technological or regulatory changes. It updates the standards or internal policies on AI systems and ensures that the use of such systems complies with the current legal and regulatory framework and to the values of the organisation.
37	37	10.3	Governance	Product and Service Development or equivalent	The Product and Service Development department uses the assessment list to evaluate AI-based products and services and logs all the results. These results are discussed at management level, which ultimately approves the new or revised AI-based applications.
38	38	10.4	Governance	Quality Assurance	The Quality Assurance department (or equivalent) ensures and checks the results of the assessment list and takes action to escalate an issue higher up if the result is not satisfactory or if unforeseen results are detected.
39	39	10.5	Governance	Human Resources	The HR department ensures the right mix of competences and diversity of profiles for developers of AI systems. It ensures that the appropriate level of training is delivered on Trustworthy AI inside the organisation.
40	40	10.6	Governance	Procurement	The procurement department ensures that the process to procure AI-based products or services includes a check of Trustworthy AI.
41	41	10.7	Governance	Day-to-day Operations	Developers and project managers include the assessment list in their daily work and document the results and outcomes of the assessment
42	42	11	Governance	Relation to existing law and processes	It is also important for AI practitioners to recognise that there are various existing laws mandating particular processes or prohibiting particular outcomes, which may overlap and coincide with some of the measures listed in the assessment list. For example, data protection law sets out a series of legal requirements that must be met by those engaged in the collection and processing of personal data. Yet, because Trustworthy AI also requires the ethical handling of data, internal procedures and policies aimed at securing compliance with data protection laws might also help to facilitate ethical data handling and can hence complement existing legal processes. Compliance with this assessment list is not, however, evidence of legal compliance, nor is it intended as guidance to ensure compliance with applicable laws.
	AID		Category