Chapter 9

Safe, Aligned, Fair, and Ethical Considerations for Artificial Intelligence and Automation

Lauren Sprague, Christopher Cischke, Samantha Walker, Jeff Wang, and Shane T. Mueller
Michigan Technological University

9.1 Introduction

Ethical and social aspects of automation and AI have appeared in philosophy journals and science fiction entertainment for decades. However, the fields most concerned with human-automation and AI interaction (ergonomics, cybernetics, cognitive systems engineering and human factors) tended to focus on cognitive and informational, and control aspects of the system. Trust and trustworthiness were mainly about the capabilities of a system and its failure modes; transparency and explainability were focused on understanding the decision logic; safety concerns often involved the system being controlled by automation (a nuclear power plant or airplane). Users were often employees (air-traffic controllers, pilots, operators, doctors), who used the systems but may not have been exposed to individual risk for using it, aside from the dilemma of giving authority to the system but having to take responsibility for its outcomes.

As search, classification/diagnosis, recommender systems, and other automation and intelligent systems became ubiquitous and in use by billions of users, new concerns have emerged regarding risks, safety, and fairness, at a societal level. Now, users have privacy concerns over the information the systems have. Safety now goes beyond the control of another system, and involves automated vehicles sharing roadways with human drivers. Transparency goes beyond developing a better mental model of a mechanism, and involves understanding conflicts of interest, development practices, data sources, and use of Personally Identifiable Information. Trustworthiness goes beyond ability and reliability, and concerns underlying data, algorithms, audits, and development practices. As intelligent algorithms have moved into all corners of society, it is not surprising that societal challenges have emerged and become increasingly important.

There are numerous related and overlapping issues related to safe and fair AI. We will discuss many concepts in this chapter regarding the ethical challenges for AI and automation systems in society. Many of these distill down to what has been called the “Value Alignment Problem” (Russell, 2014; Bostrom, 2014), which mirrors the discussion in Chapter 4 regarding human-like AI. There, we discussed how AI can take on human-like properties, for both good and deceptive purposes. The alignment problem is similar, but asks how AI can take on our values as well.

9.2 The Alignment Problem in AI

What happens when artificial intelligence no longer acts in ways that benefit humans or are in accordance with human values? Popular culture has addressed this question in various ways. AI misbehavior has been displayed in many popular works of fiction, like 2001: A Space Odyssey, The Matrix series, the Terminator series, and Ex Machina. In all of those works, the AI ceases to align its behavior and actions with human values. These illustrate the problem of AI alignment: the state in which an artificial intelligence system follows human intent and does what humans want it to do, even in situations containing considerable uncertainty. The problem of AI alignment is creating and maintaining alignment in developed systems, which AI developers and researchers consider to be a fundamental problem in the field. In one survey of Natural Language Processing researchers, 36% believed that AI could produce catastrophic outcomes in this century (Ji et al., 2024).

An early (and fictional) version of alignment comes from the author Isaac Asimov, who produced the Three Laws of Robotics:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

While a fine starting place, a more modern set of guidelines for alignment rests on four principles: Robustness, Interpretability, Controllability, and Ethicality (Ji et al., 2024). The so-called RICE principles break down into the resilience of AI systems operating across diverse or adversarial scenarios, the ability of humans to understand the inner reasoning of the AI system, the ability of humans to control (and even shut down) the system, and an unwavering commitment from the system to uphold human norms and values.

The key difficulty is communicating those values to the AI. One AI vendor, Anthropic, adheres to a constitutional approach to alignment. This approach differs little from Asimov's Three Laws except that reinforcement learning refines the AI's understanding of the constitutional principles originally specified (Bai et al., 2022). Specific principles are better, but some experiments show that a single, simple principle of “do what's best for humanity” can be generalized by simple AI models (Kundu et al., 2023).

A structured approach to this starts by decomposing the problem into three subdomains of alignment: specification alignment, process alignment, and evaluation alignment (Terry et al., 2023). In the specification alignment stage, the user tells the AI what the task is, both generally and in specifics. For example, we may ask the AI to generate code to implement a search algorithm (the general task), but that it should be written in the Python language in a manner simple enough for a novice programmer to understand (the specifics). In the process alignment stage, we constrain how the AI performs the task. In our Python example, we may specify that all code should be properly attributed to a source or that an intermediate stage describing the algorithm should be produced to allow the user to tweak the algorithm before the final code generation. Finally, in the evaluation alignment stage, we validate the output for correctness. If our algorithm doesn't properly execute, then the AI is unaligned.

As with other AI issues, the solution may not come from a technical domain. Alignment may be an economic or sociological problem. Humans have had incomplete or mis-specified contracts between themselves with well-studied results. Even with a constitutional alignment, the contract between the human and AI may be similarly incomplete because the designer of the AI failed to think of all possible contingencies, insufficient effort was put into the principles, or some principles are difficult (or impossible) to contract. To this end, all reward functions used by AI models are mis-specified or incomplete because there are implied portions that human contracts do not need (Hadfield-Menell & Hadfield, 2019).

Many of the implied rules may be considered “silly” (Hadfield-Menell et al., 2019). Alignment with human values requires knowing when silly rules need to be applied and when high-priority rules take precedence. For example, we may tell an autonomous vehicle that it shouldn't hit small animals crossing the road, but this rule should be violated if the alternative endangers human life. The balance between the important and silly rules is critical, as is understanding the relative frequency of needing to apply them.

Perhaps the approach may be to teach the AI what humans value (Butlin, 2021). Humans themselves operate under reinforcement learning – performing some action X yielded positive results, both intrinsically and extrinsically. Therefore, we will do that action more often. Conversely, some other action Y yielded negative results and will be performed less often. Since these results include relational and cultural outcomes (not only did I arrive at my intended destination, I did it without endangering other drivers and pedestrians), an AI that understands human values may be able to fill in the gaps in a contract. However, our poor introspection into our own values, the difficulty in quantifying some types of intrinsic motivations, or the indirect link between certain motivations and their outcomes may make this approach difficult. Moreover, some believe that a universal set of “human values” does not exist (Turchin, 2019), and using them to define AI safety is futile.

Ultimately, alignment is not a “set it and forget it” task. The initial alignment, whether constitutional or value-based, could be considered “forward alignment” (Ji et al., 2024), whereas the ongoing alignment refinement after the system's deployment is a backward alignment. Both are necessary to ensure AI continues to be a long-term benefit to human society.

9.2.1 The Paperclip Maximizer Problem and p(Doom)

The goal of value alignment asks whether AI is working to support the same goals and values of its users, and humans in general. Society has long experience dealing with people who do not behave in alignment with one another's or society's values, and we have organized legal contracts, laws, and punishments to reinforce these pro-social norms. For AI systems, one challenge is that the legal actions are difficult to determine because of issues related to responsibility (discussed previously in Chapter 5): AI cannot take responsibility, and so may not be a proper partner in normative trusting relationship. Two related approaches to this include contractual trust and licensure. People take responsibility for damage their vehicles do through indemnification insurance.

However, the goal alignment problem is difficult, because an AI might be enlisted to achieve goals by its user that are (with or without the knowledge of the user) misaligned with society. And to the extent that artificial intelligence becomes more capable, the potential negative outcomes become greater, and some argue threaten society's future. Bostrom (2014) posed an example of this referred to as the paperclip-maximizer thought experiment. Suppose the AI was given a simple objective: to maximize paperclip production. A powerful enough AI would attempt to convert all factories, resources, and supply chains to maximum planet-scale paperclip production. Furthermore, it might resist being turned off or having its goals changed or thwarted, and even treat other humans and obstacles that stand in its way as threats to be removed. This illustrates that alignment can be problematic even with an AI that is designed to be obedient to its user's goals.

This kind of concern, along with other primary impacts (potential for job losses, impact on education) and secondary impacts (environmental and economic impacts), have led to the popularization of \(p(Doom)\): an assessment of the probability that AI creates a catastrophic event (doing huge harm) or even the probability that AI is an existential risk to human life. Most scenarios focus on biosecurity, cybersecurity, or loss-of-control scenarios where AI “escapes” the servers via self-replicating bots and becomes impossible to control.

9.3 Frameworks for Ethical AI

Based on growing concerns about social, individual, and existential risks of AI and automation, researchers and practitioners across business, law, ethics, and the social sciences have proposed structures for understanding and governing harms. These appear in regulations, standards, and organizational policy documents. For example, legal frameworks such as the EU General Data Protection Regulation (GDPR) (European Parliament and Council of the European Union, 2016) and Artificial Intelligence Act (European Parliament and Council of the European Union, 2024) incorporate ethical guidelines determined by a group of AI experts (High-Level Expert Group on Artificial Intelligence, 2019). Standards such as the NIST AI Risk Management Framework (Tabassi, 2023) have provided guidelines for implementing and reducing AI risks, which may be used for future legislation or technology acquisition requirements. Finally, many major cloud and software providers have developed responsible-AI programs, including Microsoft (Microsoft, 2022), IBM (IBM, 2024), Amazon Web Services (Services, 2025), and Google/Google Cloud (Google, 2018). In addition, individual researcher groups have proposed various frameworks, including high-level comprehensive frameworks for risks, harms, or design trade-offs (Jobin et al., 2019; Arinaitwe, 2023) and specific guidelines for individual aspects of the problem, such as Shneiderman's \citeyearshneiderman2020 Trustworthy AI evaluation framework discussed below.

9.3.1 Framework for Harms

One example of a comprehensive framework is the “Framework for Harms” proposed by Jobin et al. (2019). They established a list of eleven overarching ethical principles most relevant to human/AI interaction: transparency, justice and fairness, non-maleficence, responsibility, privacy, beneficence, freedom and autonomy, trust, sustainability, dignity, and solidarity. These principles may be used to establish best practices for harm reduction when interacting with AI and machine learning systems. This framework is the result of a review of the literature that encompasses most of the harm-related factors discussed by others. Consequently, it is not a prescriptive list of factors that need to be addressed when implementing policy or making decisions about the capability or requirements of an AI system. Rather, it provides a broad set of concerns that could be important, and is a framework for establishing fair and ethical AI tools.

Transparency and Trust.

We covered more performance-relevant aspects of trust in Chapter 5, and transparency in Chapters 7 and 8. Each have additional aspects related to ethics and fairness, (we cover these more later in this chapter). Jobin et al. (2019) highlighted 73 sources in their content analysis that found transparency to be the most important ethical principle to consider when developing AI tools. Although the definitions of these terms are broad and varied, it is typically framed as an effort to increase explainability, interpretability, or other acts of communication–especially disclosure. That is, for cognitive work transparency might be centered on developing a better mental model, but in usage and adoption, it incorporates disclosure about how data are used, stored, and other development processes. Many researchers also emphasized a need for transparency as it relates to trust in a partner for human-machine teaming (Endsley, 2023; Morley et al., 2020). Transparency in terms of AI tools can be many different things such as disclosing of training data, labeling of algorithm code, or releasing results of testing and evaluation (Peckham, 2024). Those developing AI tools and systems can achieve greater transparency through the disclosure of information that includes but is not limited to boundary conditions, societal impact, natural language explanations, investors and stakeholders, and source code (Nikolinakos, 2023).

Justice and Fairness.

Although the concepts of justice and fairness are challenging to define in terms of human-AI interactions, they typically are used to describe distribution, access, and benefits of tools, accessibility, equity, and non-discrimination (Peckham, 2024). AI tools have a history of bias (i.e. racial bias in policing algorithms, facial recognition, and finance tools, gender bias in resume scanning algorithms) thus justice and fairness should be implemented to reduce discrimination and increase diversity, inclusion, and equality in AI-human collaboration. Moreover, AI tools are often trained on human-generated copyrighted content (images, code, written text), with the consequence that those models help most users generate similar content, without compensating those who created it. Moreover, there can be implicit and illegal biases trained into these models, and Davidson & Ravi (2020) argued that AI tools should evaluate all human collaborators equally regardless of race, gender, or other protected characteristics. In their research into anomaly detection, Zhang & Davidson (2021) found that the model could be trained to remove either individual or group bias, but the opposite bias (group or individual) could still emerge. Whether AI tools should be trained to be fair and just to the individual or the group depends on the use case for the tool. When designing a personalized chat-bot it is likely to be more beneficial to the end user if it is fair on an individual basis while policing algorithms should be focused on removing bias against groups due to a history of unfair police practices towards specific demographics in the US (Beattie et al., 2022; Fountain, 2022).

Non-maleficence.

Non-maleficence is encapsulated in Asimov's first rule, but can be difficult to implement algorithmically, especially when harms and benefits come into conflict. When considering human-AI interactions, Textor et al. (2022) found that violations of non-maleficence (do no harm) were considered to be more egregious violations than other ethical violations like prospect of success and proportionality. When designing AI tools for human teaming, developers should avoid causing harm to individuals, communities, or society as a whole depending on the use case for the tool. This concept is closely related to justice and fairness as non-maleficence emphasizes removing discriminatory bias and the reduction of physical or social harms. This can be implemented through harm-prevention strategies that focus on technical measures and governance strategies at the level of AI research, design, development, or deployment (Jobin et al., 2019).

Responsibility.

Responsible AI has become a blanket term for many of the issues discussed in this chapter. For example, Microsoft, IBM, and AWS all have responsible AI documents (Microsoft, 2022; IBM, 2024; Services, 2025). These include many of the topics covered in the larger Framework for Harms, but include topics such as fairness, transparency, explainability, privacy/security, reliability/safety, accountability/governance, and societal impact. However, responsibility has specific meanings as well, and it forms one of the central paradoxes of human-AI interaction in terms of a responsibility gap (Matthias, 2004): only humans can take responsibility for actions of the tools we use, but we still rely on and give authority to act to these systems. Jobin et al. (2019) highlighted 60 sources that consider responsibility to be an important ethical guideline to consider when designing Human-AI interaction tools and environments. Some sources define it as AI tools operating with “integrity” and establishing legal liability initially (Matsuo, 2017). Others frame it in terms of the broader societal impact (Siqueira de Cerqueira et al., 2021). With all of this to consider, Jobin et al. (2019) argues the key aspects of responsible AI in human-AI interaction should be transparency, fairness, and accountability to the end user.

Privacy.

Privacy is often at the heart of Ethical AI as it can be considered both a value to uphold and a protected right. It is most frequently defined in the terms of data security and protection. Achieving privacy in human-AI interactions can be done through technical solutions, access control, regulatory processes, more research and awareness, and through the creation and adaptation of legal frameworks to support these tools (Jobin et al., 2019). Differential privacy, a model which guarantees that an individual's data has little impact on output, is one technical solution that has been implemented into AI tools (Zhu et al., 2020).

Beneficence.

Beneficence is related to non-maleficence, but focuses on the positive benefits that accrue (rather than the importance of lack of harm). This ethical concept of promoting good (beneficence) is mentioned frequently in AI-human collaboration research, but it was very rarely defined when Jobin et al. (2019) performed their content analysis. AI research in the private sector is often focused on the benefit or good that the customer receives, while some research calls for the benefits that AI brings to be shared among humanity, society, and the environment (Jobin et al., 2019; Siqueira de Cerqueira et al., 2021). There is an expectation that AI tools will be aligned with human values but this expectation fails to take into account that human values differ from culture to culture and change over time. This issue could be mitigated by developing AI tools based on the society and cultures that will be utilizing them.

Freedom and Autonomy.

Autonomy is considered to be the most fundamental human right to self-determination through democratic means, right to establish and develop relationships with other human beings, the freedom to withdraw consent, and the freedom to flourish (Jobin et al., 2019). When considering how AI-human interactions can safeguard human freedom and autonomy, many researchers promote it through transparency and predictable AI, increasing the general public's knowledge about AI, giving notice and consent, or refraining from collecting personal data without informed consent (Jobin et al., 2019).

Sustainability.

When considering how sustainability intersects with human-AI interactions, one can consider both the environmental impact of using the AI tool and how the AI tool contributes to sustainability practices (van Wynsberghe, 2021). For AI tools to be considered “Sustainable AI” they must accomplish both. This indicates that not all AI tools can or even should strive for the Sustainable AI label as not all tools should be used to contribute to outcomes that improve sustainability. Some AI tools are run off of equipment that uses renewable resources as well as carbon off-set resources (Wu et al., 2022). However, the high energy demands of these models is still a considerable concern (Crawford, 2021).

Dignity.

Human dignity is another foundational human right that states that all people hold a special value that is inherent and fundamentally and solely tied to their humanity and not relative to any other factor. This suggests that humanity is worth preserving the existence of for its own sake. With AI tools that interface with human beings researchers call for the promoting of humanity. AI should not diminish or destroy human dignity but preserve and protect it (Jobin et al., 2019). This could be accomplished through new legislation and governance initiatives or through technical guidelines (van Est et al., 2017).

Solidarity.

When discussing solidarity as it relates to human-AI interactions, most authors reference this in terms of the labor market and how these tools are likely to impact unemployment rates and the reduction of unskilled labor jobs. This underscores the need for distributing the benefits of AI in order to respect persons or groups who may be vulnerable (Jobin et al., 2019). This could be in terms of upskilling current employees and training employees to be the managers of the AI tools (Sofia et al., 2023).

9.3.2 Summary of Frameworks for Ethical AI

As artificial intelligence gets more capable, the social and ethical impacts of AI is increasing rather than decreasing. The different frameworks for understanding the issues have substantial overlap, and only some of them have any legal backing in limited jurisdictions. For the most part, the frameworks are descriptive and normative–telling what has been considered and what practices ought to be done. This is clearly the first stage, and in coming years we might expect additional regulations to be enacted that codify some of these principles, as well as laying out punishments for violating the principles.

One important issue related to punishments is the tension between responsibility and the proclivity to anthropomorphize AI systems (see Chapter 4). We see many discussions of harmful acts performed by AI to frame the issues in terms of anthropomorphic language–the AI might escape, or attack a competitor. This framing (either by accident or deliberately) casts blame and responsibility on the AI system, rather than understanding the responsibility of control and governance that needs to be exerted by the user or organization who created the system. It is likely that appropriate governance of AI systems will only happen when we institute appropriate punishments (to humans or organizations) that ultimately bear responsibility for the actions of the AI.

Several of the concepts discussed in the Framework for Harms bear further discussion. In the next sections, we will expand on a few specific aspects that are especially useful in the context of human-AI interaction.

9.4 Fairness and Bias in Algorithms

Bias refers to a number of unwanted outcomes in both social and technical systems. It can refer to straightforward aspects of statistical bias (how far off an estimate is from the truth), legal bias (linked to discrimination law) and in sociotechnical systems (including data and models, but extending to institutions, people, and usage) (Schwartz et al., 2022).

Schwartz et al. (2022) identified three categories of bias impacting artificial intelligence:

• Systemic bias: Institutional, organizational, or historic practices that advantage some groups or individuals and disadvantage others.
• Statistical/Computational bias: Systematic error from using non-representative samples, data processes (data cleaning, over/under-fitting) etc.
• Human bias: Biases in human thought that impact how a human perceives information, (e.g. output from an AI system), that they then use to make a decision, inform others, etc.

Although these could be deliberately created by bad actors, none of them need to be done with intent–each could emerge because of reasonable decisions in which we lack awareness of the impact.

Many examples of AI bias have been present in society and media. Mozafari et al. (2020) found that an AI system misclassified posts from people of color as hate speech due to differences in regional dialects. This led to the AI system banning or removing a higher number of people of color from the platform, indicating a bias present in the AI. In another study, Thomas & Thomson (2023) evaluated a generative AI application's ability to generate an image of a journalist. They identified many biases in the AI output, including ageism, sexism, classism, and racial bias. Despite changing the user input to try to mitigate some biases, the generative AI program still produced a biased output (Thomas & Thomson, 2023).

Due to the prevalence of bias in AI algorithms, recent research has been focused on ways to mitigate or reduce bias. Roselli et al. (2019) described many ways to mitigate biases in AI, including vetting the training data before implementing it in the AI system, monitoring the production data and output, and changing the input and training data appropriately based on biases present in the output. Utilizing explainability in bias detection is important for making the decision-making process of the AI outcome explainable to the human user and AI developers (Roselli et al., 2019; Mosqueira-Rey et al., 2023). Human-in-the-loop systems also can help in bias detection by having a human interact with the AI output “behind the curtain” to mitigate bias (Mosqueira-Rey et al., 2023).

Bias detection and mitigation are important, but issues can arise where bias mitigation goes too far. Oversteps in bias mitigation can increase other biases that were not originally present in the data before the data was manipulated to mitigate one bias (Safarlou et al., 2023). For example, if a dataset is over-manipulated to mitigate racial bias found in the data, a gender bias may then emerge in the dataset that was not present prior to the manipulation of training data. Understanding how to mitigate bias in an appropriate way is important to make the AI system output as accurate as possible (Safarlou et al., 2023). Furthermore, attempts to de-bias output are subject to what “unbiased output” really entails. For example, if one prompts an AI to generate an image of a worker in a profession, should the race, gender, or age be the most typical member of that profession, or should it generate a distribution of images reflecting the actual true distribution (if that is even known)?

These tensions are especially visible in generative image systems. Investigations have shown that popular generators often reproduce and amplify stereotypes—for example, portraying leadership as white and male, associating darker-skinned men with crime, and under-representing women in high-status professions (Tiku et al., 2023; Nicoletti & Bass, 2023). Developers have responded with techniques intended to increase diversity in outputs, but those mitigations can fail as well—in the opposite direction. As Baum & Villasenor (2024) describe, efforts to force representational diversity have produced historically inappropriate images (e.g., racially diverse depictions of World War II-era German soldiers or of the U.S. Founding Fathers), prompting public criticism and product rollbacks. This highlights not that interventions are unnecessary, but that crude or context-insensitive “diversity filters” can trade one form of misrepresentation for another.

9.5 Kinds of Transparency

We discussed aspects of transparency in Chapter 7, in which transparency was primarily used to characterize features that improved users' mental models of a system. Andrada et al. (2023) show that this is just one of the many ways the term transparency is used, identifying this usage mostly with information transparency. In the broader context of AI, transparency means both observability, clarity, and disclosure to users regarding the inner workings, decision-making processes, and societal impacts of AI algorithms.

Andrada et al. (2023) defined several varieties of transparency. These include:

• Reflective transparency: an umbrella notion of “seeing into” an AI system—gaining insight into its processes, constitution, or effects to support understanding, oversight, challenge, or control. It has three subtypes:
• Information transparency: disclosure of information about the system (e.g., algorithms, training data, capabilities and limits) so stakeholders can judge fairness, trustworthiness, safety, efficacy, accountability, or regulatory compliance.
• Material transparency: visibility into material realization beyond algorithms—hardware, energy use, manufacturing and labor conditions, ecological impacts, and related socioeconomic factors. Many of the concerns about data centers are framed in terms of material transparency.
• Transformational transparency: visibility into how pervasive use of AI transforms users' neural, bodily, cognitive, epistemic, or ethical capacities (e.g., skill change, dependency, or “capability echo”). As discussed in Chapter 3, Ahmed et al. (2026) described many of these risks and impacts during the adoption stage, including de-skilling, capability erosion, automation bias, and AI anxiety. The challenge is that although we might anticipate some of the potential risks, requiring transparency about them (especially when they may not actually materialize) may be techno-phobic.
• Transparency-in-use: phenomenological (or phenomenal) transparency—the artifact “disappears” in skilled use so attention stays on the task rather than the interface (acting through the technology). For informational devices, this includes:
• Representational transparency: ease of interpreting the system's displays or representational outputs.
• Procedural transparency: how effectively, fluidly, and skillfully the device can be deployed in action.

The European Commission's High-Level Expert Group on AI outlined seven key requirements that AI systems should meet to be deemed trustworthy (High-Level Expert Group on Artificial Intelligence, 2019). Transparency is included, underscoring its critical role in how the EU is navigating the political and ethical landscape of AI development and implementation. Transparency is vital for supporting trust among users, stakeholders, and society at large (Andrada et al., 2023). It goes far beyond information-related aspects, and it involves both awareness and disclosure, providing insights into how AI algorithms operate, including their decision-making processes, data sources, potential biases, how they were implemented, and how they are controlled or governed. Further, transparency encompasses disclosing the intentions behind AI applications and their potential societal impacts, thereby enabling informed discussions and decisions regarding their use and regulation. Understanding these aspects is crucial for navigating the complex landscape of AI governance, where political and ethical considerations play pivotal roles in shaping policies and practices (Andrada et al., 2023).

9.6 Explainability, Interpretability, and Accountability

Chapter 8 covered many aspects of explainability and interpretability, but with a focus on informational explainability. One of the motivations for the resurgence of explainable AI as a field was the EU GDPR act (European Parliament and Council of the European Union, 2016), which discusses the “right to explanation” (Goodman & Flaxman, 2017). Although the interpretation of this right is somewhat disputed, the motivation and target are really about fairness and accountability. For example, if a loan is denied by an algorithm, the applicant may be owed an explanation of the reasoning behind this denial. Part of the rationale for this right is that the explanation might trace the denial back to aspects that are illegal to use in loan approvals. Even without this right, organizations using algorithmic decisions may wish for explainability, interpretability, or accountability to improve their own systematicity and governance/control of the systems, and to improve public trust in their organization. Some of the ethical and social reasons for explainable and interpretable AI include:

• Enhancing trust and accountability: Clear explanations of LLM outputs improve user understanding and trust. This transparency fosters accountability by allowing stakeholders to assess the model's ethical alignment.
• Error diagnosis and improvement: Refined explanations help identify model weaknesses, biases, and errors, enabling targeted improvements to enhance accuracy and reliability.
• User understanding and acceptance: Clear explanations empower users to grasp the model's behavior, fostering acceptance and effective utilization.
• Safety and robustness: Prioritizing explanation enhances model resilience by enabling safeguards against harmful outputs and adversarial attacks.
• Legal and ethical compliance: Interpretable outputs aid in assessing model adherence to regulations and ethical guidelines, mitigating potential risks.

Within the context of LLMs, these reasons are perhaps even more important. Understanding large language models presents significant opportunities through the integration of natural-language interfaces and interactive explanations. Leveraging natural language, a medium familiar to humans, offers a promising avenue for describing complex patterns inherent in LLM outputs, enabling tailored insights to meet individual needs. Through dialogue and analysis, users can delve deeper into explanations, fostering a collaborative exploration of complex concepts and enhancing comprehension. However, the potential risks in this approach are amplified, because the explanations provided by LLMs are typically logically sound, detailed, and easy to understand. However, (dependent on the context) they may not be true, and the LLM may not have self-reflection abilities that enable it to describe how choices or decisions were actually made.

9.7 Governance and Licensure

AI governance refers to “a system of rules, practices, processes, and technological tools” ensuring AI use aligns with strategy/values, legal requirements, and the organization's ethical-AI principles (Mäntymäki et al., 2022). Importantly, governance does not simply involve legal frameworks and authority, and aligns much more closely with the notion of control and regulation posited by the Good Regulator theorem (Conant & Ashby, 1970). We might infer from this that good governance requires good understanding of the underlying system, but many elements of governance can be implemented generically for any algorithm. Gasser & Almeida (2017) argued a three-layer model is necessary, one involving technical aspects (algorithms and data), with a second covering ethical aspects and a third related to social and legal aspects.

Governance is necessary because of the responsibility gap discussed earlier in this chapter and in Chapter 5. AI cannot bear normative responsibility, and the liability falls on people and organizations that create or use the algorithms (Ryan, 2020). Anthropomorphism is sometimes used to shift the blame onto the AI, but to manage this, we need human and organizational governance: control, oversight, and enforceable consequences.

There are many areas of society where we have done similar things. For example, if someone uses a weapon to harm others, the person (not the weapon) is still held responsible. Similarly, instructing a subordinate or hiring someone to cause harm can still result in a determination that the initiator bears responsibility. Pet owners can be responsible for damage or harm caused by their animals, and parents have been held responsible for harm caused by their children when they were deemed to have enabled the actions. More widespread is the legal requirements for liability insurance, often mandated by the government, for drivers, contractors, physicians, and other businesses. In these cases, if harms occur, the harmed party can be compensated even at times without determining responsibility.

This last approach is one of the popular avenues proposed for enforcing governance of AI. If we frame governance in terms of contracts and licensure, we can then identify consequences for breaking those contracts, and require financial compensation that is indemnified through insurance. There are several distinct facets and proposals related to this. For example, Jacovi et al. (2021) (discussed in Chapter 5) proposed a contractual trust framework: trust as a commitment to an explicit performance contract. Another aspect of this sort of governance is licensure (Tate et al., 2016). As in both human drivers and in many occupations, a test is required to gain licensure, which permits a certain set of activities, often some with and others without oversight by others. Practices under a license are often also tied to liability, malpractice, or indemnification insurance, so that violations of trust can be linked to enforceable consequences and remuneration of the harmed party: financial awards, revoked licenses, increased insurance rates, and requirements for additional oversight.

These concepts are used in many settings outside AI, but for them to be adopted within AI governance they need to be codified through regulations and standards, such as those discussed in the Frameworks section. These include legal frameworks discussed earlier such as GDPR (European Parliament and Council of the European Union, 2016); EU AI Act (European Parliament and Council of the European Union, 2024), but also documents by standards bodies such as the NIST AI RMF (Tabassi, 2023), and corporate responsible AI programs (Microsoft, 2022; IBM, 2024; Services, 2025). These can also be codified into lifecycle or acquisition governance, such as the HRL and TRL maturity levels (See et al., 2019; Human Factors and Ergonomics Society, 2021) discussed in Chapter 3.

In practical terms, the focus on governance provides a framework for many kinds of evaluations of fairness, harm, responsibility, and trustworthy systems, but it often provides only limited guidance about how to implement and improve the fairness, transparency, or ethical outcomes of AI systems. We will discuss some of these in the next section.

9.8 Practical Interventions for Improved Ethical, Safe, and Fair AI

Many of the concepts in this chapter are informative but difficult to act on–we provide definitions and examples of the kinds of harms that might emerge, but without a toolkit for improving the situation. Nevertheless, throughout this handbook, we have identified many relevant methods, instruments, and interventions that can be used to improve design of human-AI systems in general. Many of these can be tailored or adapted in particular for addressing risks and harms of AI as well. For example, assessments of trust, mental models, and explainability might all incorporate specific elements of the AI risks framework discussed earlier in this chapter. There are also several specific approaches that have been proposed to specifically increase transparency, interpretability, and governance. These include:

• Datasheets for datasets, documenting dataset motivation, composition, collection, and recommended uses (Gebru et al., 2021)
• Self-assessment checklists such as the EU Assessment List for Trustworthy AI (ALTAI) (High-Level Expert Group on Artificial Intelligence, 2020).
• Internal algorithmic audit frameworks such as SMACTR (Scoping, Mapping, Artifact Collection, Testing, and Reflection) (Raji et al., 2020)
• Fairness and bias toolkits, including IBM's AI Fairness 360 and Aequitas (Bellamy et al., 2019; Saleiro et al., 2018). These include libraries of explainable AI algorithms, as well as guidance for data presentation and evaluation.
• Risk assessments, including lifecycle risk-management guidance such as the NIST AI Risk Management Framework (Tabassi, 2023)

These each represent carefully designed approaches to address AI risk, but many of them need to be implemented at the level of an organization, with buy-in from the development team and management. Two approaches are perhaps the most accessible and useful for someone involved in human-AI research and development, and we will describe them in greater detail. These include an evaluation for trustworthy AI (Shneiderman, 2022), and an approach for improving transparency through model cards and FactSheets (Mitchell et al., 2019; Arnold et al., 2019).

9.8.1 Trustworthy AI

Chapters 5 and 6 covered trustworthiness as a concept in general, and many ways of measuring trust and trustworthiness. In those chapters, factors that led people to judge AI or automation trustworthy included reliability, timeliness, and accuracy—factors that mainly influenced system performance. But just like transparency, trustworthiness takes on additional meaning when the systems begin to have large-scale societal impacts. Arinaitwe (2023) comprehensively examined trustworthy AI properties appearing in the literature, which included 56 distinct aspects involving both traditional notions of trust related to adoption and performance, and social and ethical aspects. These included interpretability, security, human control, robustness, reliability, explainability, fairness, auditability, safety, accuracy, efficiency, performance, traceability, transparency, provenance, explicability, accountability, equality, responsibility, liability, human autonomy, human agency, human oversight, privacy, data governance, societal wellness, lawfulness, ethical behavior, unbiasedness, verifiability, sustainability, ease of use, correctness, dependability, predictability, fidelity, accessibility, reproducibility,learnability, generalization, trusted for human-machine interaction, beneficence, resilience, validity, compatibility, trialability, representation, perception, sociability, collaboration, level of automation, appearance, susceptibility, reputation, persistence, understandability, faith personal attachment, and competence. From this, it is easy to see that trustworthy AI incorporates both traditional forms (reliability, performance, ability to collaborate), and the aspects related more to ethical and fair AI.

Nevertheless, the term “Trustworthy AI” is now most often used this second way (maybe interchangeably with the term “Responsible AI”). This inolves social risks (how are training data obtained and used), disclosure of interests, contractual trust elements (if the company says they do not use your interactions to train new AI, are they telling the truth?). Because the responsibility gap—the system itself cannot take responsibility—the notion of trust and trustworthiness does not follow the normative sense of the word (Hardin, 1996; Jacovi et al., 2021), and so it becomes important to understand both the organization responsible for the system, and the process they engage in to create the system.

In his book Human-Centered AI, Shneiderman (2022) proposed a Trustworthy AI scale which serves to illustrate how the concerns over trustworthy AI have expanded and shifted from the performance-focused notions of trustworthiness that were used for simpler automation. Summarizing his dimensions, they include:

1. Independent review
2. Audit trail
3. Training data collected/assessed
4. Fairness tested
5. Explainability implemented/tested
6. Two-month usage evaluation
7. Six-month usage evaluation
8. Stakeholder review/feedback
9. Internal review for failure incidents
10. Continuous review for refinement
11. External review by independent body

He proposed that these items could be used to examine an existing AI system to give a trustworthiness score. This is not the type of individual trust scales discussed in Chapter 6. Those focused on trust as a psychological state, either dispositional trust of an individual (are they generally trusting or distrustful of others or AI/automation/robotics), or directed trust (how much does a person trust a particular system after seeing it or interacting with it). Here, it is a rubric to evaluate a system and determine if it was developed with appropriate practices that promote trustworthiness. A related engineering approach is Arinaitwe (2023), who decompose trustworthiness into sub-attributes, propose quality-attribute scenarios and design tactics for AI-enabled systems, and offer a design checklist that can be used with architecture tradeoff analysis—again focusing on system and process evaluation rather than psychological trust.

Presumably, this rubric could be used to evaluate AI systems an organization is purchasing or embedding within their own product. Normally, decisions such as this might be governed solely by cost or capability concerns. This gives another metric, which may indicate possible long-term downside risks of adopting the system. It is also likely that almost no viable systems earn perfect scores, and so it allows one to evaluate trade-offs between systems, or decide between two equally-capable systems.

9.8.2 Model Cards

One practical intervention aimed at improving transparency—especially information transparency (Andrada et al., 2023)—is the model card (Mitchell et al., 2019). Mitchell et al. (2019) proposed model cards as short documents released with trained models to disclose intended use, evaluation procedures, performance across relevant demographic or phenotypic groups (including intersectional slices), and known limitations. The original motivation was focused on transparency: clarifying and disclosing appropriate use of the model (what a model is and is not good for), so that deployment outside the model's competence envelope is less likely. However, the method also supports other notions of transparency, especially disclosure of training sets, potential biases, and other factors that impact fairness and accountability. In this way, it may help prevent misuse and usage in ways that promote bias and risk.

Closely related documentation practices include supplier-style FactSheets for AI services, which organize purpose, performance, safety (including fairness and explainability), security, and provenance claims, and which can be evaluated by consumers and other stakeholders (Arnold et al., 2019). In Chapter cognitive-tutorials we return to model cards mainly as compact user-facing documentation that can support mental-model formation and appropriate use. Here, they can provide broader information, and represent a transparency and accountability intervention. They serve as a standardized disclosure format meant to make information about a model knowable both before and during deployment.

9.9 Conclusions

As AI and automation systems have become useful outside of academic and narrow industrial settings, many concerns related to ethical issues, fairness, and social risks have become prominent. Researchers and developers involved in human-AI interaction, including UI/UX, usability, human-centered design, and the like have often been focused more narrowly on improving understanding, trust, performance, and cognitive aspects of human-AI systems. However, the more social and ethical risks may also become the responsibility of these same team members.

Numerous frameworks with overlapping goals have been defined to deal with AI risks, governance, and trustworthy and responsible AI. Although some involve software to support aspects of bias or interpretability, many involve and require more complex commitments to systematic development processes that include documenting how models were created, testing fairness, and disseminating this information to users. Although we are still in the early stages of this, regulations are likely to emerge if the harms caused by AI become more frequent or noticeable. To the extent that the industry adopts such practices on its own before such regulations are passed, many potential risks might be avoided.

Acknowledgments

This chapter was originally developed as part of a course project for HF 5430 Human-AI Interaction, Michigan Technological University, Spring 2024.

Author Contributions

LS, CC, SW, JW: writing and research of individual sections; STM: Conceptualization; background research, writing in all sections; general editing.

Conflicts of Interest

The authors declare no conflict of interest.

AI Usage Statement

Generative AI models were used for additional research, to identify missing concepts, to support better organization, and for editorial tasks such as formatting, evaluating grammar/clarity and citation collation, glossary development, and creation of figures. Content, text, and ideas are otherwise original to the human authors.

How to Cite This Chapter

Sprague, L., Cischke, C., Walker, S., Wang, J (2026). Safe, Aligned, Fair, and Ethical Considerations for Artificial Intelligence and Automation. In Shane T. Mueller (Ed.), A Handbook of Human-AI and Human-Automation Interaction. https://pages.mtu.edu/~shanem/human_ai/