Chapter 12

Human--Machine Teaming

Joanne Wang, Katie Ulinski, Yifan (Jeff) Wang, Brandon Woolman, and Samantha Walker
Michigan Technological University

Automation bias: Giuseppe Romeo and Daniela Conti. 2026. Exploring automation bias in human– AI collaboration: a review and implications for explainable AI. Ai & Society 41, 1 (2026), 259–278.

Chen J. Y. (2018). Human-autonomy teaming in military settings. Theoretical issues in ergonomics science, 19(3), 255-258.

12.1 Introduction

Maybe link Ch. 2

Just as there are various considerations in human-human teaming, human machine teaming also involves a range of similar or distinct factors.

12.2 Hybrid Problem Solving

Humans and AI have distinct abilities that can be leveraged in problem solving. Using AI in problem solving may aid humans by providing more organization to searching for information (Amabile, 2020; Raisch & Krakowski, 2022). This allows for the human members of the team to focus on other aspects of problem solving that they are more suited for. However, AI may limit human problem solving abilities by creating and imposing formal rationality on the process (Lindebaum et al., 2020). Due to this possible limitation, the creation of these teams need to keep in mind how to best leverage the abilities of all agents, human and AI alike.

Raisch and Fomina (2023) suggest a hybrid problem solving model that addresses these concerns. There are three major processes: autonomous search, sequential search, and interactive search. Autonomous search combines both generative and predictive AI which creates solutions that human agents then choose from. Sequential search starts with the AI agent exploring the problem space and solution and then human agents continue the process. In this search type, AI can help to find information about the problem space and possible solutions that can allow the human to create the overall solution. Interactive search uses predictive and generative AI but the AI and human search at the same time. Insights from both the human and the AI can inform the others' search and help them to recontextualize the problem space. Each of these processes allow the human agent to have a large portion of the problem solving process while allowing the AI to excel at what it is traditionally used for.

One important aspect of hybrid problem solving is knowledge transfer and exchange (KTE). KTE is the process of exchanging relevant and meaningful knowledge between different agents (Keifer et al., 2005, Mitton et al., 2007). This process occurs frequently and is used for defining the problem space and creating meaningful solutions. Lavis et al. (2003) created a framework to create strategies around knowledge sharing that can inform KTE in hybrid teaming. It centers five questions:

12.3 What should be transferred? (The message)

1. To whom should it be transferred to? (The target of the message)

12.4 Who is the messenger?

12.5 How should it be transferred? (communication methods and the processes that support the transfer)

12.6 And with what success? (How do we measure successful communication)

Understanding how KTE is happening can affect trust in the system and how all members of the team view different aspects of the information or material provided by the AI agents. Creating better KTE can allow for better team situational awareness (Caldwell et al., 2022). Team situational awareness allows the team to acknowledge the skills of members of the team and to leverage those skills in meaningful ways. KTE allows the team to understand information about the team and themselves.

12.6.1 Centaur and Cyborg Users

The term centaur and cyborg can be used to refer to different strategies users have when using AI to aid in problem solving (Dell'Acqua et al., 2023). A centaur is a user who divides and delegates different portions of the problem solving process to the AI. The user decides what they believe the AI will be best at and compares that to the tasks that they know need to be completed. A cyborg is a user who completely integrates their task flow with the AI and continually interacts with the system. An important distinction between the two types of users is that cyborgs don't delegate tasks to the AI system, rather they integrate the actual work for the task with the AI. This may include subtasking like starting a sentence and asking the AI to finish it. It is not clear which approach is better and the choice seems to be based on user preference, experience, and trust in the system. Below is a table provided by Dell'Acqua et al. (2023) to better explain the differences between cyborg and centaur behaviors.

Practices and Descriptions

Centaur Practices

Use the individual's knowledge of the current strengths of generative AI relative to theirs to switch between human and AI for each of the modules/sub-task of the tasks accordingly throughout the workflow.

Examples of Behavior/practices:

Mapping problem domain: asking AI for general information related to the problem's domain for the human to use for their sub-task

Gathering methods information: asking AI for specific information on methods that the human is employing to solve their sub-task

Refining human generated content: users providing their own output and using AI to refine its presentation

Cyborg Practices

Use AI for each sub-task throughout the whole workflow. Apply principles based on current knowledge about how to best elicit useful outputs from AI an/or continually question AI and experiment to reach a better output. For example:

Assigning a persona: instructing AI to stimulate a specific type of personality or character

Requesting editorial changes to AI output: asking AI to make editorial changes to the outputs AI has produced

Teaching through examples: giving example of correct answer before asking AI a question

Modularizing tasks: breaking down tasks into multiple sub-steps for AI to execute

Validating: asking AI to check its inputs, analysis, and outputs

Demanding logic explanations: asking AI to explain a confusing output; or why a particular recommendation was made

Exposing contradictions: pointing out logical or factual inconsistencies

Elaborating: asking AI to bring more breadth of details and nuance on an interesting or unexpected point

Directing a deep dive: directing AI to focus on a particular data point, content, or task

Adding user's own data: adding data after an output is generated to re-do the analysis in iterative cycles

Pushing back: disagreeing with the output and ask AI to reconsider

12.6.2 Navigating the Jagged Frontier

Recent developments in artificial intelligence (AI) has expanded its use into industry. Chat-GPT 4.0 was recently used by Boston Consulting Group employees to make consulting decisions on issues of varying difficulty; they developed an idea of the "Jagged Technological Frontier" which posits AI abilities with perceived difficulty of questions (Dell'Acqua et al., 2023). While humans can identify tasks that are similar in difficulty, AI cannot accurately predict its capabilities to solve a problem. This phenomenon leads to the jagged frontier where the AI weaves between over-competence and under-competence for solving a particular problem based on the user's perceived difficulty of the problem. This postulate has influenced the field of human-machine teaming through characterizing AI capabilities in complex human problem-solving scenarios.

While understanding this jagged frontier can inform users that AI has limits, it might not explicitly help with navigating said frontier. If an AI might fall short on similarly difficult prompts, then the user would have to also identify the program's shortcomings. The user after repeated use of a program can learn to map this jagged frontier given enough prompts, and would be better prepared to identify mistakes (Kahr et al., 2023). A novice being contracted to use AI might not have the skills to identify errors and thus will fail to correct any shortcomings of their computerized teammate.

When AI capabilities extend far beyond the frontier, this might lead to a different problem: overreliance on the system. The partnership can become much more one-sided with the human relying on the machine to make most decisions (Dell'Acqua, 2022). Extending too far beyond the frontier might put the workload exclusively on the AI, instead of working in conjunction with the human. For example, drivers might trust mirror indicators of vehicles in their blindspot and merge into an adjacent lane without physically checking the blindspot themselves. Over-trust in a system, as discussed in chapters 5 and 6, can lead to a variety of issues: failures to recognize mistakes made by the AI, or identifying conditions which make the AI unsuitable for the job. With advancements of AI beyond the frontier, perceived difficulties of tasks may become irrelevant and lead to overreliance on the AI.

When users are not over-reliant on a system, it can be a fantastic critical decision-making tool. To use AI effectively, the user should be aware of the capabilities of their system and able to predict where a problem lands on the jagged frontier. Opacity of system is important for users to understand how the AI works (Lebovitz, Lifshitz-Assaf, & Levina, 2022). As outlined in chapter 8, an explainable system can help users make decisions about when to accept AI decisions. Explainability helps reduce overreliance on a system (Vasconcelos et al., 2023). Putting this in terms of the jagged frontier, users who can understand what the AI is doing will be less likely to over-rely on the system.

The jagged frontier as an interesting new take on human-AI teaming where the human cannot accurately predict if an AI can complete tasks of similar difficulty. In practical scenarios, it would take many interactions to map the frontier and understand what types of problems where the AI would fall short. This frontier might be ignored in cases where humans are over-relying on said AI. One way that AI can help people map the frontier is by increasing explainability and showing users how they make decisions. While AI continually improves and becomes more prevalent in daily life; this jagged frontier may expand to accommodate more difficult problems or become so complex that the frontier becomes irrelevant.

12.7 Defining field of HMT and 3Cs

In the human machine teaming field, there are various aspects to consider whether machines can become good agents. However, in a real-life environment, whether machines can be good team members also depends on the overlap of capabilities. They should be designed to allow both machines and humans to leverage their respective strengths to achieve better outcomes, rather than replicating human abilities and replacing them.

In their paper, Henry, K. et al. (2022) used clinician as an example. If machine learning systems have the same capabilities as clinicians, even if they are highly dependable, they are still perceived as a threat. Clinicians describe machines not as tools or forms of automation that may replace specific functions, but rather as partners. For instance, machines can promptly attract clinicians' attention to patients who need it. Therefore, these machines enhance the decision-making process guided by clinicians. Machine design should focus on enhancing clinical judgment rather than replacing clinical judgment. This also helps maintain adoption and prevents over-reliance on machines.

To assess whether the machine functions effectively as an agent in human machine teaming, there are ten indicators available for evaluation.

Design Content

Design Process

Transparency

Augmenting cognition

Coordination

Design specifics

Observability

Directing attention

Directability

Information presentation

Predictability

Exploring the solution space

Calibrated trust

Design process

Adaptability

Common ground

Co-allocation: Involves interdependence among necessary resources, where parties have independent goals and motivations. An example is when two groups try to schedule a conference room, they both need to use it on a certain day.

Cooperation: Involves interdependence among necessary resources and activities, but parties have independent goals and motivations. For instance, in a soccer game, different teams cooperate with each other while pursuing their own goals.

Collaboration: Involves interdependence among necessary resources, goals, motivations, and activities. For example, members within the same group collaborate to achieve a common goal, typically involving coordinated efforts and shared objectives.

goals and motivations

activities

resources

Co-allocation

Independent

Independent

Interdependent

Cooperation

Independent

Interdependent

Interdependent

Collaboration

Interdependent

Interdependent

Interdependent

12.7.1 Interview Methods as part of MITRE Framework for Human Machine Teaming

Subject matter experts (SMEs) are people who are considered experts in their domain due to experience and training (Williams, 2001). Subject matter experts have been used to assess and correct AI algorithms and machine learning models in the domain of artificial intelligence (Galli, 2024). One of the best ways to elicit information about a certain domain is to conduct an interview with a subject matter expert (McDermott et al., 2018).

Interviews with SMEs are often considered semi-structured interviews, meaning that they follow a certain framework, but the questions within each section of the interview are not asked in a specific order or worded in a specific way (Lopez et al., 2023). McDermott et al. (2018) provide a semi-structured interview framework for interviewing SMEs in the domain of human-machine teaming. The framework provided by McDermott et al. (2018) consists of the following parts:

• Summary of information and purpose: Start the interview by providing the SME with a brief background on why they are being interviewed and provide them with all necessary information and context to answer the following questions regarding development in AI or human-machine teaming.

• Top Challenges: In a similar manner to a task diagram, SMEs provide challenges they have had in their career and experience that are relevant to the questions that will be asked in the interview.

• Current State/Use: This portion of the interview is to gather information about how SMEs currently engage with automation/AI or have experienced human-machine teaming in the past.

• Critical Decision Method: This final section of the introductory questions is meant to examine the critical moments where decisions were made in an example from the subject matter expert's experience. This includes understanding the challenges the SME faces and how they overcome them, especially in regards to AI or human-machine teaming.

• Human-Machine Teaming Knowledge Audit: Examining brief examples regarding decision-making in human-machine teaming using probes and guiding questions to elicit information. This final part of the interview takes the most time to complete.

Semi-structured interviews, such as the one outlined by McDermott et al. (2018), have been used to gather information from SMEs regarding human-machine teaming and artificial intelligence. A recent interview study conducted by Lopez et al. (2023), used a similar style of semi-structured interview laid out by McDermott et al. (2018) to evaluate how subject matter experts view the relationship between the human and the AI in the human-machine team. The subject matter experts in this study were U.S. Air Force pilots with experience using autonomous teammates in the aviation domain (Lopez et al., 2023). In this study, they started by gathering information from the SMEs on their use of autonomous teammates and issues that they have encountered previously. This led to a larger discussion of the trust placed in autonomous teammates in the human-machine team and the ethics of using an autonomous teammate (Lopez et al., 2023). The semi-structured interview process allowed the researchers to gain a wide understanding of the two factors that their SMEs found to be most important, which allows for further research and testing (Lopez et al., 2023). Interview methods involving subject matter experts in the domain of artificial intelligence and human-machine teaming are important for understanding the challenges that have or may be faced in the process of developing a human-machine team, which can then be used to develop more effective autonomous teammates (Williams, 2001; Galli, 2024).

12.8 Coactive design

Coactive Design represents a fresh approach tailored to address the intricate roles undertaken by humans and robots as their applications extend into complex domains. Coined to encapsulate an approach to Human-Robot Interaction (HRI) design centered on interdependence as the primary organizing principle in joint activities (Johnson et al., 2010), the term "coactive" signifies more than just the involvement of multiple parties in an activity; it denotes the reciprocal and mutually constraining nature of actions and outcomes governed by coordination.

At the heart of this approach lies the acknowledgment of reciprocal actions and outcomes forged through synchronized efforts in joint endeavors. Here, participants share responsibility for attaining collective objectives, albeit with a trade-off of individual autonomy (Bradshaw et al., 2017). Departing from task-centric approaches, coactive design prioritizes teamwork over mere task completion, recognizing the symbiotic relationship between the capabilities and responsibilities of both humans and agents.

The Coactive Design method involves three main processes (Johnson et al., 2014):

• Identification process: Identifying the requirements for observability, predictability, and directability to support interdependence in the human-robot system.

• Selection and implementation process: Selecting and implementing control algorithms, interface elements, and behaviors that enable robots to fulfill their role as teammates.

• Evaluation of change process: Iteratively evaluating and refining the design based on feedback to enhance interdependence and teamwork between humans and robots.

By following the Coactive Design method, developers can translate high-level teamwork concepts into practical control algorithms and behaviors that promote effective collaboration and adaptability in human-robot interactions.

Johnson et al. (2014) demonstrated a practical application of the Coactive Design method. Following the Coactive Design method, an interdependence analysis table for the subtask of picking up the hose was constructed in accordance with the section on the identification process as follow:

In the identification process, the task decomposition, required capacities and team role alternatives were determined. While the colors are used to assess the individual's capacity to do the task under the "performer" columns, the colors are an assessment of that team member's potential to support the performer under the "supporting team members" as shown in fig. 2. Specifically, fig. 3 shows the meaning of different interdependence combinations based on the color scheme.

For instance, identifying a hose poses little challenge for an operator. We could have invested resources into crafting an autonomous algorithm for hose recognition, but its reliability would never match that of a human. Sometimes, the optimal choice isn't obvious. Take, for instance, the task of positioning the hand for grasping—neither the robot nor the operator consistently achieved perfection. In such scenarios, it proves advantageous to accommodate both approaches, enhancing system flexibility. Should the robot falter in hand positioning, the operator can step in as a backup. Moreover, there exist more than just two options for positioning hands during grasping (Johnson et al., 2014).

In the process of selection and implementation, Johnson et al. (2014) pioneered an interface development process centered around a 3D world model. Employing inverse kinematics as the robotic solution, they designed virtual arms to visually present the inverse kinematic solution to operators prior to execution. To enhance hand positioning accuracy and ease for grasping, they introduced a graphical representation depicting the valid grasp region of the hand.

In the evaluation of change, it is imperative not only to validate that the selected mechanisms meet requirements but also to assess their broader impact on the system. Some decisions, like adopting a third-person perspective, yielded positive effects across multiple tasks. Conversely, choices like defining the grasp region only influenced the specific requirement they targeted. However, certain decisions carried the potential for negative repercussions by either obstructing requirements or modifying interdependence relationships. For instance, in our VRC project, the utilization of scripting serves as a notable example.

Acknowledgments

This chapter was originally developed as part of a course project for HF 5430 Human-AI Interaction, Michigan Technological University, Spring 2024.

Conflicts of Interest

The authors declare no conflict of interest.

AI Usage Statement

Generative AI models were used for additional research, to identify missing concepts, to support better organization, and for editorial tasks such as formatting, evaluating grammar/clarity and citation collation. Content and ideas are otherwise original to the human author contributors.