Human-in-the-Loop or Loophole? Targeting AI and Legal Accountability

There is no doubt that incorporating artificial intelligence (AI) within the targeting cycle has its operational advantages. During a complex urban scenario, the AI-driven decision-support systems (AI-DSS) has the potential to rapidly synthesize incredible volumes of data received from diverse ISR, signals intelligence, and other feeds, at a velocity no human could match. In theory, this innovation would sharpen a commander’s situational awareness, more accurately ascertain military objectives, and model collateral damage with newfound precision. 

The objective is to achieve a “cleaner” battlefield that features faster and more accurate targeting with lower collateral damage to civilians and civilian infrastructure. This integration of AI-driven systems is increasingly viewed as a mechanism to fulfill the core mandates of International Humanitarian Law (IHL). By providing commanders with more granular data and precise modeling, these tools are designed to facilitate the principle of distinction, which requires parties to target only military objectives and combatants. Furthermore, the speed and accuracy of such systems are intended to support the principle of proportionality, assisting decision-makers in ensuring that an attack’s collateral impact does not outweigh its intended military necessity. This promise of a more accurate and automated targeting system is desirable within the operational limits of IHL regarding the principles of distinction (between combatants and civilians) and proportionality (not excessive attack).

Nonetheless, as foresight systems transform from rudimentary advisory mechanisms to sophisticated recommendation systems, the potential legal liability increases the most. Artificial intelligence’s ability to process vast stores of information at high speeds adds to the “black box” problem. The operator of the system may not be able to fully comprehend the reasoning as to why the system has, through an automated process, pinpointed a particular person or object as a target. Consequently, the system must be evaluated against established matters of unquestionable legality. Such a situation produces an accountability void, which current international humanitarian law fails to address through existing standards.

The “HITL” Accountability Gap

Almost all military doctrines which implement AI promise “human-in-the-loop.” This standard suggests that a human being retains final control and thus legally and morally bears the responsibility of the engagement. But what does this mean in practice under existing legal frameworks? 

For example, when an AI-DSS self learns on terabytes of data and suggests some action which ends up being a flagrant violation of International Humanitarian Law (IHL), such as attacking a hospital which the AI was programmed to recognize as a command center, who would be responsible? 

• Is it the commander who, during a “time-critical” scenario, almost being in a desperate situation, accepted the AI’s recommendation which had almost a hundred percent probability?  
• Is it the analyst who, as a result of the target being ‘validated,’ does not have access to the AI’s millions of data points, which are used in the final conclusion? 
• Is it the person when the operational context is stripped away, who is abstract, and writes the algorithm months or years in the future?  

If the human operator’s functions are as simple as “servicing the target” which the machine picked, it is the AI which would be a ‘human-in-the-loop.’ This begins to pose a danger when human beings are left with an almost imaginary option to an answer. As the International Committee of the Red Cross (ICRC) points out, accountability is a mark of some “predictability” on how certain functions of the system will operate.

There is an ongoing concern in regard to the capabilities of advanced artificial intelligence outside the controls of the International Humanitarian Law frameworks. This “accountability by default” creates a legal paradox. By placing the entire burden on the final operator, the law ignores the reality that a human cannot exercise true agency if they lack the time or information to contest a machine’s high-confidence recommendation. It effectively transforms the human “in the loop” into a mere rubber stamp for automated processes, leaving them legally responsible for outcomes they could not realistically predict or prevent. 

A Framework for “Meaningful Human Control”

To address this gap, it is essential that commanders, operators, and their legal counsel move beyond the “in-the-loop” notion to address Meaningful Human Control (MHC) on its practical and definitional terms. This threshold has been elaborated on by numerous states and civil society organizations. MHC, defined by them is not just having a human present; a human must have legal agency to the actions taken by the AI as well. 

In the case of military lawyers and commanders justifying target selections, MHC can be described by three benchmarks.

The Comprehension Test: Can the human operator explain why the AI is recommending a particular target? At this point, an understanding of the system’s code is not important. What is essential is an understanding of the system’s inputs (e.g., “It is tracking movement”), the system’s logic (e.g., “It makes a decision on a flaggable target such as this vehicle”), and, importantly, the system’s shortcomings (e.g., “It cannot tell the difference between a combatant carrying a weapon and a civilian carrying a shovel”). Ultimately, an operator who cannot provide the “why” behind a recommendation is unable to legally or operationally validate that target.

The Time Test: Time does pose some interesting questions when considering the operator’s involvement. As planning cycles compress from hours to mere seconds, the pressure to accept an AI recommendation without scrutiny will intensify. When a person is forced to veto a machine’s decision within such a narrow time limit, they cease to exert true control and instead become a mere procedural cog. Consequently, automated legal reviews must establish realistic time standards to determine whether a human can actually perform a meaningful legal assessment under such high-velocity conditions.

The Legal Agency Test: Is the final decision on targeting made by a human? While the AI may provide the final technical recommendation, it must never be the final legal arbiter of an attack. An operator must retain the ability to disengage from the machine’s logic and evaluate the strike against the three constitutive principles of the use of force, focusing specifically on the requirement for restraint. The operator possesses the ultimate authority to dismiss any recommendation- even one with high statistical confidence- based on their unique understanding of the operational context or “gut feeling.” Ultimately, the AI serves only as a tool for computation, while the human remains the primary legal agent responsible for the decision.

Conclusion: Keeping the Law in the Loop

The reality is that AI targeted systems are not futuristic challenges, but real problems of today. Such systems, if intended, can help armed forces fulfill their IHL obligations as they are capable of processing information at scale.

Accountability, however, should not be outpaced by technology. Plugging the “human-in-the-loop” standard is bordering negligence. There should be a strong, rational, and legally supportable criterion for keeping meaningful human control. This creates a unique perspective for military leaders and jurists who focus not just on whether a human is “in the loop,” but to what extent they made their decision with full comprehension of the issue, in a timely manner, and with legal capacity. If they do not, then clearly, the loop constituted by the law has been closed.

The post Human-in-the-Loop or Loophole? Targeting AI and Legal Accountability appeared first on Small Wars Journal by Arizona State University.