Human-in-the-loop: Responsibility or illusion of control?

Why human involvement in generative AI is important, but often provides less control than it promises

A developer opens her editor. She has described a problem, provided context and set out a goal. A short while later, the AI tool delivers an answer: 5,000 lines of code. Structured, annotated and, at first glance, plausible.

Now she has to check whether everything is correct.

She scrolls. The code looks good. The structure seems well thought out, the naming is consistent, the logic follows a recognisable pattern. And yet: somewhere in these 5,000 lines there could be an assumption that is incorrect. A dependency that breaks later on. A technical decision that nobody consciously made.

How long would a full check take? Probably longer than the development. Perhaps significantly longer.

This is precisely where the problem begins: not with the AI, but with the expectations we place on the person behind it. We call them the ‘human-in-the-loop’ and reassure ourselves with the idea that control is assured.

But is it really?

What human-in-the-loop promises

At first glance, the term ‘human-in-the-loop’ sounds like a sensible response to the risks posed by generative AI. The machine does not work alone. A human remains involved. They check, evaluate, correct and make decisions. This seems to safeguard precisely what many people consider important when using AI: responsibility remains with humans.

That is precisely why the term is so appealing. It combines technological capability with a reassuring promise of control. Generative AI is allowed to support, accelerate, prepare and make suggestions. But in the end, someone still checks it. Ultimately, there is a human authority ensuring that nothing incorrect, inappropriate or risky is adopted without scrutiny.

This idea is not wrong. On the contrary: human involvement is indispensable in generative AI. The systems generate text, code, images, analyses or decision-making frameworks without truly understanding their meaning. They can formulate things convincingly and still be wrong. They can recognise patterns whilst overlooking important connections. They can deliver results that appear plausible but are based on false assumptions.

That is why we need people who understand the context, can interpret it professionally, assess quality and take responsibility. In this sense, human-in-the-loop is not a superfluous addition, but a necessary condition for the responsible use of generative AI.

It only becomes problematic when this necessary condition is turned into a convenient formula. Then it is suddenly enough if a human appears somewhere in the process, someone has seen the result, an approval is provided for, or a review step exists. On paper, this looks like control. In practice, however, it can mean very different things.

This is precisely where the real question begins: not whether a human is involved, but what role they actually play. Are they a decision-maker or a conduit? Checker or rubber-stamper? The responsible authority, or a retrospective safeguard for a process that has long since been shaped by the AI system?

The term promises control. Whether it actually achieves this, however, is not determined by the term itself. It depends on whether the human being in the process can truly understand, check and object.

The gap between presence and control

The fact that a human is involved in the process does not necessarily mean that they are actually exercising control. It is precisely this distinction that is often lost in the discussion about generative AI.

In many concepts, human-in-the-loop sounds like a simple solution. The system generates an output, the human checks it, and a decision is then made. This creates the impression of a clear, manageable process. The AI provides speed; the human provides judgement. In practice, this process is often far more fragile.

Control requires that humans understand what they are supposed to be checking. They must be able to recognise which assumptions underlie a result, what information is missing, which risks have been overlooked, and where a seemingly plausible answer leads in the wrong direction. This is demanding, particularly when generative AI does not merely make individual suggestions but delivers comprehensive results.

With 5,000 lines of code, this is obvious. But the same problem arises with a draft contract, a market analysis, a requirements specification or a decision-making document. The output appears complete. It is linguistically sound, logically structured and often astonishingly convincing. It is precisely this plausibility that makes the review more difficult.

Anyone who sees an obviously poor result will quickly become suspicious. Anyone who sees a good result reviews it differently. Perhaps more superficially. Perhaps with greater confidence. Perhaps with the tacit assumption that the rest will be fine if the first few paragraphs, functions or arguments seem coherent.

This is how the role of humans shifts. Genuine control easily turns into a plausibility check. A critical decision becomes a mere review. Responsibility becomes a tick in the box at the end of the process.

The problem, therefore, is not that humans are fundamentally unsuited to reviewing AI results. The problem lies in the expectation that this review should happen almost as an afterthought. Anyone tasked with checking needs time, technical understanding, clear criteria and the ability to reject a result.

A survey by Workday shows just how widespread this problem is: 70 per cent of executives are convinced that AI systems should always allow for human review. At the same time, 42 per cent of employees state that it is not at all clear in their company which systems actually provide for human oversight. The intention is there. The implementation is lacking. [1]

If any of these prerequisites are missing, humans remain formally involved. However, their control function becomes weaker than the term human-in-the-loop suggests. It is precisely this gap that is crucial. And it raises a question that has so far been asked too rarely in the debate on generative AI.

The wrong question arises of its own accord

If human oversight is so important in generative AI, one question seems to arise automatically: How do we involve humans in the right way?

This question sounds obvious. It fits with the concept of human-in-the-loop. It fits with governance models, approval processes and review stages. It sounds like responsibility. Nevertheless, it falls short.

It starts with the AI system and then asks where humans should be integrated. At the front, the AI generates; at the back, humans check. In between, a process emerges that may look technically sound but, organisationally, skips an important question: What exactly is the human supposed to decide at this point? This is precisely where the real bottleneck lies.

If an AI system generates 5,000 lines of code, it is not enough to say that a developer should check the output. It must be clear what she is supposed to check. Is it syntax, security, architecture, business logic, maintainability, performance, or the question of whether this solution even fits the problem? Each of these checks requires different knowledge, different time and different responsibility.

This does not apply only to code. Even with texts, analyses, requirements or decision-making documents, the central question is not simply whether a human is involved. What matters is what decision this person is supposed to make. Are they to find errors? Provide a technical assessment? Assess risks? Grant approval? Take responsibility for a recommendation? Or simply ensure that the result does not appear obviously wrong?

As long as this question remains unresolved, human-in-the-loop easily becomes a vague area of responsibility. The human is involved in some way, but their role remains unclear. They are expected to exercise control without it being defined what control means in this specific case. This creates a dangerous limbo. The AI system delivers results, the organisation relies on human review, and the reviewer is caught between both expectations. They are expected to enable efficiency and mitigate risks. They are expected to work quickly and review thoroughly. They are expected to use AI, but at the same time ensure that its errors do not slip through.

This turns a technical question into an organisational one. The real problem is not: How do we get humans into the loop? The real problem is: What decision needs to be made, on what basis, and by whom?

Human-in-the-loop therefore does not begin with the loop. It begins with the responsibility that is to be assumed within this loop in the first place.

The reversal: AI in the human loop

If the wrong question is how humans fit into the AI loop, the better question is obvious: how does AI fit into the human loop?

This shifts the perspective. The starting point is no longer the AI system and its output, which then has to be checked in some way. The starting point is the human task: understanding a problem, clarifying an objective, preparing decisions, and taking responsibility.

In this understanding, AI is not the process to which humans are added. It is a tool within a process that humans design and are responsible for.

This begins earlier than simply checking a finished result. Humans define what the issue is, which goals are to be achieved, which framework conditions apply, and which criteria a good result must meet. They decide which information is relevant, which boundaries must not be crossed, and at which points AI can provide meaningful support at all.

AI can then help to structure thoughts, generate variants, reveal patterns, or deliver initial drafts. But it operates within a framework that does not originate from itself. This framework arises from specialist knowledge, experience, context and responsibility.

It is also crucial that knowledge does not flow solely towards the AI. Anyone who provides a system with context, examples, requirements or rules is feeding knowledge into it. This knowledge must not be lost in the process. It must return in a form that people can understand, review and reuse.

This is precisely where an important difference lies compared to mere approval at the end. If humans only see a finished result, their role is limited. If, on the other hand, they set the framework, evaluate intermediate steps and classify results in a targeted manner, they remain capable of taking action.

AI in the human loop therefore does not mean less human involvement, but rather more conscious design. The human is not the final authority behind the system. They are the authority that decides what the system is used for, against what its results are measured, and when a result is good enough.

Responsibility shifts accordingly. It no longer hinges on a single review step at the end, but on the design of the entire process. Anyone wishing to use AI responsibly must therefore not only ask whether a human is involved. They must ask whether the process is designed in such a way that humans can truly lead, review and decide.

Only then does human-in-the-loop become more than a mere promise of control. The term then describes not just a human stage in the AI process, but a process in which human responsibility sets the framework.

Control starts before the output

If AI is to work within the human loop, it is not enough to simply incorporate a review step at the end of a process. Instead, the entire approach to generative AI needs to be rethought: earlier, more consciously and more concretely.

In practical terms, this means, first and foremost, making AI outputs smaller and more verifiable. A complete draft can be helpful. But the larger the output, the harder it is to exercise genuine control. Anyone tasked with checking 5,000 lines of code, a lengthy analysis or comprehensive documentation needs more than just a quick glance. It makes more sense to have intermediate steps, clear checkpoints and results that can be understood from a technical perspective.

This also involves asking the AI better questions. Not just: “Come up with a solution for me.” But rather:

• “What assumptions are you making?”
• “What alternatives are there?”
• “Where are the risks?”
• “What information is missing?”

Such questions turn a finished output into a verifiable work in progress. They help ensure that the AI is not treated as an answer machine, but as a tool for structuring and preparation.

Clear criteria are just as important. An AI result may be linguistically sound but technically weak. It may appear complete but overlook a key perspective. It may sound plausible but be based on a false assumption. That is why teams must know in advance what they are using to measure quality: technical accuracy, traceability, reliability, consistency, interoperability or decision-relevance.

For project work, this means: AI can do a lot of the groundwork, but it does not replace the need to clarify the problem. Particularly where requirements, processes, responsibilities or priorities are unclear, AI often merely generates visible ambiguity more quickly. This clarification remains a human task.

For leadership, this means not using human-in-the-loop merely as a reassuring mantra. It is not enough to formally leave responsibility with a person if that person has little time, overview or scope for decision-making within the process. Those who expect control must create the conditions for it: appropriate roles, clear responsibilities, realistic levels of scrutiny and the ability to reject AI results. This makes the use of generative AI an organisational issue – not just about which tools are used, but about how work is structured, how decisions are made and how responsibility is distributed.

Teams need to know when AI should provide support, when human clarification takes precedence and when a result must not be accepted. Leadership must ensure that these distinctions are not improvised on a case-by-case basis, but become part of the way we work.

Then human-in-the-loop becomes more concrete. It no longer merely describes the fact that a human is involved somewhere. It describes a way of working in which humans set the framework, use AI in a targeted manner and review results in such a way that responsibility can actually be assumed.

Figure: Human in the Loop of AI vs. AI in the Loop of Humans

Conclusion: Responsibility or the illusion of control?

Human-in-the-loop is not automatically wrong. On the contrary: generative AI requires human involvement, expert interpretation and responsible decision-making. Without people who understand the context, clarify objectives and critically assess results, AI remains a tool that can appear convincing without being reliable.

But that is precisely why it is not enough to simply incorporate humans into the process in a purely formal sense.

If, in the end, a person is faced with a finished AI result that is too extensive, too plausible or too difficult to comprehend, this does not yet constitute genuine control. In such cases, human involvement easily becomes a reassuring formality. Someone has looked at it again. Someone was involved. Someone has approved it. On paper, this looks like responsibility. In practice, it can be an illusion of control.

The crucial question is therefore not whether there is a human in the loop somewhere. What matters is whether that person can actually understand, review and challenge the output. Whether it is clear what decision is to be made. Whether criteria, responsibilities and boundaries are defined. And whether the organisation accepts that genuine control requires time, expertise and scope for decision-making.

Human-in-the loop only becomes responsibility when people set the framework within which AI operates. When they are not merely expected to check things at the end, but clarify beforehand which problem is to be solved, which information matters, which risks are relevant, and how a good result is measured.

What matters, therefore, is not the term itself. What matters is the design of the process. Human-in-the-loop can mean responsibility. But only if humans are not merely the final check on AI output, but rather the ones who set the framework within which AI can operate effectively. Otherwise, the promise of control remains little more than an illusion.

Notes:

[1] Workday Global Study: Closing the AI trust gap

If you want to shape the framework for AI within your organisation, then t2informatik is the right partner – a team with a genuine understanding of how responsibility arises in complex software development processes. Tell us about your situation. Alternatively, you can start by downloading our Snapshot.

Would you like to discuss ‘human-in-the-loop’ as an influencer or thought leader? Then share this post on your social media channels.

Michael Schenkel has published further posts on the t2informatik blog, including: