What is the human-in-the-loop?
Table of Contents: Definition – Human involvement – Importance – How it works – Examples – Advantages and limitations – Questions from the field – Notes
Smartpedia: Human-in-the-loop refers to an approach in which humans actively monitor, review or control AI systems to ensure the desired outcomes are achieved.
Human-in-the-loop as a concept for human-machine collaboration
Generative AI systems can create texts, generate images, produce code, analyse data or prepare decisions. Their outputs often appear plausible, but are not automatically correct, complete or appropriate. They can produce incorrect information, overlook relevant connections, adopt biases from training data, or deliver results that do not align with the actual objective.
Particularly in technically, legally or ethically sensitive areas, it is therefore not sufficient to allow AI systems to operate unchecked. The more automated systems influence decisions or shape processes, the more important it becomes to determine at which points humans should intervene, review or take responsibility.
Human-in-the-loop refers to a concept of human-machine collaboration in which humans are specifically integrated into automated or AI-supported processes. They monitor, evaluate, correct or confirm results, thereby combining technical efficiency with human judgement, contextual understanding and responsibility.
Levels of human involvement
Not every AI system involves humans in the same way. The key factor is whether humans actively review individual results, merely monitor the system, or no longer intervene in the process at all. This gives rise to different levels of human involvement.
Human-in-the-loop – sometimes abbreviated to HITL – describes the most active form of involvement. The AI system generates suggestions, assessments or results, which are then checked, corrected, confirmed or approved by humans. Humans are thus an integral part of the process.
Human-on-the-loop – sometimes simply referred to as HOTL – means that the system operates largely autonomously but is monitored by humans. They do not intervene in every single decision but can step in if anomalies, errors or risks arise. Here, humans primarily assume a control and supervisory role.
Human-out-of-the-loop – HOOTL for short – refers to systems that operate without direct human involvement. Decisions or actions are made or carried out automatically. In this case, humans are not part of the ongoing decision-making process.
The main difference between these levels lies in how closely humans are involved in the actual decision-making process. In human-in-the-loop (HITL), they play an active role; in human-on-the-loop (HOTL), they monitor the process; and in human-out-of-the-loop (HOOTL), there is no direct human control.
Why is human-in-the-loop important?
AI systems can process large amounts of data, recognise patterns and generate results at high speed. However, they are not infallible. Their outputs depend on training data, model architecture, inputs and the context of use. This can lead to results that appear plausible but are factually incorrect, incomplete, distorted or unsuitable for the specific use case.
A key problem is erroneous or fabricated content. Generative AI can produce so-called ‘hallucinations’ – information that is convincingly formulated but does not correspond to the facts. In simple application scenarios, this may be inconsequential. However, in fields such as medicine, law, finance, human resources or public communication, such errors can have significant consequences.
Added to this are biases in data and decisions. If training data contains societal biases, historical inequalities or one-sided perspectives, AI systems can adopt and reinforce these patterns. Human review can help to identify such risks, contextualise results and correct them where necessary.
Understanding context is also a key reason for human-in-the-loop approaches. AI systems recognise patterns but do not automatically understand social, cultural, technical or organisational contexts. Humans can assess whether a result is appropriate to the situation, the objective, applicable rules and ethical requirements.
Human-in-the-loop is therefore important for combining automated efficiency with human accountability. Humans do not necessarily perform every task themselves, but intervene where control, evaluation, context or accountability are required.
How does human-in-the-loop work?
Human-in-the-Loop works by integrating human intervention into automated or AI-supported processes in a targeted manner. Humans do not take over the entire process, but are involved at those points where their judgement is particularly important. This can take place before an AI system is used, whilst it is running, or after a result has been produced.
A typical application is data annotation. Humans label training data – such as images, text, audio files or documents – so that an AI model can learn from it. [2] The quality of this human preparatory work directly influences how reliably the system will perform later on.
Another approach is the verification of results. The AI system generates a suggestion, an assessment or a decision template. Humans then check whether the result is correct, appropriate and consistent with the respective objective. Depending on the process, they can approve, correct, reject or escalate the result for further processing.
Feedback loops are also a key component. When humans evaluate or correct results, this feedback can be used to improve the system. In Active Learning, the model specifically requests human assistance when it is uncertain about a prediction. [3] In Reinforcement Learning from Human Feedback (RLHF), a model learns from human preferences which results are considered more helpful, appropriate or reliable. [4]
Human-in-the-loop is therefore not a single step in the process, but a design principle for AI processes. It is crucial that human involvement is incorporated where it improves quality, safety, traceability or accountability.
Examples of human-in-the-loop
Human-in-the-loop is primarily used in situations where AI systems prepare decisions, evaluate content or support processes in which errors can have significant consequences. The nature of human involvement can vary depending on the specific use case.
In content moderation, AI systems can pre-filter posts, images or comments and flag suspicious content. Humans then check whether the content actually breaches rules, whether context needs to be taken into account, or whether a decision should be escalated.
In medical imaging, AI systems can flag abnormalities in X-rays, MRI scans or CT scans. Doctors evaluate the findings, consider further information and make the medical decision.
In lending, automated systems can analyse data and provide an assessment of creditworthiness. Human experts can review this assessment, evaluate special cases and ensure that decisions remain fair and transparent.
In software development, AI systems can suggest code, analyse errors or generate tests. Developers review the suggestions, adapt them to the context of the project and take responsibility for quality, security and maintainability. [5]
In customer service, AI systems can classify enquiries, generate suggested responses or process simple requests automatically. Humans handle complex, emotional or critical cases and ensure that the response is appropriate to the situation.
These examples show that human-in-the-loop does not mean preventing automation. Rather, it is about combining AI systems with human review and responsibility in areas where quality, fairness, security or trust are particularly important.
Advantages and limitations of human-in-the-loop
Human-in-the-loop can improve AI-supported processes by incorporating human oversight where automated systems reach their limits.
- Humans review results, identify errors, take context into account, and can correct decisions before they lead to undesirable consequences. This increases the reliability of the overall system.
- An important advantage lies in control. AI systems can provide suggestions or preliminary decisions, but humans retain the ability to assess, modify or reject them. This is particularly relevant when results have technical, legal or ethical implications.
- Transparency and trust can also be strengthened through human-in-the-loop approaches. When it is clear when humans are involved and what role they play, AI-supported processes become more transparent. Users are better able to assess whether a result has been generated automatically, reviewed by a human, or developed collaboratively.
However, human-in-the-loop also presents challenges, as human involvement must be organised, funded and quality-assured.
- Human review takes time, requires specialist knowledge and can slow down processes. The larger the data sets or the more frequently decisions need to be made, the more difficult it becomes to scale human involvement economically and organisationally.
- Furthermore, humans are not infallible either. Fatigue, time pressure, prior knowledge, personal judgements or unclear guidelines can lead to inconsistent or biased assessments. Human-in-the-loop therefore only improves a system if roles, criteria and responsibilities are clearly defined.
- Human-in-the-loop offers particular added value when human involvement is used in a targeted manner. The key is not to incorporate as many manual checks as possible, but to design the right intervention points: where uncertainty, risk, context or responsibility are particularly relevant.
Human-in-the-loop is therefore less a guarantee of error-free AI and more an approach to making automated systems more controllable, traceable and accountable through targeted human involvement.
Questions from the field
Here are some practical questions and answers:
Does human-in-the-loop automatically make generative AI safe?
No. Human-in-the-loop can reduce risks, but it cannot guarantee complete safety. People, too, can make mistakes, be biased in their judgements, or make the wrong decisions under time pressure. For HITL to be effective, it requires clear roles, transparent criteria, suitable tools, and sufficient time for review
When is human-in-the-loop particularly important?
Human-in-the-loop is particularly important when AI outcomes may have technical, legal, ethical or economic implications. This applies, for example, to sensitive data, unclear situations, exceptional cases or decisions that directly affect people. The higher the risk of an incorrect or inappropriate outcome, the more important human involvement becomes.
What is the difference between manual checking and the human-in-the-loop approach?
Manual checking often involves a person reviewing a process or result after the fact. The human-in-the-loop approach goes further: the AI can continue to analyse data, make suggestions, generate code or prepare decisions. Humans are specifically involved where verification, evaluation, correction or approval are required. The key, therefore, is not to replace automation with manual work, but to combine automated processing and human control in a meaningful way.
Who is responsible when generative AI produces incorrect results?
Responsibility remains with the people and organisations that use AI systems. AI can make suggestions, generate content or prepare decisions, but it does not assume any legal, professional or ethical responsibility. Companies must therefore ensure that AI results are properly reviewed, assessed and approved. In this context, human-in-the-loop is an important mechanism for putting review obligations, due diligence and internal responsibilities into practice.
Will human oversight become a bottleneck in the future, as generative AI produces ever more content, lines of code, suggestions or decision-making templates?
Notes (partly in German):
If you like the article or would like to discuss it, please feel free to share it in your network. And if you have any comments, please do not hesitate to send us a message.
[1] What are the prerequisites for HOTL?
[2] Datenannotation erklärt: Wie KI aus beschrifteten Daten lernt
[3] Wikipedia: Active Learning
[4] IBM: What is reinforcement learning from human feedback (RLHF)?
[5] How effective are AI-supported code reviews really?
And here you will find additional information from our t2informatik Blog:
Human-in-the-loop: Responsibility or illusion of control
Promoting AI skills in your company
When AI writes the requirements