We are measuring AI literacy in the wrong way

And what companies should be measuring instead

Generative AI now provides answers to questions that often require considerable specialist knowledge to assess. This applies to medical diagnoses just as much as it does to legal assessments, economic analyses or technical issues. The results often seem convincing, even when they contain errors. A study from the field of medical training demonstrates just how difficult it can be to spot such errors.

Doctors undergoing specialist training in general practice were presented with medical answers generated by ChatGPT 4o. Their task was to distinguish correct answers from those containing errors. On average, they succeeded in doing so in only 55 per cent of cases. Particularly noteworthy: half of the answers presented were correct, whilst the other half were incorrect. The success rate was therefore only slightly higher than what would be expected from pure guesswork. This raises a question that extends far beyond the field of medicine: How well can we judge whether a convincingly worded AI output is actually correct? [1]

Many AI literacy assessments focus on different areas. They test knowledge of AI, prompting, specific tools, data protection or governance. Companies also track how many employees have completed training or how intensively certain AI tools are being used. This information is certainly valuable. However, it often remains unclear whether employees can reliably assess the quality of the generated results.

We measure a lot, but not necessarily the right things

The EU AI Act explicitly makes AI literacy a priority for organisations. Article 4 requires providers and operators of AI systems to take measures to support the development of relevant skills among their staff and other relevant individuals. In doing so, they must take into account, amongst other things, their technical knowledge, experience, training and the specific context of use. [2]

Many companies now use competence assessments, skills matrices, maturity models and learning journeys. I believe this is fundamentally the right approach. Anyone wishing to develop AI adoption in a targeted manner should know where the organisation stands and what impact its own development activities are having.

It becomes difficult when we equate different metrics. A high training rate primarily says something about participation. A licence rate shows how widespread access to certain tools is. And when employees state that they are able to assess AI results well, what we know above all is how they rate their own abilities. Whether they actually act competently therefore remains largely unclear.

The underestimated skill: judgement

Traditional assessments often focus on fundamental knowledge, prompting, tool knowledge, and questions relating to ethics, data protection and governance. In day-to-day working life, there is another skill that, in my view, deserves more attention: employees must be able to recognise when an AI output that sounds plausible is incorrect, incomplete or problematic.

I refer to this here as judgement. By this I mean the ability to assess the quality of a generated result appropriately, to distinguish correct statements from incorrect ones, and to evaluate one’s own certainty as realistically as possible.

Generative AI, in particular, does not make this easy for us. Poor output can appear linguistically excellent, a false argument can sound plausible, and a fabricated source can look like a genuine one. Therefore, simply being able to formulate a good prompt does not mean one is capable of reliably assessing the quality of the result.

Self-perception and actual abilities

A systematic review by Tomáš Lintner demonstrates just how difficult it is to carry out a meaningful assessment. He examined 16 scientifically developed scales for measuring AI literacy. Only three of these were performance-based; thirteen were based on self-reports. Furthermore, key measurement properties of many scales have so far been examined only to a limited extent. For example, none of the analysed scales had been tested for cross-cultural validity or measurement error. [3]

Self-assessments certainly have their place. If I want to know how confident employees feel when working with AI, a self-assessment provides me with useful information. However, to assess actual abilities, I need other methods, such as specific performance tasks.

A study by Kit Double also demonstrates just how far self-perception can diverge from actual ability. The study examined what is known as metacognitive monitoring – put simply, the ability to assess the quality of one’s own judgement. Participants first rated their own monitoring ability. This was then measured using specific tasks and confidence ratings.

Several of the objective measures used even revealed a negative correlation between perceived and actually measured monitoring ability. Other measures showed no such correlation. The results make it clear how cautiously we should interpret self-reports. Just because someone considers themselves competent does not necessarily mean they actually are. [4]

How judgement could be measured more effectively

A pragmatic approach to measurement could involve specific tasks. For example, employees are presented with several AI outputs, some of which are correct and others of which contain errors. After each result, they decide whether they would trust it and also indicate how confident they are in their judgement. This makes it possible to observe how reliably a person identifies errors and how well their subjective confidence aligns with the actual accuracy rate.

Let’s imagine two people who each get eight out of ten tasks correct. At first glance, both appear to have comparable abilities. However, the first person tends to be rather uncertain, particularly regarding their incorrect answers, whilst the second defends their misjudgements with great conviction. In everyday working life, this difference can have significant consequences, because people who recognise the limits of their own judgement are more likely to seek additional information or verify a result when in doubt.

Identifying errors is still not enough

Markus Langer, Kevin Baum and Nadine Schlicker propose examining human oversight of AI using Signal Detection Theory. In doing so, they distinguish, amongst other things, between sensitivity – that is, the ability to distinguish incorrect outputs from correct ones – and response bias. The latter describes how much evidence a person needs before an output is classified as problematic. [5]

What does this mean in practice? Imagine two employees who see a suspicious AI output. Both have, in principle, sufficient specialist knowledge to recognise the error. Person A checks the result, corrects it and only then uses the output. Person B thinks to themselves: ‘It’ll be fine,’ and accepts it.

In a standard competence test, both might achieve the same score. In actual work, however, different outcomes arise. Competence alone does not guarantee performance. If we want to understand how well people work with AI, we must therefore also consider their behaviour.

What organisations should be measuring instead

In my view, we need at least three perspectives to properly assess the competent use of AI within an organisation:

1. Capability: Can people do it?

The first level concerns actual capabilities. Do employees understand the basic workings and limitations of AI? Can they recognise suitable use cases, review generated results from a technical perspective and identify errors? Do they know when additional checks are necessary?

Performance-based assessments can provide valuable insights here. Instead of a general question such as ‘How well can you evaluate AI results?’, employees could be asked to complete specific tasks: ‘Here is an AI result. Which statements are correct, where might there be errors, and how confident are you in your assessment?” I consider such tasks to be more meaningful than pure self-assessments, particularly when it comes to evaluating judgement.

2. Behaviour: Do people actually do it?

The second level focuses on actual behaviour. Employees may well know that they should check an AI result. Whether they actually do so when under time pressure is another matter.

It is therefore of interest to organisations to understand how AI is used in day-to-day work. Are generated results checked? Do employees verify important decisions using additional sources? Are identified errors corrected before information is passed on? What impact does the use of AI have on the quality of work?

Depending on the context, quality audits, work samples, process data, peer feedback or team surveys can help answer these questions. The distinction between ability and behaviour is crucial; after all, we want to know whether existing skills are actually being applied in day-to-day work.

3. System: Does the organisation facilitate good behaviour?

At the third level, the focus shifts to the organisation itself. After all, employees operate within specific structures, expectations and framework conditions.

They may well be fully aware that an AI result should be checked. However, if every additional minute spent on checking is regarded as inefficiency, this behaviour becomes less likely nonetheless. Teams may wish to experiment with AI, but are held back by a culture of blame in which every mistake is penalised. Or managers may expect rapid adoption, yet provide little scope for trying out new ways of working.

In such situations, it is worth taking a closer look at the organisational conditions. Are employees given sufficient time to learn and experiment? Are quality standards for AI-supported work clearly defined? What behaviours are encouraged through formal and informal incentives? Can teams speak openly about mistakes and poor results? And do managers have the necessary tools to support the development of new ways of working?

Interviews, focus groups, audits and organisational surveys can help to highlight such interrelationships.

Figure: Measuring AI literacy meaningfully

And what about learning?

In this model, learning can be examined through change over time. We can observe whether employees recognise problematic AI outputs more reliably, adapt their behaviour, and whether better routines develop within the organisation.

Before-and-after measurements are particularly useful for this. An organisation does not necessarily need to know whether its average AI literacy currently stands at 63 or 67 points. A far more helpful question might be: Are we improving? If, following a learning intervention, employees recognise errors more reliably, assess their own confidence more realistically and check generated results more carefully in their day-to-day work, a meaningful development becomes apparent.

Training feedback and self-assessments can complement this perspective. Participation rates and usage data also provide information. However, it is important to correctly interpret the significance of the individual metrics. Course participation rates measure attendance at learning programmes. Licence rates provide information on access to tools. Self-assessments reveal subjective perceptions, whilst performance tasks measure specific skills and behavioural data show how these are applied. Organisational diagnostics, in turn, help to identify conditions that are either conducive or hindering.

Each of these metrics sheds light on a different aspect. Only by combining them meaningfully is it possible to gain a nuanced perspective.

AI literacy does not require a perfect assessment

At present, there is no single assessment tool that is both scientifically sound and practical to use, and which fully captures AI literacy in a corporate context. Lintner’s review highlights both the diversity of existing tools and the gaps in their validation to date.

Perhaps the search for that one perfect assessment is the wrong approach anyway. Companies could gain significantly more by addressing various questions in a targeted manner using appropriate methods:

• Are our employees competent in dealing with AI?
• Do they apply these skills in their day-to-day work?
• And do we, as an organisation, create the conditions that make positive behaviour likely?

Capability, behaviour and system. Anyone who distinguishes between these three levels and observes their development over time will gain a more nuanced picture than the next AI literacy traffic light system can provide. Perhaps we should therefore focus less on the question of what competence score an organisation achieves, and more on what its employees are actually capable of, how they act, and what conditions support them in doing so.

Notes:

Are you interested in further insights from Dr Joël Krapf on artificial intelligence in the workplace? Then it’s definitely worth taking a look at Proof2Practice.

[1] Zhou, J. et al. (2025): Integrating AI into clinical education: evaluating general practice trainees’ proficiency in distinguishing AI-generated hallucinations and impacting factors. BMC Medical Education.
[2] European Union: Artificial Intelligence Act, Article 4.
[3] Lintner, T. (2024): A systematic review of AI literacy scales. npj Science of Learning.
[4] Double, K. S. (2025): Survey measures of metacognitive monitoring are often false. Behaviour Research Methods.
[5] Langer, M., Baum, K. & Schlicker, N. (2025): Effective Human Oversight of AI-Based Systems: A Signal Detection Perspective on the Detection of Inaccurate and Unfair Outputs. Minds and Machines.

Would you like to support Dr Joël Krapf and discuss AI literacy? Then please share this post on social media or within your organisation.

Dr Joël Krapf has published three more posts on the t2informatik Blog: