Phrase the same request a little differently and a language model answers in a completely different way. According to the QuantPi project team, this is the most common problem that turns up when generative artificial intelligence is tested. The researchers built a platform that creates tests on its own, spots instability and bias and produces documentation for auditors. They also tried it on a system that recommends job candidates — and at the end you can check how much you trust AI answers.
Generative artificial intelligence increasingly helps to assess job candidates, serve customers and support business decisions. Its greatest strength is speed. But it has a weakness that is easy to overlook: it can react completely differently to a small change in the wording of a question, in language or in context. For companies this means a double task. They have to detect such anomalies, and they also need documentation proving that the risk has been tested and is under control.
The QuantPi project, supported by the European Innovation Council, took on this problem. It produced a platform for managing the risks of generative artificial intelligence, built around PiCrystal technology. The tool automatically creates test suites, checks the behaviour of the model and turns the results into audit-ready documentation linked to the requirements of the European Union Artificial Intelligence Act. The platform examines systems from three angles: unstable outputs, bias, and the availability of evidence that can be presented during an audit.
What most often turns up in such tests? Lukas Bieringer, head of policy and grants at QuantPi, answers without mincing words: “It is typically inconsistent or unstable behaviour under realistic variation of inputs – the same prompt class yields materially different outputs depending on phrasing, language or context.”
In practice, a system can pass a simple benchmark smoothly and then fail when it meets real users, who phrase their requests in their own way. Only then do biases and inequalities between groups come to light, differences that are invisible in the average results.
The team checked this on a concrete case. A candidate recommender system based on a large language model was assessed, with the recruiting platform Stepstone and the AI assurance laboratory TÜV AI.Lab as partners. The conclusion matters for anyone whose CV might end up in such a system. Testing AI tools used in hiring requires datasets that are large and representative. Only they allow intersectional testing, in which overlapping characteristics of different groups can be checked. Averaging works like a curtain here: it hides important differences in how the system handles individual segments.
QuantPi’s second idea is to separate the layer of evidence from the layer of presentation. It follows from a simple observation: different people in a company need different information about the same system.
To keep these views from drifting apart, there is one rule. Bieringer explains: “The key design principle: all views must derive from the same statistical evidence base, so that a board-level statement can always be traced to a specific test result.”
Automation does not mean that human judgment becomes unnecessary. People define what the system is for, choose which definitions of fairness or safety apply, set acceptance thresholds and decide whether to deploy, delay or withdraw a tool. The division of roles is clear: the platform measures risk, and accountable people decide what level of it is acceptable.
A test carried out once is not enough forever, either. Monitoring is triggered whenever the situation changes: after a model update, a revised prompt or a new retrieval index. Input drift, output drift and changes in rules also call for reassessment, because any of these can invalidate earlier results.
According to Bieringer, for many companies the biggest obstacle is no longer awareness of the rules. He puts it frankly: “What they cannot do is produce technical evidence of conformity that holds up in front of a notified body or auditor – particularly for generative systems, where traditional benchmark reports are insufficient and harmonised standards are missing to date.” QuantPi’s platform is meant to fill exactly this gap, converting technical tests into evidence that different teams can use before and after deployment.
Generative AI can be moody: the same question asked in other words may produce a completely different answer, and average test results can hide unequal treatment of particular groups. The QuantPi project shows that a sound assessment of such systems requires automated, repeatable tests, representative data and one shared evidence base for everyone, from the analyst to the board. The last word on which risk is acceptable, however, still belongs to people.
5 questions · one minute
EN 
CS
DE
ES
FR
HU
IT
PL
PT
РУ
SK
TR
УК
AR
中文