Artificial intelligence systems

This litigation is only now emerging, and two opposite claims circulate about it: that nothing can be established, and that anything can. Both are false. A great deal can be established; some things cannot, and it is worth knowing which before a case is built on the wrong assumption.

Where the disputes come from

• Delivery of a system that does not perform as promised. An automated assistant that answers wrongly, a classifier that errs beyond the agreed threshold, a service that costs a multiple of the estimate to run.
• Harm caused by an output. Defamatory content, an answer that induced a damaging decision, an erroneous recommendation.
• Training data. Copyright-protected material, personal data processed without a lawful basis, confidential information ending up in a third party's system.
• Authorship. Who wrote a text or a program — a person or a generative system? The question has copyright consequences and, in academic and professional settings, disciplinary ones.

What can be established

How the system is built

This is the solid part. A system built on language models is not an undifferentiated black box: it is an architecture of identifiable components — which model is queried and in which version, which system instructions are placed before the user's request, which documents are supplied at request time in a retrieval-augmented architecture, which external tools it may invoke, which controls sit on the output. All of that is readable in the code and configuration, and comparable with what was contracted for.

In most supply disputes that is where the answer lies: not in the model's statistical behaviour, but in what the supplier built around it and in what it said it had built.

What was actually asked and answered

Where the system keeps interaction logs — and serious systems do, because they need them to operate — every individual request can be reconstructed precisely: the text sent, the context added automatically, the response produced, the model and version used, any tools invoked. This is ordinary forensic material and is handled like any other log.

The critical issue is retention: many providers keep such logs for short windows. In a case about a generative output, the first thing to do is to have them frozen.

Reproducibility, with a caveat

«Can you show that this system, given that question, produced that answer?» The honest answer is that one can show it can produce it, not that it necessarily did at the time.

Generative models are non-deterministic by construction, save under configurations that remove the randomness; and the supplier updates the model and the instructions over time, often without public versioning.

The origin of a text or an image

Here the greatest caution is required, and it is worth being explicit because unsustainable claims are in circulation. Tools that purport to identify text generated by a language model have high error rates that fall systematically on people writing in a language that is not their first: they are not usable as evidence, and presenting them as such is improper.

Metadata and technical marks are a different matter: some generation systems embed provenance information or watermarks in the files they produce, and the document in which a text was written retains a revision history showing whether it was typed progressively or pasted in wholesale. Those are real circumstantial elements, acquired and assessed like any other data; automated stylistic analysis is not.

Why I can take this on

Because I have built these systems and put them into production: retrieval-augmented architectures, multi-agent systems, the lifecycle management of generative components through prototyping, testing, release and monitoring. I know where a supplier writes logs and where it does not, which settings change a model's behaviour, and which commercial promises have no technical counterpart. Very few technical experts can bring that into a courtroom today.

Further reading: what can genuinely be established about an AI system →