If AI makes the finished artefact a weaker signal of capability, what should we assess instead?

For a long time, recruitment has relied heavily on artefacts as proxies for capability.

A CV represents experience. A written application represents communication and judgement. A presentation represents someone’s ability to structure an argument. A take-home exercise represents how they might approach the work.

None of those signals has ever been perfect, but AI changes them in an important way.

Producing the artefact is becoming easier.

That does not make the artefact worthless. It does make it harder to infer how much of the capability we care about is demonstrated by the finished object alone.

The interesting response is not necessarily to remove AI from assessment.

If people will use AI in the job, deliberately assessing them in an environment where they cannot use it can create a different artificiality: we may end up measuring how they work without an important part of the toolkit they will actually have.

The alternative is to move assessment closer to the process.

Give someone a difficult problem. Let them use the tools they would reasonably have at work. Then pay attention to what happens.

How do they frame the problem before delegating anything? What context do they provide? What do they ask the AI to do? What do they keep for themselves? Which outputs do they accept, challenge or reject? Do they notice when the answer is plausible but wrong? Can they explain the choices they made? Do they know when not to use AI? Can they stand behind the result?

Those behaviours expose something a polished artefact increasingly may not.

Judgement.

This could make assessment more human rather than less.

If the finished document becomes a weaker signal, conversation becomes more important. So does observation. So does asking somebody to explain not only what they produced but how they arrived there and what they would do differently.

The artefact may become easier to produce while the judgement behind it becomes more valuable to assess.

There is a connection here to the apprenticeship problem.

If AI changes how people develop capability, organisations need to think differently about how that capability is recognised as well as how it is created.

But the questions are distinct.

The apprenticeship problem is about how people become experienced.

The assessment problem is about how we know what someone can actually do.

Both become harder if we continue relying on proxies designed for a world in which producing the artefact itself was a substantial part of the evidence.

There are still contexts where unaided work matters. If the purpose of an assessment is to establish whether somebody personally possesses a specific underlying skill, restricting tools can be entirely legitimate.

The important thing is to be explicit about what is being assessed.

If the job requires somebody to work effectively with AI, then their ability to direct it, question it, combine it with their own expertise and know when not to trust it may be part of the capability rather than a way of avoiding the test.

That shifts the recruitment question.

Instead of asking only, “Did this person produce a good answer?”

We can ask, “How does this person work when producing a good answer is no longer the difficult part?”