An AI skills matrix that measures more than confidence

What an AI skills matrix should show

An AI skills matrix maps roles to observable behaviours and records the evidence behind a learning judgement. It helps you identify practice needs and plan support. It should not turn a self-rated confidence score into a claim that someone is competent.

Begin with a small set of behaviours relevant to the work. Different roles may need different expectations. A person drafting an internal note, a manager approving it and a technical owner configuring a system do not need identical training or evidence.

Use behaviours that can be observed

For a general workplace programme, consider these starting points:

• Frame a task with a clear purpose, audience and expected output.
• Select information that is appropriate for the approved environment.
• Identify unsupported claims and verify important statements.
• Recognise when the output is incomplete or unsuitable.
• Explain the review and approval steps for the task.
• Escalate a question that falls outside the person’s authority.

Adapt the list with the people who own the work. A behaviour such as “understands AI ethics” may be too broad to review consistently. “Identifies an inappropriate data request in this exercise and uses the agreed escalation route” gives the assessor something concrete to look for.

Copy the matrix fields

Create a row for each role and behaviour. Include the required level, current evidence, evidence date, reviewer, next practice activity and next review date. Keep notes about the context: demonstrating a task with one tool and dataset does not automatically establish capability in every environment.

For a simple local rubric, you could use:

1. Not yet observed: there is no suitable evidence.
2. With support: the person completes the behaviour with guidance.
3. Independently in the agreed task: the person completes and explains it without prompting.
4. Supports others in this task: the person can identify common mistakes and give useful feedback.

This is a suggested internal rubric, not an accredited proficiency scale. Agree examples of each level before using it across reviewers. “Not yet observed” should remain different from “unable to do it”.

Set different expectations by role

For an illustrative drafting role, independent use might mean producing a checked draft from approved information and following the normal review process. For a reviewing manager, the expectation might include recognising when the draft needs a subject expert and deciding whether the workflow remains appropriate.

A champion who supports colleagues may need to demonstrate the exercise and explain the escalation boundary. That does not make the champion responsible for approving new tools or data uses.

Keep the champion charter aligned with the matrix so the learning expectations match the actual responsibility.

Collect proportionate evidence

Use a suitable task, a short explanation and reviewer feedback. Avoid gathering a large amount of personal or workplace information simply because the matrix has space for it. Agree who can see records, how they will be used and how learners can correct an inaccurate judgement.

Self-report can help start a conversation and reveal where people want support. Pair it with observed practice before using the matrix for decisions that require stronger evidence. Explain that distinction to both learners and managers.

Keep it alive without making it a bureaucracy

Review the matrix when tasks, tools or responsibilities change. Use the next-practice column to drive action, then update the evidence after the activity. A colourful spreadsheet that never changes a training decision is not doing useful work.

Start with the AI training needs analysis template to choose the tasks. Explore AI foundations for every role, or contact Experrt about turning role expectations into practical learning and review activities.