Introduction

There is one word I always notice when I'm reading Ofsted reports.

Recently.

Leaders have recently introduced a new process.

They have recently appointed someone.

They have recently strengthened monitoring.

They have recently changed their approach.

Usually, those things are positive, they demonstrate that leaders have recognised a problem and are doing something about it. However, quite often another sentence follows.

It is too early to evaluate the impact.

That theme appears several times in last week's inspection reports raising an interesting question about quality improvement.

When does an improvement action actually become an improvement?

Because identifying a problem isn't enough.

Taking action isn't necessarily enough either.

Ultimately, leaders need to know whether that action has worked.

The difference between action and impact

One provider gives us a particularly useful example this week.

Inspectors found that leaders had a clear understanding of their strengths and areas for improvement. They understood why insufficient numbers of apprentices had completed their training and had taken action to address this.

Despite this Achievement was still judged Needs Attention.

Historically, too many apprentices had left before completing their programme. Inspectors recognised that leaders had acted, but concluded that it was simply too early to determine the impact of those actions.

That's a really important distinction. Leaders knew there was a problem and they understood some of the reasons behind it. They had responded, but the improvement cycle wasn't complete yet. The evidence of sustained impact simply wasn't there.

This isn't an isolated example

Similar language appears elsewhere in this week's reports. At another provider, leaders had introduced new quality assurance processes and strengthened leadership arrangements to bring greater consistency to provision. The Governance arrangements had also been strengthened.

Again, inspectors recognised these developments positively but some were still at an early stage and their impact could not yet be fully evaluated.

Attendance provides another example. New systems had recently been introduced to improve monitoring and oversight, but again there hadn't been sufficient time to evaluate whether they were making the intended difference.

The message isn't that these were the wrong actions. In fact, quite the opposite.

The message is that implementation and impact are two different stages of improvement.

New strategies need time to become culture

There is also an interesting example around inclusion.

At one provider, leaders had developed a clear inclusion strategy covering areas including culture and belonging, learner needs, achievement gaps, workplace inclusion, governance and accountability. However, the strategy was very new and had not yet become fully embedded within the organisation.

I think that's a key distinction, because writing a strategy is relatively easy,  embedding it is much harder!

People need to understand it.

Practice needs to change.

Leaders need to monitor it.

Learners need to experience it.

Then eventually, the evidence should tell us whether it has made a difference, and only then can we really evaluate its impact.

Compare that with sustained improvement

Last week's stronger judgements provide an interesting contrast. One provider received Strong judgements for both Inclusion and Achievement. Its Achievement narrative wasn't built around something leaders had recently introduced. Instead, inspectors described achievement as having improved over a sustained period.

Achievement had increased substantially over several years and was considerably above the national apprenticeship achievement rate. Most apprentices completed within their planned time, many achieved high grades and apprentices made extensive progress from their starting points.

That phrase “over a sustained period” feels like the other side of this week's story.

Not simply: We changed something.

But: We changed something, monitored it and can demonstrate that outcomes improved over time.

That's a very different position for leaders to be in.

Strong practice is continually refined

Another provider receiving several Strong judgements gives us a useful comparison. Here, inspectors described apprentices making excellent progress and producing consistently high-quality work.

What interested me most was the leadership narrative. Leaders worked closely with employers, communicated frequently and collaboratively, and regularly evaluated programmes with them to refine provision.

Those words are important.

Review. Evaluate. Refine.

They describe a cycle rather than an intervention.

Quality improvement isn't something that happens because inspection is approaching.It becomes part of how the organisation operates.

‘Recently’ isn't automatically a problem

There is an important caveat here. Not every use of the word recently in an inspection report indicates weakness, and nor should it.

Strong organisations continually introduce new ideas.

Curriculums change.

Technology changes.

Employer needs change.

Learner needs change.

Even one of the stronger providers this week had recently introduced a new inclusion strategy. Inspectors recognised sensible planned actions while also acknowledging that the approach was at an early stage and required further evaluation.

That's entirely reasonable.

Nobody should expect a newly introduced initiative to demonstrate years of impact immediately. The issue isn't whether something is recent. The issue is what leaders claim about it.

I think that's particularly relevant at this time of year.

SAR season: are we describing activity or impact?

We're right in the middle of Self-Assessment Report season and last week's reports provide a useful lesson for anyone writing or reviewing a SAR.

There can be a temptation during self-assessment to turn actions into strengths.

We've introduced...

We've implemented...

We've appointed...

We've developed...

We've launched...

Those statements describe activity, they don't necessarily describe quality, and they certainly don't automatically demonstrate impact.

A stronger self-assessment asks: What difference has it made?

And sometimes the most accurate answer will be: We don't know yet.

I don't think that's weak self-assessment. In fact, I'd actually argue it's a sign of mature self-assessment.

Your SAR doesn't need to pretend everything has worked

A genuinely useful SAR should distinguish between different stages of the improvement journey.

There are established strengths: practice that is embedded, consistently effective and supported by evidence over time.

There are emerging improvements: actions where early evidence suggests things are moving in the right direction, but there isn't yet enough evidence to claim sustained impact.

There are areas still requiring improvement: where actions haven't yet produced the intended outcome or where further intervention is needed.

Those distinctions matter because a SAR shouldn't be a sales document. It should be an accurate assessment of where the organisation is now and perhaps one of the most credible statements a provider can make is: We have taken action. Early indicators are positive, but we do not yet have sufficient evidence to demonstrate sustained impact.

That's honest. It's evidence-based. And, importantly, it tells leaders what they need to continue monitoring.

Don't confuse a plan with improvement

This week's reports reinforce something else that is worth remembering.

Having a quality improvement plan doesn't mean quality has improved. Having an action doesn't mean the problem has been solved. Having a new dashboard doesn't mean leaders understand what the data is telling them. Having strengthened governance doesn't automatically mean governance is effective. Having an inclusion strategy doesn't automatically create an inclusive culture.

All of those things may be the beginning of improvement, but the real question is what happens next.

Does practice change?

Do learners notice the difference?

Do outcomes improve?

Does the change become embedded?

Can leaders demonstrate that?

How long has something been ‘recent’?

There's another question I think leadership teams should consider. Sometimes an action genuinely is new, but sometimes organisations can spend a surprisingly long time describing something as recently introduced.

At what point should leaders expect to see evidence of impact? There isn't one answer.

Different interventions need different amounts of time, but every improvement action should have some idea of:

What will success look like?

What are the early indicators?

When should we expect to see them?

When will we formally evaluate impact?

Without that, an improvement plan can become a list of completed actions rather than a mechanism for improving quality.

Questions for leadership teams

As providers work through their SARs, I think this week's reports give us some useful questions to consider:

• Which of our strengths are genuinely established and sustained?
• Which improvements are still too new for us to evaluate properly?
• Where are we describing activity rather than impact?
• What early indicators tell us whether an intervention is working?
• When will we formally evaluate its impact?
• Who is responsible for challenging whether improvement has actually happened?
• Are there weaknesses we've known about for too long without securing sufficient improvement?
• If Ofsted inspected tomorrow, where might inspectors reasonably conclude that it is simply too early to see the impact?

Improvement needs time

I don't think the lesson from last week's reports is: Do everything earlier.

Quality improvement isn't that simple.

Some issues emerge unexpectedly. Some interventions genuinely take time. Sometimes leaders need to try something, evaluate it and change course.

The more useful lesson is: Don't wait until a problem becomes critical before responding to it.

Good self-assessment should help leaders identify emerging weaknesses early enough to act. Quality assurance should then help them monitor whether those actions are working and governance should provide the challenge that keeps improvement moving. That's how improvement becomes embedded rather than reactive.

Final thoughts

There is nothing wrong with the word recently. Every improvement has to start somewhere. However, last week's reports remind us that recognising a weakness and introducing an action aren't the end of the improvement journey.

They're the beginning.

For leaders, the harder work comes afterwards.

Embedding the change.

Monitoring it.

Challenging it.

Evaluating it.

And giving it enough time to demonstrate whether it has actually made a difference.

That's particularly important during SAR season, because a good SAR doesn't need to claim that every improvement action has already worked. It needs to accurately understand where the organisation is on its improvement journey.

And sometimes the most honest conclusion is: We've started. We think it's working. But we don't know enough yet.

The important thing is knowing what evidence you'll need before you can confidently say: It worked.

How AiVII can support

Through AiVII Consulting, we work with FE & Skills providers to support self-assessment, quality improvement and inspection readiness, providing independent challenge around not only what actions have been taken, but whether there is sufficient evidence to demonstrate their impact.

During SAR development, that can mean helping leadership teams distinguish between genuine strengths, emerging improvements and areas where further action is still required, ensuring the final self-assessment reflects the reality of the organisation rather than simply recording activity.

The AiVII Platform provides the technology layer, bringing together performance, quality assurance, learner voice, compliance and improvement activity so that leaders can see patterns and track change over time rather than viewing individual interventions in isolation.

Follow AiVII for weekly Ofsted insight briefings, toolkit interpretation and practical guidance for FE & Skills leaders.