How we reach a rating

To support the transparency and consistency of our judgements, we will introduce a scoring framework into our assessments.

Where appropriate, we’ll continue to describe the quality of care using our 4 ratings: outstanding, good, requires improvement, or inadequate.

When we assess evidence, we assign scores to the key evidence categories for each quality statement that we’re assessing. Ratings will be based on building up scores from quality statements to an overall rating.

This approach makes clear the type of evidence that we have used to reach decisions.

Scoring

Using scoring as part of our assessments will help us be clearer and more consistent about how we’ve reached a judgement on:

• the quality of care in a service
• how well a local authority is delivering its duties under the Care Act
• the performance of an integrated care system

For example, for a rating of good, the score will tell us if this is either:

• in the upper threshold, nearing outstanding
• in the lower threshold, nearer to requires improvement.

Similarly, for a rating of requires improvement, the score would tell us if it was either:

• in the upper threshold, nearing good
• in the lower threshold, nearer to inadequate.

Our quality statements clearly describe the standards of care that people should expect.

To make things clearer and more consistent, we’ll set out the types of evidence we’ll focus on in each evidence category when we’re assessing a quality statement.

To assess a particular quality statement, we will take into account the evidence we have in each of the key evidence categories. This will vary depending on the type of service or organisation. For example, the evidence we will collect for GP practices will be different to what we’ll have available to us in an assessment of a home care service.

We will collect different evidence when we assess local authorities and integrated care systems.

Evidence could be information that we either:

• already have, for example from statutory notifications
• actively look for, for example from an on-site inspection.

Depending on what we find, we give a score for each evidence category that is part of the assessment of the quality statement. All evidence categories and quality statements are weighted equally.

Scores for evidence categories relate to the quality of care in a service or performance of a local authority or integrated care system:

4 = Evidence shows an exceptional standard
3 = Evidence shows a good standard
2 = Evidence shows some shortfalls
1 = Evidence shows significant shortfalls

As we are moving away from assessing at a single point in time, in future we will likely assess different areas of the framework on an ongoing basis. This means we can update scores for different evidence categories at different times.

Any changes in evidence category scores can then update the existing quality statement score.

We will follow these initial 3 stages for all assessments:

1. Review evidence within the evidence categories we’re assessing for each quality statement.
2. Apply a score to each of these evidence categories.
3. Combine these evidence category scores to give a score for the related quality statement.

After these stages, we build up scores from quality statements to an overall rating. This depends on the type of assessment.

For service providers

• The quality statement scores are combined to give a total score for the relevant key question. This score generates a rating for each key question (safe, effective, caring, responsive, and well-led).
• We then aggregate the scores for key questions to give a rating for our view of quality at an overall service level.

We will initially only publish the ratings for providers, but we intend to publish the scores in future.

For local authorities

• The quality statement scores are combined to give an overall score and a rating.

Ratings will be indicative for the pilot assessments until we move to ongoing assessment.

For integrated care systems

• The quality statement scores are combined to give a theme score.
• Theme scores are then combined to give a score for each of the 3 themes and an overall score and rating.

By using the following rules, we can make sure any areas of poor quality are not hidden.

If the key question score is within the:

• good range, but there is a score of 1 for one or more quality statement scores, the rating is limited to requires improvement
• outstanding range, but there is a score of 1 or 2 for one or more quality statement scores, the rating is limited to good.

We do not have rules or limiters for different combinations of evidence category scores. But we can apply our professional judgement if the quality statement score produced does not reflect quality for that topic.

Our judgements go through quality assurance processes.

We will need to collect evidence for all the quality statements for services that have not previously been inspected or rated before we can publish scores and ratings.