How we reach a rating

How we reach a rating

To support the transparency and consistency of our judgements, we have introduced a scoring framework into our assessments.

Where appropriate, we’ll continue to describe the quality of care using our 4 ratings: outstanding, good, requires improvement, or inadequate.

When we assess evidence, we assign scores to the key evidence categories for each quality statement that we’re assessing. Ratings will be based on building up scores from quality statements to an overall rating.

This approach makes clear the type of evidence that we have used to reach decisions.

Scoring

Using scoring as part of our assessments will help us be clearer and more consistent about how we’ve reached a judgement on:

  • the quality of care in a service
  • how well a local authority is delivering its duties under the Care Act
  • the performance of an integrated care system

For example, for a rating of good, the score will tell us if this is either:

  • in the upper threshold, nearing outstanding
  • in the lower threshold, nearer to requires improvement.

Similarly, for a rating of requires improvement, the score would tell us if it was either:

  • in the upper threshold, nearing good
  • in the lower threshold, nearer to inadequate.

Our quality statements clearly describe the standards of care that people should expect.

To make things clearer and more consistent, we’ll set out the types of evidence we’ll focus on in each evidence category when we’re assessing a quality statement.

To assess a particular quality statement, we will take into account the evidence we have in each of the key evidence categories. This will vary depending on the type of service or organisation. For example, the evidence we will collect for GP practices will be different to what we’ll have available to us in an assessment of a home care service.

We will collect different evidence when we assess local authorities and integrated care systems.

Evidence could be information that we either:

  • already have, for example from statutory notifications
  • actively look for, for example from an on-site inspection.

Depending on what we find, we give a score for each evidence category that is part of the assessment of the quality statement. All evidence categories and quality statements are weighted equally.

Scores for evidence categories relate to the quality of care in a service or performance of a local authority or integrated care system:

4 = Evidence shows an exceptional standard
3 = Evidence shows a good standard
2 = Evidence shows some shortfalls
1 = Evidence shows significant shortfalls

As we are moving away from assessing at a single point in time, in future we will likely assess different areas of the framework on an ongoing basis. This means we can update scores for different evidence categories at different times.

Any changes in evidence category scores can then update the existing quality statement score.

We will follow these initial 3 stages for all assessments:

  1. Review evidence within the evidence categories we’re assessing for each quality statement.
  2. Apply a score to each of these evidence categories.
  3. Combine these evidence category scores to give a score for the related quality statement.

After these stages, we build up scores from quality statements to an overall rating. This depends on the type of assessment.

For service providers

  • The quality statement scores are combined to give a total score for the relevant key question. This score generates a rating for each key question (safe, effective, caring, responsive, and well-led).
  • We then aggregate the scores for key questions to give a rating for our view of quality at an overall service level.

By using the following rules, we can make sure any areas of poor quality are not hidden:

  • If the key question score is within the good range, but one or more of the quality statement scores is 1, the rating is limited to requires improvement
  • If the key question score is within the outstanding range, but one or more of the quality statement scores is 1 or 2, the rating is limited to good.

We will initially only publish the ratings for providers, but we intend to publish the scores in future.

For local authorities

  • The quality statement scores are combined to give an overall score and a rating.

By using the following rules, we can make sure any areas of poor quality are not hidden.

  • If one or more quality statements have a score of 2, there cannot be an overall rating of outstanding.
  • If one or more quality statements have a score of 1, there cannot be an overall rating of either good or outstanding.

For integrated care systems

  • The quality statement scores are combined to give a score for each of the 3 theses (leadership, integration and quality and safety).
  • Theme scores are then combined to give an overall score for the integrated care system.

We do not have rules or limiters for different combinations of evidence category scores. But we can apply our professional judgement if the quality statement score produced does not reflect quality for that topic.

Our judgements go through quality assurance processes.

For services that have not previously been inspected or rated, we will need to assess all quality statements in a key question before we publish the rating. For newly registered services, we’ll usually assess all quality statements within 12 months.