How we determine ratings for local authorities

A local authority’s rating can change only when we carry out a comprehensive assessment.

To support the consistency and transparency of our judgements, we use a scoring framework for our assessments of local authorities.

For each quality statement in the assessment framework, we review evidence across a range of categories and apply a quality statement score in line with the scoring framework, as well as using professional judgement to determine this score.

The scoring framework to support decisions is:

1 = Evidence shows significant shortfalls

2 = Evidence shows some shortfalls

3 = Evidence shows a good standard

4 = Evidence shows an exceptional standard 

How we calculate quality statement scores

We collect evidence across a range of categories for each quality statement. However, we no longer award scores at evidence category level.

Scoring at quality statement level helps us to take account of all evidence across the whole quality statement area and to make a proportionate judgement. This also enables us to emphasise people’s outcomes and experiences in our judgements, including those of unpaid carers. 

How we calculate the overall rating

We combine the scores for all the quality statements assessed to give an overall score as a percentage. All quality statements are weighted equally. The percentage score is then used to determine an overall rating for the local authority’s adult social care provision using these thresholds:

• 25 to 38% = inadequate
• 39 to 62% = requires improvement
• 63 to 87% = good
• 88% and above = outstanding

The overall percentage score indicates a more detailed position within the rating scale. This will help us to see if performance is moving up or down within a rating.

For example, for a rating of good, the score will tell us if this is either:

• in the upper threshold, nearing outstanding
• in the lower threshold, nearer to requires improvement

Similarly, for a rating of requires improvement, the score would tell us if it was either:

• in the upper threshold, nearing good
• in the lower threshold, nearer to inadequate

Example ratings

24/36 (maximum score) = 66%

This is in the lower threshold of a rating of good. If one or more quality statements have a score of 1, there cannot be an overall rating of either good or outstanding. 

How we ensure consistent judgements

We use rating characteristics and rating principles to support professional judgement when making decisions about scoring. Our judgements and ratings need to be proportionate to all available evidence and the specific facts and circumstances.

Our rating principles will normally apply and will be balanced by assessment teams using their professional judgement. The following principles apply when we determine an overall rating:

• The quality statements are equally important and should be weighted equally.
• If one or more quality statements have a score of 2, there cannot usually be an overall rating of outstanding.
• If one or more quality statements have a score of 1, there cannot usually be an overall rating of either good or outstanding.

There may be circumstances where we need to make a decision that is not in line with our ratings principles. This is particularly the case where we have identified concerns as part of an assessment. In making this decision, we will consider the following criteria:

• The extent and impact of the concerns on people who use services and the risk to quality and safety. For example, if an assessment identified concerns related to safeguarding that have a potentially very significant impact on people, it may increase the impact of this area on the overall rating.
• Our confidence in the local authority to address the concerns, or where it has already taken action.

Where a rating decision is not consistent with the principles, we will record the rationale clearly in the assessment report and the decision will be in line with our quality assurance processes.

How we use data to make our judgements

We use data and metrics throughout the assessment process to help us identify areas for further investigation, as well as groups or people to ask for their feedback. Data that we gather during an assessment is evidence in itself and supports our conversations with local authorities.

We have developed a standard set of metrics derived from national data collections that we consider as evidence under some quality statements. The indicators have been thoroughly reviewed and are appropriate for this purpose.

You can see a detailed list of metrics derived from national data collection, which we consider as evidence in assessments, in our metrics for local authority assessment framework. We may periodically make changes to the data and metrics we consider as the data landscape continues to evolve. We will update our information when new metrics are added, or when we make changes to existing metrics.

However, data is only one part of the evidence we use to form our judgements, as we consider all the other information gathered during the assessment. We will assess data and metrics alongside other types of evidence to reach informed judgements, and standard metrics may be triangulated and supplemented with locally-held data and information.

Our assessment reports only include the evidence that is needed to provide the context and to underpin our judgements. They do not include every item of data or evidence.

Our assessment team will answer specific queries during the assessment.

Quality assuring our judgements and reports

Each draft assessment report goes through a rigorous quality assurance process. This includes a review by a calibration panel. This panel reviews and validates the scores and ratings from the assessments to make sure they provide a reliable and consistent view of quality across local authorities.

The panel will include representatives from external organisations as well as colleagues in CQC.

Each calibration panel typically looks at 3 to 4 reports, which are moderated against all previous reports. We have a single detailed record of the individual scores for every local authority, which is available to the standing members of the panel to use as a reference tool.