Criterion Based Audit

Add to favourites

Criterion based audit involves selecting aspects of health care and systematically evaluating them against explicit criteria and agreed standards. Where indicated, changes are implemented at an individual, team or practice level to meet those standards. Further monitoring is used to confirm improvement in healthcare delivery.

Criterion Based Audit

What is criterion based audit?

This is what you will know as a standard audit. It involves selecting aspects of health care provided by the practice and systematically evaluating them against explicit criteria and agreed standards. Where indicated, changes are implemented at an individual, team or practice level to meet those standards. Further monitoring is used to confirm improvement in healthcare delivery.

Why would you choose this?

This method is helpful to evaluate ‘real’ practice against ‘best’ practice and improve quality in specific areas e.g. are we prescribing Direct Oral Anticoagulant (DOAC) as per best practice for patients with Atrial Fibrillation (AF)

What is it useful for?

• In a defined area with a small number of measurable criteria with scope for change e.g. looking at DOAC prescriptions for AF.

• To benchmark performance against other practices when data is available. For example, comparing performance within GP clusters.

What is it not useful for?

• In a large area requiring measurement of a large number of criteria e.g. looking at quality of Diabetes care.

• When you are unsure about what to change and may have to test different ideas

Who can do/lead it?

Any clinician in the practice team who understands the method and is confident in applying it. A whole team approach will improve outcome

How do you do it? (eight-stage cyclical process)

1. Reason for choice of audit: The topic chosen, and reasons for choosing, should be clearly defined, ideally with evidence to justify the potential change. This requires communication to the wider team to ensure commitment from all interested parties.

2. Criterion or criteria chosen: Clear criteria (simple, logical statements that describe specific and measurable health care items or activities and can be used to assess its quality) need to be chosen at the onset.

3. Standards set: Recent research and discussion with wider team should be used to develop standards (these quantify level of care to be achieved for criteria).

4. Preparation and planning: Taking time to discuss the project with the whole team.

5. Data collection (1): The criteria are used to collect initial data and should be summarised against the defined standard.

6. Change(s) to be evaluated: The team to evaluate how their performance differs from their standard, and to discuss and decide on plans for change. These are then implemented for an agreed period of time before a second data collection.

7. Data collection (2): These results should be summarised alongside the initial collection and presented to the team showing what change has been achieved.

8. Conclusion: The team can then decide what further change is required and how this change can be sustained. Multiple audit cycles may be required to create sustainable change and ‘normalise’ new practice

Who else does it involve?

All relevant members of the practice team involved in the specific criteria selected, whole team involvement important to ensure change occurs

Top tips

• All normal Top Tips (listed on page four) for QI apply here.

• Define specific criteria and avoid negative criteria.

• Derive standards from professional consensus within the practice.

• Formulate action plans that specifically address relevant problems.

• ‘Close the loop’ by collecting data again after a defined period of time

Further Info

The report template can be downloaded here.

Criterion Based Audit Example

Introduction

The use of direct oral anticoagulation medication (DOACS) is increasing. One of our patients had not had bloods checked in two years. Guidance on monitoring has been issued by NICE.

Criteria

These are based on NICE CKS guidance. (https://cks.nice.org.uk/anticoagulation-oral)

1. Patients with normal renal function on rivaroxiban, dagibatran and apixiban DOACs should have had U&E and Full Blood Count (FBC) within the last 12 months.

2. Patients on DOACs with Estimated Glomerular Filtration Rate (eGFR) 30-60 should have U&E checked in the last six months

Standards

A standard of 90% was agreed by the clinical team as some patients may cancel appointments and not have bloods taken within the time frame

Preparation and planning

The project was discussed by GPs, nurses and the administration team. No monitoring system was in place. One of the administrative team searched for all patients on DOACs and the date of last blood test and if the eGFR was recorded

Data collection 1  Jan 2016

 Data collection 1 n (%)Standard                      

Patients on DOACs with normal renal function

 

12 

On DOAC, normal renal function and U&E and FBC in last 12 months

 

5 (42%)90%

On DOAC and eGFR 30-60

 

24 

On DOAC eGFR 30-60 and U&E and FBC in last six months

 

16 (67%)90%

 

Changes implemented

After discussion of results between GPs, nursing staff and admin staff, those missing bloods were contacted and testing arranged. A new recall system was developed. A diary entry was created for all patients on DOACs. A monthly search would be carried out to identify new patients started on this medication – those who were on the drug without a recall code.

Data collection 2 Jan 2017

 Data collection 1 n (%)Data collection 2 n (%)Standard

Patients on DOACs with normal renal function

 

1218 

On DOAC, normal renal function and U&E and FBC in last 12 months

 

5 (42%)17 (94%)90%

On DOAC and eGFR 30-60

 

2430 

On DOAC eGFR 30-60 and U&E and FBC in last six months

 

16 (67%)28 (93%)90%
Total number of patients on rivaroxiban, dagibatran and apixiban = 48

We have improved from data collection one and reached our standard.

Conclusions

Although we have reached our standard there were still three patients who had not had bloods taken as per protocol. One of these had recurrent hospital admissions and actually had all the bloods taken in hospital, the other two patients had not attended for appointments. We have discussed how to arrange testing for them and they have now been contacted and agreed to attend. To maintain this change, we are instituting six monthly audit that the PM will carry out: all patients who do not have appropriate monitoring will be brought to the attention of a GP. We have worked as a team to evaluate our performance and design and implement a sustainable change to improve the care for this group of patients

Discussion on example

Some audits like this example require two data collections and can be completed relatively quickly. Others may require several small data collections (e.g. patients admitted to a nursing home should have a key information summary and anticipatory care plan completed within six weeks). The important thing is to demonstrate sustainable improvement, not how quickly the practice reaches its target. There may be several changes the practice needs to institute over consecutive cycles. It is worth taking the time required to design the audit properly: cycles that are too small may not demonstrate measurable improvement, and ones that are too long may reduce momentum and motivation in staff. As mentioned in the Top Tips section it is also important to sustain change once it has occurred. In the above example although the practice has demonstrated change, any improvement would be limited if the change was not sustained. They have demonstrated a plan to monitor this

Criterion Based Audit resources