Testing change ideas

This page provides guidance on how to test change ideas

Testing changes

Testing changes helps you build your knowledge about what works in your system, and why. The most important thing about testing is that you have a theory. Each time you do a test you need to make a prediction about what will happen. If the prediction is wrong, then you need to adjust your theory.  Simply trying changes without a prediction is not testing and does not help build knowledge.

As you build your knowledge about a change you will want to test it under different conditions to see if your theory still holds with different people, shifts, or circumstances.  As you build your testing you will get more confident in your predictions, develop the change within your context and increase your degree of belief in the change.

The Plan-Do-Study-Act (PDSA) cycle is shorthand for testing a change — by planning it, trying it, observing the results, and acting on what is learned. This is the scientific method, used for action-oriented learning.

Using the PDSA cycle involves testing new change ideas on a small scale and building knowledge iteratively. A test does not equate to a change idea. You will likely need several PDSA tests about each change idea to really understand whether and why it works.

It is important to record your tests so that the learning is not lost. A template is available within the PDSA tool.

Tips for testing changes
  • Stay a cycle ahead.
    When designing a test, imagine at the start what the subsequent test or two might be, given various possible findings in the "Study" phase of the Plan-Do-Study-Act cycle.
  • Scale down the scope of tests.
    Dimensions of the tests that can be scaled down include the number of service users, practitioners, and others involved in the test ("Sample the next 10" instead of "Get a sample of 200"), and the location or duration of the test ("Test it in xx team for one week").
  • Pick willing volunteers. Work with those who want to work with you.
    ("I know Mr. Jones will help us" instead of "How can we convince Ms. Smith to buy in?")
  • Avoid the need for consensus, buy-in, or political solutions.
    Save these for later stages. When possible, choose changes that do not require a long process of approval, especially during the early testing phase.
  • Don’t reinvent the wheel.
    Instead, replicate changes made elsewhere. For example, instead of creating your own atrial fibrillation treatment protocol, try modifying another hospital’s protocol.
  • Pick easy changes to try.
    Look for the concepts that seem most feasible and will have the greatest impact.
  • Avoid technical slowdowns.
    Don’t wait for the new computer to arrive; try recording test measurements and charting trends with paper and pencil instead.
  • Reflect on the results of every change.
    After making a change, a team should ask: What did we expect to happen? What did happen? Were there unintended consequences? What was the best thing about this change? The worst? What might we do next? Too often, people avoid reflecting on failure. Remember that teams often learn very important lessons from failed tests of change.
  • Be prepared to end the test of a change.
    If the test shows that a change is not leading to improvement, the test should be stopped. Note: "Failed" tests of change are a natural part of the improvement process. If a team experiences very few failed tests of change, it is probably not pushing the boundaries of innovation very far.