Tuesday, March 15, 2011

Are We Captives of Control Charts

Joan asked me post this on her behalf because she is traveling this week.  
Once upon a time histograms, bar charts, and line graphs were the most important tools for displaying infection control data.  We used them to show differences between events and to track changes over time.

Now someone has decided that control charts are essential tools for measuring success in healthcare.  We spend a lot of time constructing charts that show upper and lower confidence limits.  We faithfully plot our own results on them and hope that we will not meet the criteria for being “out of control”. 

We have rules for what it means to be out of control:  one spike above two standard deviations, a series of X in a row above or below the central line, or a series of ups and downs.  How many of us have been called to task because a dreaded spike has appeared in our data.

Would it surprise you to know that control charts are business tools?  They were not introduced into health care by epidemiologists.  They were adopted by people in healthcare who had business backgrounds.  We have adopted them because they seem like better tools than our old stand-bys.

What are we really comparing ourselves to when we use control charts?  The goal of control charts is to keep a process average.  The center line represents average performance and, in business, the goal is to keep it there.

In healthcare we are always striving to be better.  In most cases this means that we want the lines to move steadily up or down.  We want to have more of what is good and less of what is bad.  So why are we caught up in control charts?

This month I am linking you to a business consulting website that explains control charts.  Being a business site, the explanations are about applying control charts to the production of widgets.  Notably absent are recommendations to use control charts to measure complex healthcare processes that involve not just machinery, but the unpredictability of people (both patients and workers).

I think it is time that we move away from the model of infection prevention as a business.  A big step in doing so is to use statistics that correctly show what we really are all about.

Please let me know if there are any questions I can answer about these articles.

Individual control charts:  different types of control charts, what they tell us and when to use them.   Available here.

An example of misuse of statistical process control in healthcare.   

An example of overcontrolling a process:  Over-reacting to points on a control chart that are above or below a confidence level. 

5 comments:

  1. I think that the work you are doing to reduce healthcare-acquired infections in clinical settings is to be admired and is very worthwhile. I wish you many successes in your efforts.

    However, this blog was not written by someone who understands control charts, how they are used, or the information that is contained in the www.spcforexcel.com website. Your blog is very misleading.

    Your blog provides links to three of the newsletters on our website. I don't mind people linking our newsletters. In fact, I think it is great. We have a wealth of free knowledge there - not just on control charts but on many other statistical techniques. If you hadn't brought in the website, I would probably not have responded to your blog. I wrote this month's newsletter on the purpose of control charts. I would ask that you read it. You can see my full response at a link at the bottom of the newsletter.

    I end the reponse with this: If you are interested in exploring how control charts can be used in your work, please let me know. I will be happy to work with you (no charge) plus supply our SPC for Excel software for your use (again no charge). Just contact me.

    Let me know, but please take the time to read my March 2011 newsletter on the website.

    Best Regards,

    Bill

    ReplyDelete
  2. I have taught statistical techniques for nearly 30 years with clients from some of the world's leading manufacturing companies.
    Your article indicates that you have misunderstood the fundamentals of control charts.
    Infection prevention is not a business it is a process, and the aim of a control chart is to see how a process behaves, it shows you what is happening. A control chart can be used in any sphere of life where we have a process and some measurements. We use the chart to find out how the process is behaving. We check to see if the process is stable or unstable and then take the appropriate action(s). May I suggest that you read a good book on SPC, my favourite is Understanding Variation by Donald J Wheeler, it is not a long book, and is really quite readable.
    Hope this helps
    Kevin Gardner (England)

    ReplyDelete
  3. I also recommend Understanding Variation by Don Wheeler. It's such an easy read. My work is performance measurement, and I find that as long as you collect data regularly enough, control charts provide the ability for management to only respond when there is a true signal in the performance measure, as opposed to reacting to every month-to-month difference (most of which are just normal variation). Eileen, please don't give up on control charts!

    ReplyDelete
  4. Thank you for your response. I am glad that my posts are being read and provoking feedback.(Incidentally, my co-worker Eileen posted the original blog for me while I was out of town. The contents of the blog are totally my thoughts and opinions.)

    In the spirit of debate, I have some comments in response.

    I agree that statistical process control techniques can be used in a variety of settings. However, my issue is whether these are the best measures to apply to infection control situations. And I have several reasons to believe they are not.

    First, health care professionals are not manufacturing things. We are fixing things so to speak. A crude comparison might be with insurance estimators who must make a decision to repair a vehicle or declare it beyond repair. In health care we do not declare anyone beyond repair. In the statistical process control method, these situations are included in process measurement and affect "control". In other words, the repair person is being penalized for trying to fix something to standards.

    Second, in manufacturing one theoretically can control all phases of the process. Raw materials, machines, human activities and output can all be measured and standardized. When situations are deemed out of control, there are a finite number of components to analyze and correct. In health care, however, the number of components is almost infinite including (1) a variety of providers who interact with each other differently with each patient, (2) a variety of providers who interact differently with each patient, and (3) a variety of services to be provided, each with its own complex interactions,

    Third, in industry what is being manufactured has no role in the outcome of the process. The product is a completely a result of outside activities. In health care, patients have a major role in the outcomes. Biological factors that are unknown or currently uncontrollable by health care professionals alter the outcomes of health care services. In addition, the right of every person to manage a lifestyle that may negatively affect outcomes plays a major role in the final "product" of the health care process.

    Finally, the goal of statistical process controls is to maintain the process within specified parameters. Since the goal of health care is constant improvement in outcomes, staying within confidence intervals is not the appropriate paradigm. In fact, the rules that designate a process out of control may in fact, be illustrating improvement either by consistently going above or below a mean or outside a confidence interval.

    SPC has played an important role in health care in that it has focused providers on the need to establish standards of care. I think it is time that we acknowledge the important contribution that SPC has made to improving quality in health care. However, I think that it is also time to move to a quality measurement process that takes into account the many factors that affect our outcomes.

    Joan Mallick, Ph.D., R.N.

    ReplyDelete
  5. Hello Joan,

    I doubt if you will post this. My name is Bill McNeese and it is our website (www.spcforexcel.com) that Eileen decided to reference. After reading the blog, it was clear to me (and others it appears), that there was a misunderstanding about what control charts are used for. That led me to write my last newsletter on the purpose of control charts (which appears to have sent some of my readers here). I recommend you read it because it addresses the issues you raise above. Plus, there is a link to more complete response to this blog - including a link to a presentation given by a doctor (and Harvard professor) who has a well-thought out process for continuous improvement in infection reduction - and it includes the use of control charts to see improvements and maintain them). The goal of SPC is not to maintain the process within specified parameter. It is continuous improvement over time.

    In the response on the website, I offer to work with you (no charge) to see what type of data you are looking at and if control charts can be used. I also included an offer of a free copy of our software (SPC for Excel). This includes control charts but many other statistical tools. This offer still stands.

    You wish to move beyond control charts, but you don't give any examples of what you would do to measure performance. If control charts don't work for you, don't use them. But it is rare that plotting data over time doesn't help you understand the process.

    Best Regards,

    Bill

    ReplyDelete