Statistical Process Control Implementation in Web Analytics: Key Concepts - How to determine out of control scenarios

So far we have only discussed the basics of the control chart. How do we detect special causes occurring on the chart? Below is the short guide to realistically implementing out-of-control rules in web analytics (excerpted from "Use and interpretation of statistical quality control charts" by James C. Benneyan):

1. Any point above the UCL - as you saw on the previous page. This is not usually a problem with respect to the most common web analytic variables, but you need to re-confirm the real cause.

2. Any point below the LCL - This means a real problem has occurred, like unique daily visitors falling below the LCL (lower control limit). This is worth investigating. When the root cause has been determined, corrective action will be formulated and implemented.

3. More than eight consecutive points above the mean - This usually happens when an improvement plan has been permanently implemented, resulting in substantial traffic improvement. If this cause has been re-confirmed, then a recalculation of control limits is recommended to reflect the new stable condition.

4. More than eight consecutive points below the mean - This is a sign of a problem slowly affecting the website. An example of such a situation includes an indexing problem with inner pages; this may not be immediately noticeable, especially if your website has thousands of indexable URLs. This results in pages slowly dropping in the Google index, which also lowers the web traffic.

5. Six consecutive points with either an increasing or decreasing trend - If the traffic trend were six consecutive points downtrend, then it is worth investigating to determine if something has been adjusted in the website (e.g pages accidentally blocked in robots.txt). An up trend is not always a sign of a problem, but a sign of progress (improving website traffic).

6. No variation - If there is no variation, it is also worth investigating, because your charts should show normal variations. It may mean something is wrong with your web analytics software or the accuracy of your data.

If the control charts of your web analytics data do not appear to fall under any of the six classifications above, it is said that your process or website is "stable."

See the sample screen shot below of out-of-control scenarios:

Image credits: GmcGlinn (http://commons.wikimedia.org/wiki/User:GMcGlinn )