Please use this identifier to cite or link to this item:
Title: The probability of being identified as an outlier with commonly used funnel plot control limits for the standardised mortality ratio.
Authors: Seaton, Sarah E.
Manktelow, Bradley N.
First Published: 16-Jul-2012
Publisher: BioMed Central Ltd
Citation: BMC Medical Research Methodology, 2012, 12 : 98
Abstract: Background: Emphasis is increasingly being placed on the monitoring of clinical outcomes for health care providers. Funnel plots have become an increasingly popular graphical methodology used to identify potential outliers. It is assumed that a provider only displaying expected random variation (i.e. ‘in-control’) will fall outside a control limit with a known probability. In reality, the discrete count nature of these data, and the differing methods, can lead to true probabilities quite different from the nominal value. This paper investigates the true probability of an ‘in control’ provider falling outside control limits for the Standardised Mortality Ratio (SMR). Methods: The true probabilities of an ‘in control’ provider falling outside control limits for the SMR were calculated and compared for three commonly used limits: Wald confidence interval; ‘exact’ confidence interval; probability-based prediction interval. Results: The probability of falling above the upper limit, or below the lower limit, often varied greatly from the nominal value. This was particularly apparent when there were a small number of expected events: for expected events ≤50 the median probability of an ‘in-control’ provider falling above the upper 95% limit was 0.0301 (Wald), 0.0121 (‘exact’), 0.0201 (prediction). Conclusions: It is important to understand the properties and probability of being identified as an outlier by each of these different methods to aid the correct identification of poorly performing health care providers. The limits obtained using probability-based prediction limits have the most intuitive interpretation and their properties can be defined a priori. Funnel plot control limits for the SMR should not be based on confidence intervals.
DOI Link: 10.1186/1471-2288-12-98
eISSN: 1471-2288
Version: Publisher Version
Status: Peer-reviewed
Type: Journal Article
Rights: Copyright © 2012 Seaton and Manktelow; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Description: PMCID: PMC3441904
Appears in Collections:Published Articles, Dept. of Health Sciences

Files in This Item:
File Description SizeFormat 
10.1186_1471-2288-12-98.pdfPublished (publisher PDF)780.15 kBAdobe PDFView/Open

Items in LRA are protected by copyright, with all rights reserved, unless otherwise indicated.