ARTICLES

Confidence Intervals

BIS.Net Team

Confidence intervals help in making reliable inferences. Instead of providing a point estimate that sample statistics, such as the average, standard deviation and proportion do, they provide an interval in which the population parameter is likely to fall. This requires a different way of thinking.

The point estimate approach assumes the sample statistic is equal to the population parameter and hence results in wrong decisions, which can be very costly. A politician who samples voter preferences may become complacent by assuming the preference vote has no error if it is in her or his favor. Similarly, the politician can become complacent if he or she assumes there is a large error, without evidence, if the result is not in his favor.

To understand confidence intervals the difference between a statistic and parameter must first be understood. A statistic is a summary measure (average, standard deviation, proportion etc., based on the sample. A parameter is the true summary measure of the population. The population is the total entity we are interested in, so not necessarily a human population. Examples are the cans in a batch, number of houses in a suburb, or trees in a neighborhood.

Confidence intervals, by providing a margin of error, help decision makers make more informed decisions in the presence of uncertainty. A Doctor who understands that a patient’s systolic measurements can fall above 160 only 5 percent of the time will be less likely to prescribe blood pressure medication, then he would if he based his decision on a single statistic of 160 assuming every reading will be 160

A confidence interval is an interval whose span is determined by the chosen level of confidence. A 95% confidence interval has a smaller span than a 99% confidence interval. The higher the level of confidence the wider the interval. This makes sense because the wider interval will have a greater chance of covering the true population parameter.

The specified percent for the confidence interval is the percentage of times that the population parameter will fall within the confidence interval, if the sampling were repeated an infinite number of times. If a politician surveys her constituents and finds that with a 95% certainty between 51% to 54% will vote for her, based on a sample of say 100, then this does not man there is a 95% chance that the number of constituents will vote for her. On a one-off basis the only thing that can be concluded is that some voters will vote for her and some for her opponents. When one tosses a coin, the fact that 50% of the time it will be a head is irrelevant. What is relevant that it will either be a head or a tail.

Confidence intervals nevertheless are important. Clearly, a confidence interval of 51% to 55% provides more assurance than a confidence interval of .1% to 99.9%. As long as the decision maker is aware that there is always a possibility of the parameter falling outside the interval and that the percent confidence applies to repeated long-term sampling

Thus, even though the confidence level itself is meaningless for any single instance, confidence intervals provide reassurance, or caution and if used for all decision making ensures that in the long-term correct decisions are made. This can only be a good thing.

Even if used repeatedly, the actual percent confidence interval must be seen only as an approximation that could be out by as much as 5%. For example, even though a 95% confidence interval is specified the actual confidence level may be as high as 99% (conservative), or as low as 90%. There are many reasons for this. One reason is that data is not normally distributed, another is that for proportions, data is discrete. These problems are discussed under sections for specific confidence intervals in the Knowledge-Center which can be accessed from the links below.

Confidence intervals assume random sampling. Sampling is rarely truly random.

A wise analyst will thus use fuzzy thinking and assume the decision world is not precise. A wise decision make will use confidence intervals instead of point estimates. A wise decision maker will use the specified level of confidence as a guide and not make the same mistake that point estimate decision makers make, i.e. assume the numbers are real.