The Interpretation of Confidence Intervals

The Meaning of Confidence Intervals
Confidence Interval interpretations are a wide and chaotic mess. But they need to be interpreted well.

Confidence Intervals are an indication of IMPRECISION in the estimation of any parameter. Wider the interval, the lesser the precision. They are a measure of estimation not testing. CI gives a range of estimates that are likely to be true for the population. They are in the same units/metric as the measurement. Regardless of how wide CIs are, the estimate from the sample is the best indicator of the population value. The Confidence Intervals provide simultaneously an idea of the likely direction and magnitude of the underlying association and the random variability of the point estimate. The two-sided P-value, on the other hand, indicates only the degree of consistency between the data and a single hypothesis, and thus reveals nothing about the magnitude or even the direction of the association, or the random variability of the point estimate (Bandt and Boen, 1972).

CI holds the true error rate to the chosen level. (Schmidt, 1996).

The width of the CI is dependent on:

1. Sample Size.  Larger the Sample size, Narrower the confidence interval
2. Variability of the characteristic being studied. Less variability, the narrower the confidence interval.
3. Degree of confidence required. More the confidence required, the wider the confidence interval

Variability

Width is directly dependent on the magnitude of the Standard Error (which is calculated using the sample size and the standard deviation).

Presentation of Confidence Intervals
Altman et al. recommend that ALL the following be presented
* The risk estimate
* The Confidence interval
* The test statistic
* The degree of freedom
* The p-value


Overlapping Confidence Intervals
Thompson, 2002. Ottenbacher, 1996
Overlapping CIs around point estimates across similar studies indicate credible estimates of the same population parameter. Even though the p-values may or may not be significant, overlapping CIs show that effect estimates are in the same direction.

Points near the center of a range of infinite nested CIs (various confidence levels) are more compatible with data that points farther away from the center.

What can you do?

To see the entire set of possible confidence intervals, construct a P-value function (Birnbaum, 1961; Miettinen, 1985b; Poole, 1987a). This function, also known as a consonance function (Folks, 1981) or
confidence-interval function (Sullivan and Foster, 1990), reflects the connection between the definition of a two-sided P-value and the definition of a two-sided confidence interval (i.e., a two-sided confidence interval comprises all points for which the two-sided P-value exceeds the alpha level of the interval). The P-value function gives the two-sided P-value for the null hypothesis,
as well as every alternative to the null hypothesis for the parameter.
{Not sure how to interpret this. @confusion}

References:

1. Statistics with Confidence: Confidence Intervals and Statistical Guidelines (Book with Diskette for Windows 95, 98, NT). 2nd ed. BMJ Books; 2000. Available from: http://www.worldcat.org/isbn/0727913751.
2. Schmidt FL. Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods. 1996;1(2):115-129. Available from: http://dx.doi.org/10.1037/1082-989X.1.2.115.
3. Thompson B. What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes. Educational Researcher. 2002 Apr;31(3):25-32. Available from: http://dx.doi.org/10.3102/0013189X031003025.
4. Ottenbacher KJ. The Power of Replications and Replications of Power. The American Statistician. 1996;50(3). Available from: http://dx.doi.org/10.2307/2684673.
5. Gigerenzer G, Krauss S, Vitouch O.  The null ritual: What you always wanted to know about significance testing but were afraid to ask. In The Sage handbook of quantitative methodology for the social sciences, edited by D. Kaplan, 391–408. Thousand Oaks, CA: Sage. Available from: http://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf
6. Goodman, Steven N. 1993. p values, hypothesis tests, and likelihood: Implications for epidemiology of a neglected historical debate. American Journal of Epidemiology 137 (March): 485–96.
7. Hubbard, Raymond, and M. J. Bayarri. 2003. Confusion over measure of evidence (p’s) versus errors (α’s) in classical statistical testing (with comments). The American Statistician 57 (August): 171–82.