Sampling
Sampling Procedure
Estimates derived from a sample survey are affected by two types of errors: 1) non-sampling errors and 2) sampling errors. Non-sampling errors are the results of mistakes made in implementing data collection and data processing, such as failure to locate and interview the correct household, misunderstanding of the questions on the part of either the interviewer or the respondent, and data entry errors. Although numerous efforts were made during the implementation of the 2008-09 Kenya Demographic and Health Survey (2008-09 KDHS) to minimize this type of error, non-sampling errors are impossible to avoid and difficult to evaluate statistically.
Sampling errors, on the other hand, can be evaluated statistically. The sample of respondents selected in the 2008-09 KDHS is only one of many samples that could have been selected from the same population, using the same design and expected size. Each of these samples would yield results that differ somewhat from the results of the actual sample selected. Sampling errors are a measure of the variability between all possible samples. Although the degree of variability is not known exactly, it can be estimated from the survey results.
Sampling error is usually measured in terms of the standard error for a particular statistic (mean, percentage, etc.), which is the square root of the variance. The standard error can be used to calculate confidence intervals within which the true value for the population can reasonably be assumed to fall. For example, for any given statistic calculated from a sample survey, the value of that statistic will fall within a range of plus or minus two times the standard error of that statistic in 95 percent of all possible samples of identical size and design.
If the sample of respondents had been selected as a simple random sample, it would have been possible to use straightforward formulas for calculating sampling errors. However, the 2008-09 KDHS sample is the result of a multi-stage stratified design, and consequently, it was necessary to use a more complex formula. The computer software used to calculate sampling errors for the 2008-09 KDHS is the sampling error module in ISSA (Integrated System for Survey Analysis). This module uses the Taylor linearization method of variance estimation for survey estimates that are means or proportions. Another approach, the Jackknife repeated replication method is used for variance estimation of more complex statistics such as fertility and mortality rates.
In addition to the standard error, the design effect (DEFT) for each estimate is also calculated. The design effect is defined as the ratio between the standard error using the given sample design and the standard error that would result if a simple random sample had been used. A DEFT value of 1.0 indicates that the sample design is as efficient as a simple random sample, while a value greater than 1.0 indicates the increase in the sampling error due to the use of a more complex and less statistically efficient design. Relative errors and confidence limits for the estimates are also computed. Sampling errors for the 2008-09 KDHS are calculated for selected variables considered to be of primary interest for the women's and men's samples. The results are presented in this appendix for the country as a whole, for urban and rural areas, and for 8 provinces. For each variable, the type of statistic (mean, proportion, or rate) and the base population are given in Table B.1. Tables B.2 to B.12 present the value of the statistic (R), its standard error (SE), the number of unweighted (N) and weighted (WN) cases, the design effect (DEFT), the relative standard error (SE/R), and the 95 percent confidence limits (R±2SE), for the selected variables including fertility and mortality rates. The sampling errors for mortality rates are presented for the whole country for the five-year period preceding the survey and by residence and province for the ten-year period preceding the survey. The DEFT is considered undefined when the standard error considering a simple random sample is zero (when the estimate is close to 0 or 1). In the case of the total fertility rate, the number of unweighted cases is not relevant, as there is no known unweighted value for woman-years of exposure to childbearing.
The confidence interval (e.g., as calculated for children ever born to women age 40-49) can be
interpreted as follows: the overall average from the national sample is 5.601 and its standard error is
0.144. Therefore, to obtain the 95 percent confidence limits, one adds and subtracts twice the standard
error to the sample estimate (i.e., 5.601 ± 2×0.144; in others words between 5.313 and 5.888). There
is a high probability (95 percent) that the true average number of children ever born to all women
aged 40 to 49 is between 5.313 and 5.888.
For the women, the relative standard errors (SE/R) for the means and proportions range between 2 percent and 15 percent, with an average relative standard error of 6.2 percent; the highest relative standard errors are for indicators with very small values (e.g., Currently using IUD at 15 percent, Currently using condom at 15 percent, and maternal mortality ratio at 16 percent). If indicators with very high values of relative standard errors (less those three indicators) were removed, then the average drops to 5.6 percent. So in general, the relative standard error for most indicators for the country as a whole is small, except for indicators of very small size. The relative standard error for the total fertility rate is small (under 4 percent). However, for the childhood mortality rates, the average relative standard error at the national level is much higher, about 11 percent.
There are differentials in the relative standard error for indicators by sub-populations. For example, for the variable Unmet need for family planning, the relative standard errors as a percent of the estimated mean for the whole country, urban areas and rural areas are 3.2 percent, 5.6 percent, and 3.7 percent, respectively.
For the total women sample, the value of the design effect (DEFT) averaged over all variables is 1.83, which means that due to multi-stage clustering of the sample the average standard error is increased by a factor of 1.83 over that in an equivalent simple random sample.
Weighting
Estimates derived from a sample survey are affected by two types of errors: 1) non-sampling errors and 2) sampling errors. Non-sampling errors are the results of mistakes made in implementing data collection and data processing, such as failure to locate and interview the correct household, misunderstanding of the questions on the part of either the interviewer or the respondent, and data entry errors. Although numerous efforts were made during the implementation of the 2008-09 Kenya Demographic and Health Survey (2008-09 KDHS) to minimize this type of error, non-sampling errors are impossible to avoid and difficult to evaluate statistically.
Sampling errors, on the other hand, can be evaluated statistically. The sample of respondents selected in the 2008-09 KDHS is only one of many samples that could have been selected from the same population, using the same design and expected size. Each of these samples would yield results that differ somewhat from the results of the actual sample selected. Sampling errors are a measure of the variability between all possible samples. Although the degree of variability is not known exactly, it can be estimated from the survey results.
Sampling error is usually measured in terms of the standard error for a particular statistic (mean, percentage, etc.), which is the square root of the variance. The standard error can be used to calculate confidence intervals within which the true value for the population can reasonably be assumed to fall. For example, for any given statistic calculated from a sample survey, the value of that statistic will fall within a range of plus or minus two times the standard error of that statistic in 95 percent of all possible samples of identical size and design.
If the sample of respondents had been selected as a simple random sample, it would have been possible to use straightforward formulas for calculating sampling errors. However, the 2008-09 KDHS sample is the result of a multi-stage stratified design, and consequently, it was necessary to use a more complex formula. The computer software used to calculate sampling errors for the 2008-09 KDHS is the sampling error module in ISSA (Integrated System for Survey Analysis). This module uses the Taylor linearization method of variance estimation for survey estimates that are means or proportions. Another approach, the Jackknife repeated replication method is used for variance estimation of more complex statistics such as fertility and mortality rates.
In addition to the standard error, the design effect (DEFT) for each estimate is also calculated. The design effect is defined as the ratio between the standard error using the given sample design and the standard error that would result if a simple random sample had been used. A DEFT value of 1.0 indicates that the sample design is as efficient as a simple random sample, while a value greater than 1.0 indicates the increase in the sampling error due to the use of a more complex and less statistically efficient design. Relative errors and confidence limits for the estimates are also computed. Sampling errors for the 2008-09 KDHS are calculated for selected variables considered to be of primary interest for the women's and men's samples. The results are presented in this appendix for the country as a whole, for urban and rural areas, and for 8 provinces. For each variable, the type of statistic (mean, proportion, or rate) and the base population are given in Table B.1. Tables B.2 to B.12 present the value of the statistic (R), its standard error (SE), the number of unweighted (N) and weighted (WN) cases, the design effect (DEFT), the relative standard error (SE/R), and the 95 percent confidence limits (R±2SE), for the selected variables including fertility and mortality rates. The sampling errors for mortality rates are presented for the whole country for the five-year period preceding the survey and by residence and province for the ten-year period preceding the survey. The DEFT is considered undefined when the standard error considering a simple random sample is zero (when the estimate is close to 0 or 1). In the case of the total fertility rate, the number of unweighted cases is not relevant, as there is no known unweighted value for woman-years of exposure to childbearing.
The confidence interval (e.g., as calculated for children ever born to women age 40-49) can be
interpreted as follows: the overall average from the national sample is 5.601 and its standard error is
0.144. Therefore, to obtain the 95 percent confidence limits, one adds and subtracts twice the standard
error to the sample estimate (i.e., 5.601 ± 2×0.144; in others words between 5.313 and 5.888). There
is a high probability (95 percent) that the true average number of children ever born to all women
aged 40 to 49 is between 5.313 and 5.888.
For the women, the relative standard errors (SE/R) for the means and proportions range between 2 percent and 15 percent, with an average relative standard error of 6.2 percent; the highest relative standard errors are for indicators with very small values (e.g., Currently using IUD at 15 percent, Currently using condom at 15 percent, and maternal mortality ratio at 16 percent). If indicators with very high values of relative standard errors (less those three indicators) were removed, then the average drops to 5.6 percent. So in general, the relative standard error for most indicators for the country as a whole is small, except for indicators of very small size. The relative standard error for the total fertility rate is small (under 4 percent). However, for the childhood mortality rates, the average relative standard error at the national level is much higher, about 11 percent.
There are differentials in the relative standard error for indicators by sub-populations. For example, for the variable Unmet need for family planning, the relative standard errors as a percent of the estimated mean for the whole country, urban areas and rural areas are 3.2 percent, 5.6 percent, and 3.7 percent, respectively.
For the total women sample, the value of the design effect (DEFT) averaged over all variables is 1.83, which means that due to multi-stage clustering of the sample the average standard error is increased by a factor of 1.83 over that in an equivalent simple random sample.