Health Policy and Planning Advance Access published November 26, 2013 Published by Oxford University Press in association with The London School of Hygiene and Tropical Medicine ß The Author 2013; all rights reserved.
Health Policy and Planning 2013;1–7 doi:10.1093/heapol/czt088
Improving the collection of knowledge, attitude and practice data with community surveys: a comparison of two second-stage sampling methods Rosemary H. Davis and Joseph J. Valadez*
*Corresponding author. Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK. E-mail: [email protected]
Accepted
17 October 2013
Background Second-stage sampling techniques, including spatial segmentation, are widely used in community health surveys when reliable household sampling frames are not available. In India, an unresearched technique for household selection is used in eight states, which samples the house with the last marriage or birth as the starting point. Users question whether this last-birth or last-marriage (LBLM) approach introduces bias affecting survey results. Methods
We conducted two simultaneous population-based surveys. One used segmentation sampling; the other used LBLM. LBLM sampling required modification before assessment was possible and a more systematic approach was tested using last birth only. We compared coverage proportions produced by the two independent samples for six malaria indicators and demographic variables (education, wealth and caste). We then measured the level of agreement between the caste of the selected participant and the caste of the health worker making the selection.
Results
No significant difference between methods was found for the point estimates of six malaria indicators, education, caste or wealth of the survey participants (range of P: 0.06 to >0.99). A poor level of agreement occurred between the caste of the health worker used in household selection and the caste of the final participant, (K ¼ 0.185), revealing little association between the two, and thereby indicating that caste was not a source of bias.
Conclusions Although LBLM was not testable, a systematic last-birth approach was tested. If documented concerns of last-birth sampling are addressed, this new method could offer an acceptable alternative to segmentation in India. However, interstate caste variation could affect this result. Therefore, additional assessment of last birth is required before wider implementation is recommended. Keywords
LQAS, lot quality assurance sampling, second-stage sampling, surveys, reliability testing, community health, India, Indian health systems, sampling methodology, segmentation sampling, caste, KAP
1
Downloaded from http://heapol.oxfordjournals.org/ at University of California, Santa Cruz on November 9, 2014
Department of International Health, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK
2
HEALTH POLICY AND PLANNING
KEY MESSAGES
An innovative second-stage sampling alternative must be subjected to rigorous assessment before accepting it as a reliable alternative to established approaches.
In the context of Odisha India, using the location of the last birth in the village as a starting point for sampling was an acceptable method. However, it needed very systematic rules in order to be reliably used.
The last-birth method should be tested in other locations of India and elsewhere to assess its reliability.
Introduction
Methods Segmentation technique Segmentation sampling was applied using a standard approach (Valadez et al. 2007): (1) identify a community leader with good knowledge of the village, (2) with their help draw a rough map of the village and subdivide the village into two or more equal segments using estimates of the number of households residing
Downloaded from http://heapol.oxfordjournals.org/ at University of California, Santa Cruz on November 9, 2014
Due to increased international funding for health systems development and the corresponding need for accountability, there is an increased demand from public health managers for effective monitoring and evaluation of progress towards international targets, such as the Millennium Development Goals (Boerma and Stansfield 2007; Murray and Frenk 2008; Chan et al. 2010). Increasingly, household health surveys are used in developing countries for programme monitoring and evaluation as a main source of information on health status and service coverage (Boerma and Stansfield 2007). Although many surveys are financed by large bilateral donors and international organizations such as the Demographic and Health Survey (DHS) and the Multi-Indicator Country Survey, other surveys with a programmatic or specialized focus are financed by other sources such as state and national governments, non-governmental organizations and foundations. As the budgets available for surveys can vary, there is increasing dialogue in the international literature about alternative sampling approaches and their relative merits for both data acquisition as well as their cost (Singh et al. 1996). These debates concern both primary and second-stage sampling. Credible surveys require a representative sample using a reliable randomizing procedure. Although simple random sampling (SRS) is a gold standard for participant selection, SRS is often impractical in the absence of accurate sampling frames (Bostoen and Chalabi 2006). An early solution to second-stage sampling was the ‘random walk’ technique developed by the World Health Organization for the cluster survey used by the Expanded Programme on Immunisation (EPI) (Henderson and Sundaresan 1982). Despite its wide application, the methodological rigor of this technique has been questioned, as the household selection may be opened to bias (Milligan et al. 2004) with households at the centre of villages or clusters being favoured for selection (Kok 1986; Oliphant et al. 2006). Segmentation sampling has been found to be a more rigorous second-stage sampling technique (Turner et al. 1996) and is now advocated in several survey guidelines (Magnani 1997; World Health Organization 2005; SMART 2006). In our experience, about 1 hr is required for a trained person to develop a credible but crude map of a village working with a knowledgeable informant, a process which is required in segmentation sampling. This temporal cost, while acceptable, motivates practitioners to search for quicker approaches. In addition to accepted second-stage sampling methodologies, new techniques are continually proposed, but they require testing. In India, a novel technique of household selection is being used in eight states (in addition to segmentation) to support assessment of the National Vector Borne Disease
Control Program (NVBDCP). This method uses the local knowledge of village community health workers to identify the household with the last birth or the last marriage (LBLM). The 10th household to the right of this household is then selected for interview. The method assumes that the health worker, who provides services to women and children, knows the location of the household with the last birth (LB) or last marriage (LM) and does not rely on the use of a birth register. The LBLM sampling protocol lacks sufficient structure to be used in a consistent manner as the data collector is given the opportunity to select whether to use either the LB or LM, which could result in an opportunistic sample if the choice is based on the proximity to the health worker. In addition, the instruction to ‘move to the 10th house on the right’ is unclear where there are no, or
Health Policy and Planning 2013;1–7 doi:10.1093/heapol/czt088
Improving the collection of knowledge, attitude and practice data with community surveys: a comparison of two second-stage sampling methods Rosemary H. Davis and Joseph J. Valadez*
*Corresponding author. Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK. E-mail: [email protected]
Accepted
17 October 2013
Background Second-stage sampling techniques, including spatial segmentation, are widely used in community health surveys when reliable household sampling frames are not available. In India, an unresearched technique for household selection is used in eight states, which samples the house with the last marriage or birth as the starting point. Users question whether this last-birth or last-marriage (LBLM) approach introduces bias affecting survey results. Methods
We conducted two simultaneous population-based surveys. One used segmentation sampling; the other used LBLM. LBLM sampling required modification before assessment was possible and a more systematic approach was tested using last birth only. We compared coverage proportions produced by the two independent samples for six malaria indicators and demographic variables (education, wealth and caste). We then measured the level of agreement between the caste of the selected participant and the caste of the health worker making the selection.
Results
No significant difference between methods was found for the point estimates of six malaria indicators, education, caste or wealth of the survey participants (range of P: 0.06 to >0.99). A poor level of agreement occurred between the caste of the health worker used in household selection and the caste of the final participant, (K ¼ 0.185), revealing little association between the two, and thereby indicating that caste was not a source of bias.
Conclusions Although LBLM was not testable, a systematic last-birth approach was tested. If documented concerns of last-birth sampling are addressed, this new method could offer an acceptable alternative to segmentation in India. However, interstate caste variation could affect this result. Therefore, additional assessment of last birth is required before wider implementation is recommended. Keywords
LQAS, lot quality assurance sampling, second-stage sampling, surveys, reliability testing, community health, India, Indian health systems, sampling methodology, segmentation sampling, caste, KAP
1
Downloaded from http://heapol.oxfordjournals.org/ at University of California, Santa Cruz on November 9, 2014
Department of International Health, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK
2
HEALTH POLICY AND PLANNING
KEY MESSAGES
An innovative second-stage sampling alternative must be subjected to rigorous assessment before accepting it as a reliable alternative to established approaches.
In the context of Odisha India, using the location of the last birth in the village as a starting point for sampling was an acceptable method. However, it needed very systematic rules in order to be reliably used.
The last-birth method should be tested in other locations of India and elsewhere to assess its reliability.
Introduction
Methods Segmentation technique Segmentation sampling was applied using a standard approach (Valadez et al. 2007): (1) identify a community leader with good knowledge of the village, (2) with their help draw a rough map of the village and subdivide the village into two or more equal segments using estimates of the number of households residing
Downloaded from http://heapol.oxfordjournals.org/ at University of California, Santa Cruz on November 9, 2014
Due to increased international funding for health systems development and the corresponding need for accountability, there is an increased demand from public health managers for effective monitoring and evaluation of progress towards international targets, such as the Millennium Development Goals (Boerma and Stansfield 2007; Murray and Frenk 2008; Chan et al. 2010). Increasingly, household health surveys are used in developing countries for programme monitoring and evaluation as a main source of information on health status and service coverage (Boerma and Stansfield 2007). Although many surveys are financed by large bilateral donors and international organizations such as the Demographic and Health Survey (DHS) and the Multi-Indicator Country Survey, other surveys with a programmatic or specialized focus are financed by other sources such as state and national governments, non-governmental organizations and foundations. As the budgets available for surveys can vary, there is increasing dialogue in the international literature about alternative sampling approaches and their relative merits for both data acquisition as well as their cost (Singh et al. 1996). These debates concern both primary and second-stage sampling. Credible surveys require a representative sample using a reliable randomizing procedure. Although simple random sampling (SRS) is a gold standard for participant selection, SRS is often impractical in the absence of accurate sampling frames (Bostoen and Chalabi 2006). An early solution to second-stage sampling was the ‘random walk’ technique developed by the World Health Organization for the cluster survey used by the Expanded Programme on Immunisation (EPI) (Henderson and Sundaresan 1982). Despite its wide application, the methodological rigor of this technique has been questioned, as the household selection may be opened to bias (Milligan et al. 2004) with households at the centre of villages or clusters being favoured for selection (Kok 1986; Oliphant et al. 2006). Segmentation sampling has been found to be a more rigorous second-stage sampling technique (Turner et al. 1996) and is now advocated in several survey guidelines (Magnani 1997; World Health Organization 2005; SMART 2006). In our experience, about 1 hr is required for a trained person to develop a credible but crude map of a village working with a knowledgeable informant, a process which is required in segmentation sampling. This temporal cost, while acceptable, motivates practitioners to search for quicker approaches. In addition to accepted second-stage sampling methodologies, new techniques are continually proposed, but they require testing. In India, a novel technique of household selection is being used in eight states (in addition to segmentation) to support assessment of the National Vector Borne Disease
Control Program (NVBDCP). This method uses the local knowledge of village community health workers to identify the household with the last birth or the last marriage (LBLM). The 10th household to the right of this household is then selected for interview. The method assumes that the health worker, who provides services to women and children, knows the location of the household with the last birth (LB) or last marriage (LM) and does not rely on the use of a birth register. The LBLM sampling protocol lacks sufficient structure to be used in a consistent manner as the data collector is given the opportunity to select whether to use either the LB or LM, which could result in an opportunistic sample if the choice is based on the proximity to the health worker. In addition, the instruction to ‘move to the 10th house on the right’ is unclear where there are no, or
