STATISTICS IN MEDICINE, VOL. 9 , 3 5 4 3 (1990)
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS OF VITAMIN A IN DEVELOPING COUNTRIES JAMES M. TIELSCH A N D KEITH P. WEST, Jr. International Center for Epidemiologic and Preventive Ophthalmology, Dana Center, Wilmer Ophthalmological Institute, The Johns Hopkins University School of Medicine Wilmer Room 120, 600 Norrh Wolfp Street Baltimore, Maryland 2I205, U.S.A.
SUMMARY Recent investigations conducted in Indonesia have identified an important role for adequate vitamin A nutriture in preventing mortality in young children. A randomized, community-based trial of vitamin A supplementation in northern Sumatra demonstrated a 34 per cent reduction in mortality among pre-school aged children who received twice yearly vitamin A supplementation. These results have stimulated renewed interest in vitamin A as a potentially important mediator of improved child survival. Cost and efficiency considerations are important in these types of trials but must be balanced with logistical constraints that are unique to these environments. A number of these issues are discussed and some general guidelines are proposed for such investigations.
INTRODUCTION
A recent series of studies conducted in Indonesia have shed new light on the role of adequate vitamin A nutriture in preventing mortality among young children in developing areas of the world. The approach taken by these studies, especially those used in the large intervention trial in Aceh Province on northern Sumatra, may also serve as reasonable models for cost-efficient designs in similar situations. This paper will discuss the background for the Aceh vitamin A mortality study, the basic design of the study, a short description of the results, and some general comments on cost efficiency considerations for community-based prevention trials.
BACKGROUND Vitamin A deficiency leading to severe xerophthalmia is the leading cause of blindness among young children in the world.' It is endemic in South and Southeast Asia, East and Southern Africa, and areas of the Middle East, and Central and South America.', It has been well known that mortality rates among children with vision threatening corneal xerophthalmia are very high, up to 50 per cent and higher.' These severe cases are also strongly associated with severe malnutrition Milder signs and symptoms of and with coexistent gastrointestinal and respiratory disease. vitamin A deficiency, such as night blindness and Bitot's spots, were generally considered to have little significance themselves except as community markers of risk for severe corneal involvement. 3'
0277-67 15/90/01003549$05.00 0 1990 by John Wiley & Sons, Ltd.
'
Received June 1989
36
J. M.TIELSCH AND K. P. WEST, Jr
Recent investigations in West Java, Indonesia, however, have demonstrated that children with even mild clinical signs of xerophthalmia have mortality rates four to twelve times higher than age matched normal ~ h i l d r e nAmong .~ survivors, children with mild xerophthalmia also have a threefold excess risk of diarrhoea and a two-fold excess risk of respiratory disease, the major killers of young children in most of the developing world? These findings led to the central question of the Aceh study: can vitamin A supplementation of clinically normal children living in areas with endemic vitamin A deficiency reduce their risk of death? The underlying assumption is that areas with high rates of clinical disease would also have a high prevalence of subclinical vitamin A deficiency. DESIGN To answer this question, a randomized, community intervention trial was conducted in Aceh Province on northern Sumatra. 450 villages with more than 21,000 pre-schbol aged children were evenly randomized to receive or not receive a vitamin A intervention programme. The intervention programme consisted of semi-annual supplementation of all children of one to five years of age with 200,000 International Units of vitamin A, the standard prophylactic dose recommended for prevention of xerophthalmia by WHO and UNICEF.’. Data collection consisted of two surveys approximately one year apart. During the baseline round, all households with children one to five years of age were identified, numbered, and interviewed to determine household composition and demographics, a history of previous child death in the household, a short morbidity history on all eligible children and a variety of other characteristics of the household. All children were then brought to a central site in the village for an ocular examination. Children with clinical signs of xerophthalmia were treated and excluded from the study. A 10 per cent subsample of children also received detailed anthropometric measurements and a dietary interview was conducted with the mother. Two rounds of vitamin A supplementation separated by approximately six months followed the baseline survey. These supplementation rounds were conducted by local village-based volunteers following guidelines established by the Government of Indonesia’s vitamin A deficiency control programme. The follow-up survey round was conducted approximately one year following the baseline round and consisted of similar data collection activities. The vital status of children identified at baseline was determined and an ocular examination was conducted on survivors. The subsample again received detailed anthropometric measurements and all children were treated with 200,000 IU of vitamin A. RESULTS A total of 21,147 children between the ages of 12 and 71 months at baseline were available for the mortality analysis.’ The two groups were similar on baseline demographic and socioeconomic factors including age, sex, occupation of the head of household, maternal education, previous history of child death in the family, sources of drinking water, and distances to various community facilities such as schools and the health ~ e n t r e .Baseline ~ health and nutritional status indicators were also similar, including recent histories of fever, cough, measles, and anthropometric indices such as height for age and weight for height.7 Control children had slightly higher rates of xerophthalmia at baseline, 2.3 per cent versus 1.9 per cent (these cases were treated in both groups and excluded),but this difference was confined to males.’ There was also a slight excess prevalence
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS
37
Table I. Mortality rates for programme and control groups, Aceh Province, Indonesia ~
~~
1 to 5 year old mortality
Group
N
Deaths
per lo00
Relative risk (95 per cent CI)
Vitamin A Control
10,917
53
10,230
75
4.9 7.4
1.0 1.51 (1.03, 2.28)
Taken from Reference 7
of diarrhoea among controls, 8.7 per cent versus 7.1 per cent in the programme group and this was seen primarily in females7 Mortality results are presented in Table I and demonstrate a 34 per cent reduction in child mortality in the programme group as compared to controls. The confidence interval on the relative risk has accounted for randomization at the village versus individual leveL8 Adjustment for baseline imbalance on xerophthalmia or diarrhoea did not significantly alter the relative risk estimate or the confidence i n t e r ~ a l . ~A, further analysis comparing the mortality experience of participants and non-participants among the vitamin A programme group demonstrated that the intent-to-treat analysis originally conducted may have provided a conservative estimate of the impact of vitamin A supplementation on child m ~ r t a l i t y Other .~ results also showed significant reductions in the prevalence of xerophthalmia in the programme group as compared to the controls and improved ponderal growth among males.73l o The results of this trial and the preceding observational study have generated a renewed interest in vitamin A on the part of scientists and policy makers worldwide.", l 2 They have also generated controversy and the need to explore these ideas further in other areas of the world with endemic vitamin A deficiency. Such studies are now under way in a number of countries. COST AND EFFICIENCY CONSIDERATIONS The opportunities in developing countries for epidemiologic field research including both preventive and therapeutic trials are numeroub. This is particularly true as host country institutions and local investigators become increasingly skilled in conducting public health research. There are, however, a variety of unique obstacles to conducting epidemiologic research in such setting^.'^, l4 Cost and efficiency must always be balanced with administrative and logistical constraints which can often dominate the design and implementation of such studies. For example, some elements of effective design may be statistically inefficient but logistically necessary. This underlying principle pervades many of the specific issues raised during initial design discussions and each decision must be tested against the realities of the field environment. Cost considerations for a specific project are also important in relation to the resources generally available to the health sector in that country. High profile, very expensive research projects conducted in an environment where even basic needs cannot be met are usually inappropriate. However, it is also important to estimate the long term cost savings to the health sector that can be anticipated if the trial is successful in identifying an intervention that can reduce morbidity or improve prognosis. A number of these cost and efficiency considerations will be discussed using community-based preventive trials of vitamin A as a model.
38
J. M. TIELSCH AND K. P. WEST, Jr
Table 11. Child mortality rates and sample size requirements for preventive trials measuring the impact of vitamin A supplementation on reduction in child mortality Study location
1 to 5 year old mortality rate
Sample size requirement
Aceh Province, Indonesia Sarlahi District, Nepal Arssi/Bale Regions, Ethiopia
8/1000/Year 25/1000/Year 30/1000/Year
54,955 17,585
14,655
Sample size requirements for each group are based on an estimated reduction in baseline mortality of 25 per cent, type I1 error of 0.10, type I error of 0.05 (two-sided), clustering effect of 1.3, and a 15 per cent increase in sample size to account for losses to follow-up, refusals, etc.
Study Population The choice of a study population for a community-based preventive trial is key to cost containment. A population with high risk of disease will keep the sample to a reasonable size. The importance of assessing impact in high risk populations is well demonstrated when comparing the estimated sample size requirements using the observed mortality rates from the Aceh study and the expected rates in Nepal and Ethiopia where two similar trials are currently under way (Table 11). The sample size requirements in a relatively moderate risk population such as Aceh Province in Indonesia are more than three times that needed for higher risk populations in the lowlands of Nepal and southeastern Ethiopia. While the general rule of choosing high risk populations is a good one, often the populations at highest risk are also the most inaccessible. Choosing them as the study population can result in significantly increased communications, transportation, maintenance, and supervisory costs, thereby negating many of the savings which would accrue from a reduced sample size.
Participation rates High participation rates are essential for reliable inference from community-based preventive trials of treatment efficacy or programme effectiveness. Non-participants in public health programme interventions are invariably different from participants and are often those at highest risk. In trials of programme effectiveness, low participation rates can obscure real intervention efficacy by treating only those subjects easiest to reach and least likely to benefit. Community level acceptance is key to achieving adequate participation. Each trial needs to be fully discussed with local officials at every level and the public health and medical communities need to be supportive of the project. In certain areas of the world the co-operation of local, indigenous practitioners is also necessary for the population to feel confident about the project. Numerous community meetings require a great deal of time and effort but can yield an informed and co-operative study population. If proper attention is paid, there is little reason why participation rates should not reach 85 per cent or higher.
Unit of randomization The unit of randomization should be chosen to balance statistical efficiency with logistic demands. While larger units such as households or villages are less efficient when the disease under question clusters, the logistics of the project may preclude randomizing at the individual level. This is
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASEDTRIALS
39
especially true in placebo controlled trials when the effect of the treatment is long lasting, such as immunization trials or large dose vitamin A supplementation trials. In those situations, larger randomization units may, in fact, increase efficiency if they reduce accidental contamination of the control group with the treatment. In addition, certain types of interventions do not lend themselves to randomization at the individual level. Interventions such as health education, water supply development, or latrine building programmes can only be conducted effectively at the community level. Variable selection The selection of specific data items to be collected at baseline and during each follow-up round has important implications for costs and data quality. As the number of variables to be measured exceeds a practical limit, costs may rise disproportionately to their scientific value while the quality of every data item deteriorates. In large scale preventive trials, questions should be tightly focused on the primary variables of interest. Close attention should be paid to items that define the outcome variable and to key data items that can be used to check the comparability of the randomized groups at baseline. Collection of an extensive series of baseline variables that have little relationship to the outcome should be avoided as this effort may only compromise the quality of the key data items. The educational level of field workers can vary tremendously from one setting to another. Most often they will be less experienced with the detail and rigor required for adequate data quality than experienced research nurses or other personnel used in western clinical trials. When education and experience of the field staff are limited, great care must be taken to focus their attention on those data items which are most important. The complexity of the variables being measured must also be adjusted to the level of sophistication of both the field staff and the community. Forms and questionnaires must often be translated from a western language to a national language and then again into a local dialect or language. Each point of translation must be carefully checked for meaning especially if local terms are thought to be specific to certain clinical signs or symptoms. The larger the number of questions, the more complicated and laborious this process becomes. In addition, supervision of the interview process becomes more difficult. For a variety of reasons, some of which have been discussed, each variable must be carefully considered and justified prior to being incorporated into the data collection process. While there are no general rules about the appropriate number of data items, it is often helpful to plan on using at least one-third fewer items than are considered during initial design discussions. Data entry and management In the current environment the use of microcomputers for data entry and initial management is possible in almost every situation in which a large scale preventive trial would be undertaken. The advantages of intelligent data entry for error detection and correction are well known. They also permit the timely production of error reports that can be returned to the field for any necessary resolution. To optimize communications, the field headquarters and the data centre should be located as close to one another as possible. The feasibility of this will be determined by the adequacy of the physical facilities, the electrical supply and other environmental conditions. The close proximity of these two key operational centres will facilitate timely and efficient use of summary reports for quality control. Close interaction will also permit identification of specific areas where additional training and/or supervision may be necessary. Physical proximity can also serve to maintain the cohesiveness of the entire project team.
40
J. M. TIELSCH AND K. P. WEST, Jr
Salary structure The primary determinant of cost for community-based preventive trials is the salary structure of the staff. Large scale community trials often require large numbers of field workers dedicated fulltime to the data collection effort over an extended period of time. Even relatively small differences in salary structure can have a large impact on total costs. Salaries of field staff are usually significantly lower than similar positions in developed countries. This differential can be as high as 20-fold. In general, salaries of the locally hired field staff should follow the guidelines of the local collaborating agency, be that the Ministry of Health, a local university or a non-governmental organization. On-site expatriate involvement can also significantly increase project costs. A typical benefits package for an expatriate scientist living in a developing country may include housing, a vehicle, home leave, and the costs of western-style education for his/her children in addition to the traditional university benefits. In general, an expatriate should not be placed in a position that justifiably can be filled by a qualified local scientist both for cost reasons and to encourage local institutional development. Significant cost savings can also be realized in a setting where there is long term, institutional collaboration. In such situations many of the administrative and set-up costs for a number of projects can be incorporated into one unit, resulting in greatly improved efficiency and productivity. While overall personnel costs are usually much less expensive for field trials conducted in developing countries, other costs such as supplies, vehicles, petrol, and major equipment can be considerably higher. Often the availability of certain items are quite limited, and the technology in other support services such as printing and computer services can be less than typically found in developed countries. This will often force one to use a more labour intensive approach for certain tasks such as editing, typing, and filing. One other cost associated with large intervention trials is the opportunity cost to the local health system. The employment of local administrative and scientific staff is often in direct competition with the needs of local agencies. In these situations the local applicability of the results of the project become critically important in justifying the diversion of resources and expertise from the local health system. In summary, community-based preventive trials conducted in developing countries are an effectiveway to test interventions that may have an important impact on the health and well being of populations in large areas of the world. There are unique challenges associated with conducting such studies, but flexibility in implementation without compromising rigorous design standards is possible and practical. These types of studies offer additional cost and efficiency challenges that have important consequences for implementation in environments where skilled personnel and other resources are often overextended. REFERENCES
1. Sommer, A. Nutritional Blindness, Oxford University Press, Oxford, 1982. 2. Tielsch, J. M. and Sommer, A. ‘The epidemiology of vitamin A deficiency and xerophthalmia’, Annual Review of Nutrition, 4, 183-205 (1984). 3. Sommer, A., Tarwotjo, I., Hussaini, G., Susanto, D. ‘Increased mortality in children with mild vitamin A deficiency’, Lancet, 2, 585-588 (1983). 4. Sommer, A., Katz, J., Tarwotjo, I. ‘Increased risk of respiratory disease and diarrhea in children with preexisting mild vitamin A deficiency’, American Journal of Clinical Nutrition, 40, 1090-1095 (1984). 5. Sommer, A. Field Guide to the Detection and Control of Xerophthalmia, second edition, World Health Organization, Geneva, 1982. 6. WHO/UNICEF/IVACG Task Force. Vitamin A Supplements: A Guide to Their Use in the Treatment and Prevention of Vitamin A Deficiency and Xerophthalmia, World Health Organization, Geneva, 1988.
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS
41
7. Sommer, A., Tarwotjo, I., Djunaedi, E., West, K. P. Jr., Loedin, A., Tilden, R., Mele, L. and the Aceh Study Group. ‘Impact of vitamin A supplementation on childhood mortality: A randomized controlled community trial’, Lancet, 1, 1169-1 173 (1986). 8. Zeger, S. L. and Edelstein, S. L. ‘Poisson regression with a surrogate X; an analysis of vitamin A and Indonesian children’s mortality’, Applied Statistics, 38, 309-3 18 (1989). 9. Tarwotjo, I., Sommer, A., West, K. P. Jun., Djunaedi, E., Mele, L., Hawkins, B. and the Aceh Study Group. ‘Influence of participation on mortality in a randomized trial of vitamin A prophylaxis’, American Journal of Clinical Nutrition, 45, 1466-1471 (1987). 10. West, K. P. Jun., Djunaedi, E., Pandji, A., Kusdiono, Tarwotjo, I., Sommer, A., and the Aceh Study Group. ‘Vitamin A supplementation and growth: a randomized community trial’, American Journal of Clinical Nutrition, 48, 1257-1264 (1988). 11. United Nations Administrative Coordinating Committee Subcommittee on Nutrition. ‘Statement on vitamin A and mortality’, Food and Nutrition Bulletin, 8, 65 (1986). 12. Editorial. ‘Fall and rise of the anti-infective vitamin’, Lancet, 1, 1191 (1986). 13. Ekanem, E. E. ‘Field epidemiology: methodological constraints and limitations in the developing world’, Public Health, London, 99, 33-36 (1985). 14. de Marr, E. W. J., Chaudhury, R. R., Ekue, J. M. K., Granata, F., Walker, A. N. ‘Management of clinical trials in developing countries’, Journal of International Medical Research, 11, 1-5 (1983).
DISCUSSION
Dr. Jonas Ellenberg: We have had problems doing studies in foreign countries if we don’t have people with a high degree of training in field research on site in each of the villages. Did you use natives of the villages to do your field work? Dr. Tielsch Our field workers are not actually villagers themselves. They may be from the provincial capital or the district capital or something like that. We tried to take high school graduates from those particular environments. We don’t stick an individual in a village and leave them there for two years. We essentially organize and rotate teams through the villages. For example, in Java the local trained team returned every four months to collect follow-up data on the preceding four months. We have management people who are responsible for assuring that happens reliably. Dr. Friedewald: Why are you doing studies in Nepal and Ethiopia? Wasn’t the Indonesian trial conclusive?
Dr. Tielsch: We are doing studies in Nepal and Ethiopia because there is enormous controversy in the international health community about the Indonesian trial. Many people don’t believe the results because of the large size of the effect. Even immunization can’t reduce total mortality by a third. The general international scientific community wants more data before recommending the routine prophylactic use of vitamin A. Dr. Lang: Have any governments, in particular Indonesia, begun providing vitamin A supplementation following reports of the results of this trial, or are governments waiting for results from additional studies?
Dr. Tielsch: There were ongoing government programmes already in India, Bangladesh, and in certain areas in Indonesia. Indonesia had already decided to nationalize this programme, but recognized that it could not include all 7000 islands immediately. Therefore, Indonesia offered villages in this particular area the opportunity to be randomized. After our results became known,
42
J. M. TIELSCH AND K. P. WEST, Jr
several countries expressed much stronger interest in mass distribution of vitamin A, but no others have gone to a national programme yet. Both from this trial and from another study performed in central Java, the government in Indonesia is convinced of the value of MSG fortification of vitamin A. Both studies showed almost identical results in terms of mortality reduction. The Indonesian government intends to use fortification, which is less expensive, rather than supplementation.
Dr. Meier: We have grossly underestimated the potential of cluster units in clinical trials generally. You have made excellent points concerning some of your design’s features. For example, the lack of contamination and the simplicity of covering everybody rather than having to refer to a list to determine who received the pill is a major benefit. The use of clusters is an opportunity for improvement in efficiency in many kinds of clinical trials. Dr. Prentice: What was the intraclass correlation in your Indonesian studies? Dr. Tielsch The clustering led to about a 30 per cent increase in variance. Dr. Prentice: How did you deal with potential liver damage with this large dose of 200,000 international units? Dr. Tielsch Concerns about liver damage are related solely to people who are relatively sufficient in vitamin A. Among the children we studied, the mean vitamin A serum level is roughly 18 instead of roughly 50 among children in the United States. Dr. Yusuf: What did the study cost in total? Dr. Tielsch: The total costs were roughly 1.4 million dollars. Dr. Tognoni: How did you account for the so-called placebo effect? Your presence may have affected the level of general medical care in the different villages. The observed decrease in mortality could have been due to less diarrhoea or fatal infection. Dr. Tielsch: The active treatment for this particular study involved one person, nominated by the head of the village, who gave out vitamin A capsules twice a year to children under six years of age in his village. It is hard to believe that one stroll through the village every six months could have a placebo effect that would reduce mortality by 30 per cent. Dr. Tognoni: Did you count the diarrhoea cases? Dr. Tielsch: We ascertained cases of diarrhoea at baseline and adjusted for their effect in the mortality analysis. We anticipate that supplementation reduces either the incidence or the severity of diarrhoea and respiratory tract infections. In fact there are longitudinal observational data from Indonesia that support this idea. I don’t see any other way the intervention could be having such a powerful impact on total mortality because these diseases dominate childhood mortality. Specific morbidity studies are ongoing in central Java. Their designs require that households be visited two to three times a week to monitor the duration and severity of episodes of diarrhoea. These logistically intensive kind of operations are very different from mortality studies.
COST AND EFFICIENCY CONSIDERATIONS I N COMMUNITY-BASED TRIALS
43
Dr. Yusuf: I think there is a strong biological rationale for a reduction in diarrhoea and respiratory illnesses due to vitamin A supplementation. Vitamin A deficiency is one of the most common reasons for reduced resistance to infections in children. Having worked in villages in India, I think it is very unlikely that somebody who is a high school graduate going around distributing a pill once every six months actually makes a large impact on general medical care in a general setting. I worked for three months in these villages and tried very hard to d o various things, but it is very difficult. People are resistant to changing habits or lifestyle.
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS OF VITAMIN A IN DEVELOPING COUNTRIES JAMES M. TIELSCH A N D KEITH P. WEST, Jr. International Center for Epidemiologic and Preventive Ophthalmology, Dana Center, Wilmer Ophthalmological Institute, The Johns Hopkins University School of Medicine Wilmer Room 120, 600 Norrh Wolfp Street Baltimore, Maryland 2I205, U.S.A.
SUMMARY Recent investigations conducted in Indonesia have identified an important role for adequate vitamin A nutriture in preventing mortality in young children. A randomized, community-based trial of vitamin A supplementation in northern Sumatra demonstrated a 34 per cent reduction in mortality among pre-school aged children who received twice yearly vitamin A supplementation. These results have stimulated renewed interest in vitamin A as a potentially important mediator of improved child survival. Cost and efficiency considerations are important in these types of trials but must be balanced with logistical constraints that are unique to these environments. A number of these issues are discussed and some general guidelines are proposed for such investigations.
INTRODUCTION
A recent series of studies conducted in Indonesia have shed new light on the role of adequate vitamin A nutriture in preventing mortality among young children in developing areas of the world. The approach taken by these studies, especially those used in the large intervention trial in Aceh Province on northern Sumatra, may also serve as reasonable models for cost-efficient designs in similar situations. This paper will discuss the background for the Aceh vitamin A mortality study, the basic design of the study, a short description of the results, and some general comments on cost efficiency considerations for community-based prevention trials.
BACKGROUND Vitamin A deficiency leading to severe xerophthalmia is the leading cause of blindness among young children in the world.' It is endemic in South and Southeast Asia, East and Southern Africa, and areas of the Middle East, and Central and South America.', It has been well known that mortality rates among children with vision threatening corneal xerophthalmia are very high, up to 50 per cent and higher.' These severe cases are also strongly associated with severe malnutrition Milder signs and symptoms of and with coexistent gastrointestinal and respiratory disease. vitamin A deficiency, such as night blindness and Bitot's spots, were generally considered to have little significance themselves except as community markers of risk for severe corneal involvement. 3'
0277-67 15/90/01003549$05.00 0 1990 by John Wiley & Sons, Ltd.
'
Received June 1989
36
J. M.TIELSCH AND K. P. WEST, Jr
Recent investigations in West Java, Indonesia, however, have demonstrated that children with even mild clinical signs of xerophthalmia have mortality rates four to twelve times higher than age matched normal ~ h i l d r e nAmong .~ survivors, children with mild xerophthalmia also have a threefold excess risk of diarrhoea and a two-fold excess risk of respiratory disease, the major killers of young children in most of the developing world? These findings led to the central question of the Aceh study: can vitamin A supplementation of clinically normal children living in areas with endemic vitamin A deficiency reduce their risk of death? The underlying assumption is that areas with high rates of clinical disease would also have a high prevalence of subclinical vitamin A deficiency. DESIGN To answer this question, a randomized, community intervention trial was conducted in Aceh Province on northern Sumatra. 450 villages with more than 21,000 pre-schbol aged children were evenly randomized to receive or not receive a vitamin A intervention programme. The intervention programme consisted of semi-annual supplementation of all children of one to five years of age with 200,000 International Units of vitamin A, the standard prophylactic dose recommended for prevention of xerophthalmia by WHO and UNICEF.’. Data collection consisted of two surveys approximately one year apart. During the baseline round, all households with children one to five years of age were identified, numbered, and interviewed to determine household composition and demographics, a history of previous child death in the household, a short morbidity history on all eligible children and a variety of other characteristics of the household. All children were then brought to a central site in the village for an ocular examination. Children with clinical signs of xerophthalmia were treated and excluded from the study. A 10 per cent subsample of children also received detailed anthropometric measurements and a dietary interview was conducted with the mother. Two rounds of vitamin A supplementation separated by approximately six months followed the baseline survey. These supplementation rounds were conducted by local village-based volunteers following guidelines established by the Government of Indonesia’s vitamin A deficiency control programme. The follow-up survey round was conducted approximately one year following the baseline round and consisted of similar data collection activities. The vital status of children identified at baseline was determined and an ocular examination was conducted on survivors. The subsample again received detailed anthropometric measurements and all children were treated with 200,000 IU of vitamin A. RESULTS A total of 21,147 children between the ages of 12 and 71 months at baseline were available for the mortality analysis.’ The two groups were similar on baseline demographic and socioeconomic factors including age, sex, occupation of the head of household, maternal education, previous history of child death in the family, sources of drinking water, and distances to various community facilities such as schools and the health ~ e n t r e .Baseline ~ health and nutritional status indicators were also similar, including recent histories of fever, cough, measles, and anthropometric indices such as height for age and weight for height.7 Control children had slightly higher rates of xerophthalmia at baseline, 2.3 per cent versus 1.9 per cent (these cases were treated in both groups and excluded),but this difference was confined to males.’ There was also a slight excess prevalence
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS
37
Table I. Mortality rates for programme and control groups, Aceh Province, Indonesia ~
~~
1 to 5 year old mortality
Group
N
Deaths
per lo00
Relative risk (95 per cent CI)
Vitamin A Control
10,917
53
10,230
75
4.9 7.4
1.0 1.51 (1.03, 2.28)
Taken from Reference 7
of diarrhoea among controls, 8.7 per cent versus 7.1 per cent in the programme group and this was seen primarily in females7 Mortality results are presented in Table I and demonstrate a 34 per cent reduction in child mortality in the programme group as compared to controls. The confidence interval on the relative risk has accounted for randomization at the village versus individual leveL8 Adjustment for baseline imbalance on xerophthalmia or diarrhoea did not significantly alter the relative risk estimate or the confidence i n t e r ~ a l . ~A, further analysis comparing the mortality experience of participants and non-participants among the vitamin A programme group demonstrated that the intent-to-treat analysis originally conducted may have provided a conservative estimate of the impact of vitamin A supplementation on child m ~ r t a l i t y Other .~ results also showed significant reductions in the prevalence of xerophthalmia in the programme group as compared to the controls and improved ponderal growth among males.73l o The results of this trial and the preceding observational study have generated a renewed interest in vitamin A on the part of scientists and policy makers worldwide.", l 2 They have also generated controversy and the need to explore these ideas further in other areas of the world with endemic vitamin A deficiency. Such studies are now under way in a number of countries. COST AND EFFICIENCY CONSIDERATIONS The opportunities in developing countries for epidemiologic field research including both preventive and therapeutic trials are numeroub. This is particularly true as host country institutions and local investigators become increasingly skilled in conducting public health research. There are, however, a variety of unique obstacles to conducting epidemiologic research in such setting^.'^, l4 Cost and efficiency must always be balanced with administrative and logistical constraints which can often dominate the design and implementation of such studies. For example, some elements of effective design may be statistically inefficient but logistically necessary. This underlying principle pervades many of the specific issues raised during initial design discussions and each decision must be tested against the realities of the field environment. Cost considerations for a specific project are also important in relation to the resources generally available to the health sector in that country. High profile, very expensive research projects conducted in an environment where even basic needs cannot be met are usually inappropriate. However, it is also important to estimate the long term cost savings to the health sector that can be anticipated if the trial is successful in identifying an intervention that can reduce morbidity or improve prognosis. A number of these cost and efficiency considerations will be discussed using community-based preventive trials of vitamin A as a model.
38
J. M. TIELSCH AND K. P. WEST, Jr
Table 11. Child mortality rates and sample size requirements for preventive trials measuring the impact of vitamin A supplementation on reduction in child mortality Study location
1 to 5 year old mortality rate
Sample size requirement
Aceh Province, Indonesia Sarlahi District, Nepal Arssi/Bale Regions, Ethiopia
8/1000/Year 25/1000/Year 30/1000/Year
54,955 17,585
14,655
Sample size requirements for each group are based on an estimated reduction in baseline mortality of 25 per cent, type I1 error of 0.10, type I error of 0.05 (two-sided), clustering effect of 1.3, and a 15 per cent increase in sample size to account for losses to follow-up, refusals, etc.
Study Population The choice of a study population for a community-based preventive trial is key to cost containment. A population with high risk of disease will keep the sample to a reasonable size. The importance of assessing impact in high risk populations is well demonstrated when comparing the estimated sample size requirements using the observed mortality rates from the Aceh study and the expected rates in Nepal and Ethiopia where two similar trials are currently under way (Table 11). The sample size requirements in a relatively moderate risk population such as Aceh Province in Indonesia are more than three times that needed for higher risk populations in the lowlands of Nepal and southeastern Ethiopia. While the general rule of choosing high risk populations is a good one, often the populations at highest risk are also the most inaccessible. Choosing them as the study population can result in significantly increased communications, transportation, maintenance, and supervisory costs, thereby negating many of the savings which would accrue from a reduced sample size.
Participation rates High participation rates are essential for reliable inference from community-based preventive trials of treatment efficacy or programme effectiveness. Non-participants in public health programme interventions are invariably different from participants and are often those at highest risk. In trials of programme effectiveness, low participation rates can obscure real intervention efficacy by treating only those subjects easiest to reach and least likely to benefit. Community level acceptance is key to achieving adequate participation. Each trial needs to be fully discussed with local officials at every level and the public health and medical communities need to be supportive of the project. In certain areas of the world the co-operation of local, indigenous practitioners is also necessary for the population to feel confident about the project. Numerous community meetings require a great deal of time and effort but can yield an informed and co-operative study population. If proper attention is paid, there is little reason why participation rates should not reach 85 per cent or higher.
Unit of randomization The unit of randomization should be chosen to balance statistical efficiency with logistic demands. While larger units such as households or villages are less efficient when the disease under question clusters, the logistics of the project may preclude randomizing at the individual level. This is
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASEDTRIALS
39
especially true in placebo controlled trials when the effect of the treatment is long lasting, such as immunization trials or large dose vitamin A supplementation trials. In those situations, larger randomization units may, in fact, increase efficiency if they reduce accidental contamination of the control group with the treatment. In addition, certain types of interventions do not lend themselves to randomization at the individual level. Interventions such as health education, water supply development, or latrine building programmes can only be conducted effectively at the community level. Variable selection The selection of specific data items to be collected at baseline and during each follow-up round has important implications for costs and data quality. As the number of variables to be measured exceeds a practical limit, costs may rise disproportionately to their scientific value while the quality of every data item deteriorates. In large scale preventive trials, questions should be tightly focused on the primary variables of interest. Close attention should be paid to items that define the outcome variable and to key data items that can be used to check the comparability of the randomized groups at baseline. Collection of an extensive series of baseline variables that have little relationship to the outcome should be avoided as this effort may only compromise the quality of the key data items. The educational level of field workers can vary tremendously from one setting to another. Most often they will be less experienced with the detail and rigor required for adequate data quality than experienced research nurses or other personnel used in western clinical trials. When education and experience of the field staff are limited, great care must be taken to focus their attention on those data items which are most important. The complexity of the variables being measured must also be adjusted to the level of sophistication of both the field staff and the community. Forms and questionnaires must often be translated from a western language to a national language and then again into a local dialect or language. Each point of translation must be carefully checked for meaning especially if local terms are thought to be specific to certain clinical signs or symptoms. The larger the number of questions, the more complicated and laborious this process becomes. In addition, supervision of the interview process becomes more difficult. For a variety of reasons, some of which have been discussed, each variable must be carefully considered and justified prior to being incorporated into the data collection process. While there are no general rules about the appropriate number of data items, it is often helpful to plan on using at least one-third fewer items than are considered during initial design discussions. Data entry and management In the current environment the use of microcomputers for data entry and initial management is possible in almost every situation in which a large scale preventive trial would be undertaken. The advantages of intelligent data entry for error detection and correction are well known. They also permit the timely production of error reports that can be returned to the field for any necessary resolution. To optimize communications, the field headquarters and the data centre should be located as close to one another as possible. The feasibility of this will be determined by the adequacy of the physical facilities, the electrical supply and other environmental conditions. The close proximity of these two key operational centres will facilitate timely and efficient use of summary reports for quality control. Close interaction will also permit identification of specific areas where additional training and/or supervision may be necessary. Physical proximity can also serve to maintain the cohesiveness of the entire project team.
40
J. M. TIELSCH AND K. P. WEST, Jr
Salary structure The primary determinant of cost for community-based preventive trials is the salary structure of the staff. Large scale community trials often require large numbers of field workers dedicated fulltime to the data collection effort over an extended period of time. Even relatively small differences in salary structure can have a large impact on total costs. Salaries of field staff are usually significantly lower than similar positions in developed countries. This differential can be as high as 20-fold. In general, salaries of the locally hired field staff should follow the guidelines of the local collaborating agency, be that the Ministry of Health, a local university or a non-governmental organization. On-site expatriate involvement can also significantly increase project costs. A typical benefits package for an expatriate scientist living in a developing country may include housing, a vehicle, home leave, and the costs of western-style education for his/her children in addition to the traditional university benefits. In general, an expatriate should not be placed in a position that justifiably can be filled by a qualified local scientist both for cost reasons and to encourage local institutional development. Significant cost savings can also be realized in a setting where there is long term, institutional collaboration. In such situations many of the administrative and set-up costs for a number of projects can be incorporated into one unit, resulting in greatly improved efficiency and productivity. While overall personnel costs are usually much less expensive for field trials conducted in developing countries, other costs such as supplies, vehicles, petrol, and major equipment can be considerably higher. Often the availability of certain items are quite limited, and the technology in other support services such as printing and computer services can be less than typically found in developed countries. This will often force one to use a more labour intensive approach for certain tasks such as editing, typing, and filing. One other cost associated with large intervention trials is the opportunity cost to the local health system. The employment of local administrative and scientific staff is often in direct competition with the needs of local agencies. In these situations the local applicability of the results of the project become critically important in justifying the diversion of resources and expertise from the local health system. In summary, community-based preventive trials conducted in developing countries are an effectiveway to test interventions that may have an important impact on the health and well being of populations in large areas of the world. There are unique challenges associated with conducting such studies, but flexibility in implementation without compromising rigorous design standards is possible and practical. These types of studies offer additional cost and efficiency challenges that have important consequences for implementation in environments where skilled personnel and other resources are often overextended. REFERENCES
1. Sommer, A. Nutritional Blindness, Oxford University Press, Oxford, 1982. 2. Tielsch, J. M. and Sommer, A. ‘The epidemiology of vitamin A deficiency and xerophthalmia’, Annual Review of Nutrition, 4, 183-205 (1984). 3. Sommer, A., Tarwotjo, I., Hussaini, G., Susanto, D. ‘Increased mortality in children with mild vitamin A deficiency’, Lancet, 2, 585-588 (1983). 4. Sommer, A., Katz, J., Tarwotjo, I. ‘Increased risk of respiratory disease and diarrhea in children with preexisting mild vitamin A deficiency’, American Journal of Clinical Nutrition, 40, 1090-1095 (1984). 5. Sommer, A. Field Guide to the Detection and Control of Xerophthalmia, second edition, World Health Organization, Geneva, 1982. 6. WHO/UNICEF/IVACG Task Force. Vitamin A Supplements: A Guide to Their Use in the Treatment and Prevention of Vitamin A Deficiency and Xerophthalmia, World Health Organization, Geneva, 1988.
COST AND EFFICIENCY CONSIDERATIONS IN COMMUNITY-BASED TRIALS
41
7. Sommer, A., Tarwotjo, I., Djunaedi, E., West, K. P. Jr., Loedin, A., Tilden, R., Mele, L. and the Aceh Study Group. ‘Impact of vitamin A supplementation on childhood mortality: A randomized controlled community trial’, Lancet, 1, 1169-1 173 (1986). 8. Zeger, S. L. and Edelstein, S. L. ‘Poisson regression with a surrogate X; an analysis of vitamin A and Indonesian children’s mortality’, Applied Statistics, 38, 309-3 18 (1989). 9. Tarwotjo, I., Sommer, A., West, K. P. Jun., Djunaedi, E., Mele, L., Hawkins, B. and the Aceh Study Group. ‘Influence of participation on mortality in a randomized trial of vitamin A prophylaxis’, American Journal of Clinical Nutrition, 45, 1466-1471 (1987). 10. West, K. P. Jun., Djunaedi, E., Pandji, A., Kusdiono, Tarwotjo, I., Sommer, A., and the Aceh Study Group. ‘Vitamin A supplementation and growth: a randomized community trial’, American Journal of Clinical Nutrition, 48, 1257-1264 (1988). 11. United Nations Administrative Coordinating Committee Subcommittee on Nutrition. ‘Statement on vitamin A and mortality’, Food and Nutrition Bulletin, 8, 65 (1986). 12. Editorial. ‘Fall and rise of the anti-infective vitamin’, Lancet, 1, 1191 (1986). 13. Ekanem, E. E. ‘Field epidemiology: methodological constraints and limitations in the developing world’, Public Health, London, 99, 33-36 (1985). 14. de Marr, E. W. J., Chaudhury, R. R., Ekue, J. M. K., Granata, F., Walker, A. N. ‘Management of clinical trials in developing countries’, Journal of International Medical Research, 11, 1-5 (1983).
DISCUSSION
Dr. Jonas Ellenberg: We have had problems doing studies in foreign countries if we don’t have people with a high degree of training in field research on site in each of the villages. Did you use natives of the villages to do your field work? Dr. Tielsch Our field workers are not actually villagers themselves. They may be from the provincial capital or the district capital or something like that. We tried to take high school graduates from those particular environments. We don’t stick an individual in a village and leave them there for two years. We essentially organize and rotate teams through the villages. For example, in Java the local trained team returned every four months to collect follow-up data on the preceding four months. We have management people who are responsible for assuring that happens reliably. Dr. Friedewald: Why are you doing studies in Nepal and Ethiopia? Wasn’t the Indonesian trial conclusive?
Dr. Tielsch: We are doing studies in Nepal and Ethiopia because there is enormous controversy in the international health community about the Indonesian trial. Many people don’t believe the results because of the large size of the effect. Even immunization can’t reduce total mortality by a third. The general international scientific community wants more data before recommending the routine prophylactic use of vitamin A. Dr. Lang: Have any governments, in particular Indonesia, begun providing vitamin A supplementation following reports of the results of this trial, or are governments waiting for results from additional studies?
Dr. Tielsch: There were ongoing government programmes already in India, Bangladesh, and in certain areas in Indonesia. Indonesia had already decided to nationalize this programme, but recognized that it could not include all 7000 islands immediately. Therefore, Indonesia offered villages in this particular area the opportunity to be randomized. After our results became known,
42
J. M. TIELSCH AND K. P. WEST, Jr
several countries expressed much stronger interest in mass distribution of vitamin A, but no others have gone to a national programme yet. Both from this trial and from another study performed in central Java, the government in Indonesia is convinced of the value of MSG fortification of vitamin A. Both studies showed almost identical results in terms of mortality reduction. The Indonesian government intends to use fortification, which is less expensive, rather than supplementation.
Dr. Meier: We have grossly underestimated the potential of cluster units in clinical trials generally. You have made excellent points concerning some of your design’s features. For example, the lack of contamination and the simplicity of covering everybody rather than having to refer to a list to determine who received the pill is a major benefit. The use of clusters is an opportunity for improvement in efficiency in many kinds of clinical trials. Dr. Prentice: What was the intraclass correlation in your Indonesian studies? Dr. Tielsch The clustering led to about a 30 per cent increase in variance. Dr. Prentice: How did you deal with potential liver damage with this large dose of 200,000 international units? Dr. Tielsch Concerns about liver damage are related solely to people who are relatively sufficient in vitamin A. Among the children we studied, the mean vitamin A serum level is roughly 18 instead of roughly 50 among children in the United States. Dr. Yusuf: What did the study cost in total? Dr. Tielsch: The total costs were roughly 1.4 million dollars. Dr. Tognoni: How did you account for the so-called placebo effect? Your presence may have affected the level of general medical care in the different villages. The observed decrease in mortality could have been due to less diarrhoea or fatal infection. Dr. Tielsch: The active treatment for this particular study involved one person, nominated by the head of the village, who gave out vitamin A capsules twice a year to children under six years of age in his village. It is hard to believe that one stroll through the village every six months could have a placebo effect that would reduce mortality by 30 per cent. Dr. Tognoni: Did you count the diarrhoea cases? Dr. Tielsch: We ascertained cases of diarrhoea at baseline and adjusted for their effect in the mortality analysis. We anticipate that supplementation reduces either the incidence or the severity of diarrhoea and respiratory tract infections. In fact there are longitudinal observational data from Indonesia that support this idea. I don’t see any other way the intervention could be having such a powerful impact on total mortality because these diseases dominate childhood mortality. Specific morbidity studies are ongoing in central Java. Their designs require that households be visited two to three times a week to monitor the duration and severity of episodes of diarrhoea. These logistically intensive kind of operations are very different from mortality studies.
COST AND EFFICIENCY CONSIDERATIONS I N COMMUNITY-BASED TRIALS
43
Dr. Yusuf: I think there is a strong biological rationale for a reduction in diarrhoea and respiratory illnesses due to vitamin A supplementation. Vitamin A deficiency is one of the most common reasons for reduced resistance to infections in children. Having worked in villages in India, I think it is very unlikely that somebody who is a high school graduate going around distributing a pill once every six months actually makes a large impact on general medical care in a general setting. I worked for three months in these villages and tried very hard to d o various things, but it is very difficult. People are resistant to changing habits or lifestyle.
