Evaluation and Program Planning 42 (2014) 50–56

Contents lists available at ScienceDirect

Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan

Success and challenges of measuring program impacts: An international study of an infant nutrition program for AIDS orphans Deborah Sturtevant a,*, Jane S. Wimmer b,1 a b

Sociology and Social Work, Hope College, 41 Graves Place, Holland, MI 49423, United States Dalton State College School of Social Work, 650 College Drive, Dalton, GA 30720, United States

A R T I C L E I N F O

A B S T R A C T

Article history: Received 16 March 2013 Received in revised form 20 September 2013 Accepted 29 September 2013

The HIV/AIDS epidemic in Zambia threatens maternal survival and jeopardizes the ability for families to care for their children. The Christian Alliance for Children in Zambia (CACZ) operates a program called Milk and Medicine (M&M) that distributes food, formula, and medicine at churches in the compounds. This article reports on a mixed methods study to evaluate the outcomes of the M&M program. On-site interviews with families combined with an analysis of a longitudinal data set were the methods used. The results of the study showed families face continuous hardship including hunger, unemployment, disease, and loss. Families expressed appreciation for the program and its staff and suggested improvements. The longitudinal data review helped researchers to recommend an improved protocol for data management. Improved data will assist researchers in an on-going evaluation to compare the growth rates of children in the study to the Zambian normal growth charts. Lessons learned from this evaluation validated the use of mixed methods design for exploratory research on an emerging program. Lessons were also learned about the difficulty of working in natural settings with political and cultural variations. Future evaluations of the M&M program are expected to shed light on more specific program impacts. ß 2013 Elsevier Ltd. All rights reserved.

Keywords: Mixed methods AIDS Orphans Supplemental feeding program Vulnerable children Zambia

1. Introduction

1.1. The setting

This program evaluation used a mixed-methods approach to evaluate the Milk and Medicine (M&M) program, listening to caretaker’s stories of program impact shared in interviews and focus groups and looking at existing retrospective data records of malnourished infants. Research was conducted in Lusaka, Zambia and designed and led by two social work professors from the United States. This paper reflects the successes and confounding challenges of a program evaluation designed to measure the impact of a community-based and family-focused infant feeding program. The findings reported in this paper led to data collection improvements and program recommendations. Implementation of incremental changes have resulted in more robust data and improved and expanded programs which are reviewed in on-going research and planned for dissemination in future publication.

The Milk and Medicine (M&M) program established in 2004 is a service of the Christian Alliance for Children in Zambia (CACZ), a Zambian faith-based NGO established in 1997 by Alliance for Children Everywhere (ACE). ACE is a US based agency that raises funds through philanthropy, non-profit partnerships, and private foundation grants. M&M’s goal is to improve infant health and strengthen families to prevent child abandonment and institutionalization. The program utilizes three inter-related strategies: distribution and provision of nutritional supplements for infants living in families, medicine when it is not available to families through other means, and social work support. The main site of the program is at CACZ office headquarters in a residential area on the outskirts of Lusaka, the capital city of Zambia. The site includes a crisis nursery for infants and a residential home for toddlers, each serving approximately 15 children, as well as offices for social workers and administrators. The social work offices were used for the family interviews and data collection in this study. Children referred to M&M through the Child Protection Unit of the Police Department, University Teaching Hospital, the Department of Social Services, non-governmental organizations, community health clinics, CACZ’s House of Moses crisis care nursery, or

* Corresponding author at: Department of Sociology and Social Work, 41 Graves Place, Hope College, Holland MI 49422, United States. Tel.: +1 616 395 7916; fax: +1-706-272-4563. E-mail addresses: [email protected] (D. Sturtevant), [email protected] (J.S. Wimmer). 1 Tel.: +1 706 272 2685; fax: +1 706 2520. 0149-7189/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.evalprogplan.2013.09.004

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

by word of mouth. At the time of this initial evaluation in April 2009, there were four distribution sites in compounds in and around Lusaka. Tins of powdered formula were distributed to infants under age six months whose caregivers or mothers could not provide nutrition through breastfeeding. After the age of six months, ground nuts to grind then add to porridge were added until 18 months, and only ground nuts were provided from 18 to 60 months. 1.2. Scope of the problem of infant starvation The United Nations Millennium Development Goals (United Nations, 2012) focused on meeting the needs of undernourished infants and young children, maternal health, and the response to HIV/AIDS. Unfortunately, challenges have persisted (Brinkman, de Pree, Sanogo, Subran, & Bloem, 2010; Lombe & Ochumbo, 2008; Naidoo, Rennert, Lung, Naidoo, & McKerrow, 2010; Peterson, 2009; Sztam, Fawzi, & Duggan, 2010). Sub-Saharan Africa leads the globe in the impact of HIV/AIDS with 22.4 million mother-infant cases of the 33.4 million worldwide cases (UNAIDS, 2009, p. 11). Approximately 390,000 new infant HIV infections were recorded in the region in 2008, representing 91% of the new infections among all children (p. 21). Roughly 3 million children have contracted HIV/AIDS, with most becoming infected by their mothers during birth or breastfeeding. Chopra, Daviaud, Pattinson, Fonn, and Lawn (2009) stated that annually HIV/AIDS causes thousands of unnecessary maternal and infant deaths in South Africa. They estimate that the maternal mortality rate for HIV positive women is 10 times higher than for HIV negative and that ‘‘the top priority for child survival is clearly the prevention of HIV in children . . .[and] to address neonatal deaths, which account for a third of child deaths’’ (p. 37). In Uganda, the relationship between child health, infant nutrition, and maternal HIV status was the focus of a study interviewing 144 HIV positive mothers/caregivers of children under age five (Bukusuba, Kikafunda & Whitehead, 2009; Magezi, Kikafunda, & Whitehead, 2008). The study found multiple factors related to poor childhood nutrition, including low dietary diversity, malaria, poverty, and low education of the mother/caregiver. Findings indicated that ‘‘poor nutrition mainly affects children between the ages of one and three years’’ (p. 1355). In an attempt to reduce mother-to-child HIV transmission, the 2001–2005 programs in Coˆte d’Ivoire promoted short-term breast feeding and free infant formula for HIV affected families. Becquet et al. (2007) followed 557 children and found that the HIV transmission rate for short-term breast-fed infants was 7% and for formula fed infants 6%, as compared with 22% for long-term breastfed infants, thus stressing the need for available appropriate nutrition for young children. Poverty is often closely linked to malnutrition and illness. Between 1991 and 2006 poverty rates in Zambia increased from 49% to 53% and in 2006 rural poverty rates stood at 78% (Central Statistical Office Zambia, n.d.). The UNICEF policy on infant feeding and HIV that were available at the time of this initial program evaluation recommended exclusive breastfeeding for HIV-infected women for the first six months of life unless replacement feeding is, ‘‘acceptable, feasible, affordable, sustainable and safe’’ (UNICEF, n.d., p. 1). The newer World Health Organization guidelines recommend infants for the first six months of life should be breastfed exclusively because it increases the likelihood of survival in sub-Saharan African countries. While HIV transmission is possible, increased access in recent years to anti-viral medications by HIV positive breastfeeding mothers enables them to continue breastfeeding until the infant is twelve months old. The WHO guidelines are a revision from past practices that encouraged women to breastfeed exclusively until 6 months and then stop

51

when they could provide food sufficient and safe for the child (WHO, 2010). As of the 2012 UN progress report, sub-Saharan Africa failed to make progress to improve the outcomes for infants (United Nations, 2012). According to the Zambian Country Report on HIV/ AIDS (Chandang’oma, Chabwela, & Banda, 2010) 14.3% of Zambia’s population, over 900,000 people, are living with HIV/AIDS (pp. xiii– xiv). Approximately 95,000 of the infected population are children under the age of 14 (UNAIDS, 2008). Women between 30 and 34 have a 26% infection rate, and mother-to-child transmission accounts for 21% of new HIV infections (Chandang’oma, Chabwela, & Banda, 2010, p. 28). Many of those infected are young adults, whose income and roles as parents are critical to the survival of families and society. The loss of parents to AIDS has left more than 600,000 Zambian children as either single (one parent deceased) or double (both parents deceased) orphans (p. 35) and the orphan status of an infant, whether because of maternal death related to childbirth or to AIDS, is closely related to infant malnutrition and death. The estimated under age five years mortality rate for Zambia was 118.8 deaths per 1000 children (p. 9) and 2006 government statistics reported that 54.2% of Zambian children aged three months to 59 months were underweight (Central Statistical Office Zambia, n.d.). 2. Qualitative evaluation At the request of sponsor funding agencies, the administrative staff of Alliance for Children Everywhere invited two researchers to Lusaka, Zambia to interview M&M caregivers (see Appendix A). The goal of on-site investigation was to interview M&M families, to analyze their responses in the aggregate, and to recommend program improvements. Human subject review processes by colleges of the co-investigators approved the research. The two US based researchers considered both cultural and logistical limitations when designing the qualitative research. The Zambian CACZ staff recruited the participant families and four local social workers who served as interviewers. The researchers did not have direct involvement with the Milk and Medicine program and both were consider objective ‘‘external investigators’’ for the evaluation (Fitzpatrick, Sanders & Worthen, 2010). 2.1. Logistics All interviews and focus groups were held at the CACZ main office, a site that was familiar to the participants. A CACZ van driver left early each morning for pick-up points around Lusaka and arrived at 8:00 am with eight or nine families consisting of one or two caregivers and several young children. CACZ staff prepared a breakfast for families who settled around the grounds of the agency as they waited for their interview time. The atmosphere of the setting was relaxed with much friendly visiting and laughter among families. CACZ staff provided entertainment and treats for the children. In total 33 families willingly took part in the study and every family who was invited to come for an interview participated. Although families were not aware that there would not be compensation for their participation, each family was given a bag of sustenance food and small gifts for the children as they prepared to re-board the van to go home. 2.2. Design and reliability Zambian research assistants received an intensive full day training and close supervision during data collection. Caregiver interviews were held in the language most comfortable for the participant: Bemba, Nyanja, Lozi, or English. Each interview was audio recorded, with two caregivers participating in the morning

52

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

each day. The interviews were translated and transcribed into English each afternoon. The use of four interviewers/translators helped lessen ‘‘contamination of projection’’ (Boyatzis, 1998, p. 13) in which the local staff social workers might attribute to the participants their own needs and experiences. The US researchers and local interviewers attempted to stay aware the risk of projection and to promote interviewer neutrality (Patton, 2002); these issues were especially of concern since data were translated. The use of both individual interviews (n = 24) and focus groups (n = 9) added variety to data collection methods. Data saturation was reached through the large number of participants and their ability to be interviewed in their native language coupled with the daily translation of eight interviews by the Zambian interviewers and the review by the researchers each evening. The two focus groups were held by one of the researchers through translation and added to the interview data that had been collected earlier in the process. 2.3. Participants Interviews were confidential and informed consent was obtained. Thirty-three families with 41 children provided data and most participants were women, often the child’s grandmother or aunt. Children ranged in age from one to four with an average age of two. Thirteen of the children had received crisis residential care at House of Moses before being reunited with families. 2.4. Coding and analysis Reliability of qualitative data was the goal of extensive use of multiple coders (Creswell, 2008; Merriam & Associates, 2002; Patton, 2002). The coding of qualitative data used the constant comparative method (Glaser & Strauss, 1967) and noted themes among participants as these became prevalent in interviews. The first analysis was done on-site in Lusaka by one researcher to assure that sufficient quantity and quality of data were gathered. Back in the US two researchers and a student assistant re-coded data after collection was completed providing triangulation of analysis. Although the original themes gathered as the interviews progressed were the basis of the second analysis, these were refined as analysis continued. 2.5. Qualitative findings Three themes of family stress emerged from the interviews and focus groups: poverty, parental death or limitations, and specific unmet needs for the well-being of the families. Additionally, families spoke of their positive relationships with the Milk and Medicine program and made recommendations for program improvement (see Fig. 1). 2.5.1. Poverty As anticipated, the families who needed assistance from M&M lived in extreme poverty. The survival of their children, especially the underweight infants, was in constant jeopardy. One caregiver specifically described the lack of food in her family the day before coming to the interviews, saying, ‘‘Most of the time he wants to eat but I have no food to give him. Here where you are seeing me, I did not eat anything last night; we just slept hungry.’’ Substandard housing was also described. A couple caring for eight children in a one-room house reached a crisis with the birth of twins. The mother stated, ‘‘I am renting and there are times when I fail to pay. . .. We cannot be chased here and there from the landlords. I can’t go anywhere with these children.’’ Although more than half of the participants were employed, their marginalized employment did not provide sustenance for the

Family Stressors

Poverty Illness

Orphan Status

HIV/AIDS Cause Undisclosed Other Situations

Unmet Needs

Food Material needs Education for children Employment

Milk and Medicine Program

Relationship with CACZ Recommended Improvements Importance of the Program

Fig. 1. Themes from family interviews.

family. In many families one or both caregivers were HIV+, had other illnesses, or suffered debilities of old age. These factors directly contributed to the lack of income for the family. Seven of the single caregivers had HIV/AIDS. One young mother shared, HIV/AIDS has really made us suffer because I used to be a very strong person. . . but now the way you are seeing me is not the way I used to be. Even when I would work I can feel my body is weak. Another mother, who is HIV+ and has survived the death of her child’s father by several years, described breast feeding for 4 months and her concern as she took her HIV daughter for testing twice after that time. Her family consists of herself and her sister and 4 children and she said ‘‘everyone looks to me for support’’ from her poorly paid job in a restaurant. 2.5.2. Parental death or illness Approximately two-thirds of the families were caring for orphaned children, often because of AIDS. A slender, elderly looking grandmother, whose infant granddaughter was HIV+, said ‘‘most of my family members have been affected by the same disease [AIDS], and they have died.’’ Another added, ‘‘Yes, it is true. . .. Even my granddaughter’s mother, it is the same disease that has killed her.’’ The devastation of AIDS in one family was poignantly described by a sad young mother who discussed her anger when her HIV+ husband admitted to ‘‘bringing the disease into the home’’ and went on to say that two of her sisters and her brother are all also HIV+. The death of the child’s mother often precipitated the need for the Milk and Medicine program. Tragically symbolizing the collapse of families in Zambia, a grandmother caring for seven children ages 2 years to 18 years tearfully described the death of the mother of the two youngest children, 2 years and 5 years, to AIDS, and the deaths of the other mothers to AIDS, cancer of the cervix, and breast cancer. A teenage sister caring for her infant sibling described her mother’s death: So mum became very sick during her eighth pregnancy such that delivery was through an operation at hospital. . .. Both mum and the baby were safe, but mum became so sick after a few days. In short, mum died one month from that, and baby was left. As this young woman tried to hold the family together, there was no money for formula and the infant faced certain death if the Milk and Medicine program had not been available. For poor families in Zambia the death of a newborn following the mother’s death is an anticipated occurrence. In cases where children’s mothers were living and caring for them, multiple births and HIV/AIDS both played a role in the

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

mothers’ inability to breast feed. A mother with beautiful, healthy two-year-old triplets said, ‘‘Their father rejected them at age of eight months. . .. He said he could not bear triplets.’’ Of the sample of 33 families, six had twins in the program. Two sets were orphaned and in the other four cases the mothers did not know their own HIV status. All relied on the Milk and Medicine program for the children’s survival. Several HIV+ mothers described the dilemma of breastfeeding and HIV status. One mother sought to avoid breastfeeding of her second child and sorrowfully described the tragedy of her first child: We [husband and wife] tested positive. The girl also got it through breast feeding. I did not know that I had a problem. When I learned that I was positive and the girl too, I continued breast feeding because she was already positive too. 2.5.3. Unmet needs Although a number of unmet needs were expressed, almost all families indicated the need for more food. This was of particular concern to M&M since its goal is child nutrition. Mealie meal for preparing nshima, the staple of the Zambian diet, was not included in the program food supplement in the early years. As the children grew older, this core part of their diet was the primary food that families were struggling to purchase. The first grandmother interviewed in the research was caring for two orphaned granddaughters. AIDS had decimated her family and left her with no support system. She described herself as in complete poverty, in poor health, and with no means of income. She had depended on the program for 3 years, since her youngest granddaughter, who is HIV+, entered it when she was 2 weeks old and was orphaned by AIDS. This grandmother set the tone for the request that was repeated by almost all participants, saying, ‘‘I would like if at all they can be supplying mealie meal for us. . .. We are so grateful, only we want them to be supplying us with mealie meal.’’ In many families such as this one, other children shared the food that was intended by the program to meet the nutritional needs of one child. One mother of twins cares for eight young children; six are children of her three sisters. As she clearly stated, ‘‘In African tradition they are brothers and sisters.’’ She commented of the shared community responsibility to children, ‘‘We depend on others.’’ Another mother of twins, who were born prematurely, explained, ‘‘I share with the neighbors. . . [but] I give away very little.’’ Families also described the inability to buy clothes, blankets and other necessities, and to pay school fees. A woman who took in twin infants abandoned by their parents in a neighboring house, and who had eight children of her own said, ‘‘The twins have no clothes and have no blankets. They sleep on the floor now, but with this winter season I don’t know what to do.’’ In spite of her own family’s poverty, she said, ‘‘I could not just let the children die’’ when she found them alone in the empty house. Although the children in the program were under school age, the inability to pay school costs is a problem that will impact these children as they grow older. One elderly caregiver in poor health clearly described the dilemma: ‘‘Can they help me educate this child or not? that is what I am thinking – school – because school, even if I die and she gets educated, she can keep herself alive and maybe even help others.’’ Employment opportunities or financial capital to establish a small business in order to meet the children’s needs were often discussed; for example, one caregiver mentioned needing a sewing machine for her husband who had skill as a tailor. A grandmother described what she would like in order to make her family more secure. My request to this program is I am asking for capital so that I can be selling in the market. I am hard working in business but I

53

have no money to keep going. If I am helped with capital to start selling something, I can help the children with food and clothes. Another mother caring for a large extended family described this need for capital with a clear plan. When asked what she would like to see added to the program she responded: To be helped as guardian with loans because children need a lot; they need to go to school, to be clothed, other things. If given a loan I would join my widowed sister in the rice business because I could not go wrong. We could be given terms of reference in terms of how we could pay back, because there are many people in need of that service. 2.5.4. Milk and Medicine program All of the participants were thankful for the work of staff at CACZ. A widowed grandmother who cared for six children, one of whom had temporarily been placed in the premature crisis nursery, expressed the thoughts of many caregivers: ‘‘It has been so wonderful working with them [CACZ staff]; they are good people. It has been wonderful to be part of the program.’’ None of the participants expressed a complaint about any staff or interactions, even though interviewers assured participants that comments would be confidential and that their thoughts about problems would be used to improve the program. In spite of the continued needs because of poverty, interviewees clearly thought the program was important for the survival of their children. A mother of twins whose husband died when she was two months pregnant and who had no breast milk said, ‘‘I had no idea how I was going to take care of my children. . .. They were so small that I was even afraid to look at them. I just used to cry.’’ The following quotation is a sample of the expressions of desperation and appreciation that families felt: I started coming here after my husband died and I had no food. I had no money to buy milk for my daughter. . .. I have seen my daughter come back to life since I thought she would die. I was scared; you know what it is when you see your partner dying, and I thought she would soon follow her father. But thank God my daughter is alive. 2.6. Qualitative limitations One area of qualitative limitations is the question of program influence on the participants. All caregivers were currently receiving services from M&M and had been invited by the social workers at the program to participate. Bias in the selection of satisfied families as well as the families’ desire to remain in the program might have influenced responses. In order to mitigate this impact, outside interviewers from the local community were used and families were assured that their individual comments would be collected and used in the aggregate. Local interviewers translated four languages which was both a strength and limitation of the qualitative study. The interviews were audio recorded each morning and the interviews were translated and transcripts were typed in English each afternoon. This process clearly lost something in translation since the typed interviews rarely reflected the length and intensity of the interviews as they were taking place. 3. Quantitative evaluation 3.1. Logistics Records for 2004–2008 were noted on paper in distribution journals that one US researcher examined as part of the initial

54

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

program evaluation. The retrospective records review helped the research to determine recommendations for improved data management protocol. Statistics on infant birth date, gender, monthly weight, and age were inconsistently gathered beginning in 2004. Initially, collection of accurate and reliable data was inhibited by lack of staff capacity. Often the family caregiver and the distribution social worker were under pressure from waiting crowds, accompanying children, and families’ needs to quickly share health concerns. Caregivers and infants waited in line for food distribution in varying sites and weather conditions. Primitive scales usually hung from tree branches and children were dressed according to the weather. 3.2. Records Approximately half (n = 16) of the interviewees were represented in both data sets. The three-year longitudinal data set included records for 142 children through 2008. The reasons for admission were given for 116 children. Of these, 53 children were referred because one or both parents had HIV/AIDS. Thirty-five children (30%) were orphaned with 15 single orphans and 20 double orphans. Twenty-eight children (24%) were in fragile health. Typically, the caretaker/guardian was a single parent, one or both grandparents, an aunt or other family member or a neighbor. 3.3. Quantitative findings To determine whether children are healthy based on weight, the Gomez classification (used in this initial program evaluation) uses weight-for-age and compares a child’s weight to that of a normal child at the 50th percentile. By this classification children who are mildly malnourished are between 75 and 90% of weightfor-age those moderately malnourished are between 60 and 75% of weight-for-age and children under 60% of weight to age are considered severely malnourished (Gomez, Galvan, & Cravioto, Frenk, 1955). The retrospective examination of the existing data (2004–2008) recorded weights in kilograms (2.2 pounds equals 1 kg). There were a total of 142 children in the initial data set, 60 boys (42%) and 82 girls (58%). The data revealed that children were admitted to M&M at the average age of six months and the attendance rate was 47%, approximately 6 times per year. The weight at admission was recorded for 87 children and the average weight at admission for boys was 5.3 kg (n = 32) compared to girls at 4.2 kg (n = 55). Only 74 of the children had birth dates, distribution dates, and monthly recorded weights. These children could be compared to the Zambian Health Ministry’s normal growth charts and of these sixty-four children (88%) were below growth chart standards. While the data showed children gained weight over time in the program, their average monthly weights continued to be below median weights (50th percentile) by normal growth chart standards. 3.4. Quantitative limitations The longitudinal data limitations included variations in recorded information at the distribution sites. In some cases, if there was no change in weight from the previous month, the social worker did not record information. Birth dates were on some records, but not all. Overall, the data was not uniform and dates were inconsistently recorded. In some cases children’s data was discontinued and it was unknown if the cause was that the child’s health improved, the family’s circumstances changed, or the child died. In the years between 2004 and 2008 improvements in record keeping included the inclusion of birth dates, attendance rates,

monthly weights, and age in months, and the dates of distribution, but the records needed to be more consistent and computerized to use for evaluation purposes. 4. Recommendations 4.1. Recommendations for future program evaluation The recommended changes made by researchers included keeping more accurate and consistent quantitative data collection, use of computer software, and additions to data that included birth dates, distribution dates, and monthly weights of children. Comments regarding current health status, disabilities, and family caregiver’s concerns, and reason for discharge reasons that included mortality rates were recommended for future reports. New scales provided enabled staff to both record weights and length measurements of the children. The addition of the measure of length provided potential to use World Health Organization standards for measuring underweight, malnourished, or stunted growth in future evaluations. 4.2. Recommendations for program improvement The strength of the program, at the time evaluated, appeared to be the provision of powdered formula for infants who might otherwise have died or entered orphanage care. The program was advised to give families the appropriate amount of formula based on prescription to support a healthier start and lay the foundation for adequate physical, mental, and emotional growth. A recommendation to provide for supplemental food for caregiver’s families included mealie meal for nshima, rice, and cooking oil. The addition of a nutritional vitamins and minerals was also recommended. As a related consideration, the program needed to employ more than one social worker. Social worker’s professional skills are considered critical for program monitoring, support services, crisis intervention, and networking. Community-based feeding programs have limits to their capacity to maintain families indefinitely, thus the role of the social workers hired following the study was also to develop plans and resources to mitigate the difficulty of reaching the end of the program benefits. Recommendations were made following this evaluation that led to modifications to enable grant funding and enhanced on-going analysis. 5. Future considerations 5.1. Program focus Recommendations continue to be incorporated in an on-going investigation and researchers expect to see continued improvements in the program’s management of data. The goal of the M&M program evaluation and the implementation of the research recommendations should lead to improvements in weight-to-age and length-to age ratios as compared with normal growth charts. Nevertheless, it should be recognized that the children who participate in M&M are severely malnourished and living in deep poverty, and that some factors cannot be controlled for in a community-based program. While the youngest and most vulnerable children continue to receive highest priority, the increased support for caregiver’s families and the long term commitment this program makes to vulnerable children is noteworthy and distinguishes it from many supplemental feeding programs. In spite of the disappointment in the initial evaluation of unclear quantitative data, the interview and focus group data

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

supported the finding that the program was vital to the survival of the infants receiving aid. Although several recommendations to improve the program’s effectiveness came out of this research, according to the caregivers, many, if not all, of the children on the program would not have lived if the assistance had not been available. The Milk and Medicine program is sustained through international non-governmental organizational partnerships outside more traditional US government. Nongovernmental programs like M&M, if improved through increased professionalization and accountability, could be replicated globally. M&M as a model program has potential to keep families together and to save the lives of millions of HIV/AIDS orphaned children. 6. Conclusion Infant malnutrition, poor maternal health, and HIV/AIDS have been closely linked and had devastating consequences in subSaharan Africa. This paper reports on the early years of an emerging program. This evaluation of the M&M program in Zambia gives insight into the value of assistance to children affected by the issue. However, experiences also highlight the difficulty of conducting quantitative evaluation with data gathered over several years in multiple community sites by staff whose focus was on the survival of infants rather than on the evaluation outcomes of a program. Additional challenges were faced while conducting qualitative evaluation with families who spoke one of four different languages and who were currently benefiting from the program being assessed. 6.1. Lessons learned The success of the qualitative plan of this research was largely based on the buy-in from local program staff. The evaluation was proposed by one of the US researchers during a preliminary visit more than a year before the on-site evaluation. Agency staff persons were engaged by e-mail to design the on-site interview plans, hire local interviewers, and recruit participant families. The logistics including transportation, the timing and pace of interviews, babysitting for children brought to the interviews, and a morning meal for the families were considered by the staff, and the researchers followed the recommendations of staff in all aspects. Staff and families were comfortable with the interviews translated by local professionals and in any of the local languages that were most appropriate for the caregiver. Additionally, the US researchers came as volunteers and all funds from the research grant that were not used for expenses were donated to the program. On-site training of the agency staff as well as the local interviewers before research began assured that everyone knew the purposes and methods of the evaluation. The researchers lived on-site during the two weeks of the project and also brought with them two other professional volunteers who helped with other aspects of agency program training during our stay. Overall, the mutual respect between the evaluation group and the staff, and the overall concern of everyone for the well-being of the children and families, combined to make a hectic and sometimes stressful two weeks a heartwarming experience for all concerned. The disappointing aspect of this initial program evaluation was the difficulty in using quantitative data which had been collected over a period of several years. Because data collection was a minor component of the distribution of food, data flaws were pervasive. These data problems emphasize the complexity of monitoring community based feeding programs, or any community based intervention program in which survival in dire circumstances must take precedence over collection of research data. After the initial program evaluation, grant funding was obtained in fall 2009 to proceed with the program evaluation through on-going collection

55

of thorough longitudinal data and by funding M&M program improvements that were recommended by the researchers. Staff training and improved measurement tools are expected to bring a clearer statistical look, over time, at the impact of this program on infants’ health and development. Important measurements of infant survival, program cost effectiveness, and psychological benefits of family care over institutional placement remain challenges for evaluation of community-based work. Large sums of international aid have been spent to reach UN Millennium Goals in many geographic regions and in all areas of concern for human well-being. Finding ways of measuring outcomes in light of extreme poverty, intervening variables in natural settings, and cultural and political variations will remain a challenge to both region-wide and local intervention programs. Appendix A Participant information

Interview #__________

Milk and Medicine Program Lusaka, Zambia Interview face sheet dataChild nameOrphan statusAgeRelationship to guardianDate of BirthFamily compositionSexOther programs/servicesDate of admissionReason for admissionWeight at admissionGuardian statusCurrent weightSupplements Interview questions 1. Tell me about your family. Probes: Who lives here? How are they related to the child who received the Milk and Medicine Program? How is the child who received the Milk and Medicine Program doing now? Is he/she still living with you? How old is he/she now? Is he/she attending school? What is his/her health? Does he/she have any special problems? What does he/she like to do? 2. How has HIV/AIDS affected your family? 3. Tell me about your family’s involvement with the Milk and Medicine Program? Probes: When were you involved with the program? How many children received services? 4. What were the most important things about the Milk and Medicine Program? (helped most) 5. What would you have liked to see changed about the program? (more helpful) 6. What was your relationship with the staff at House of Moses? 7. Is there anything else you would like to tell me about your participation in the program? References Becquet, R., Bequet, L., Ekouevi, D. K., Viho, I., Sakarovitch, C., Fassinou, P., et al. (2007). Two-year morbidity-mortality and alternative to prolonged breast-feeding among children born to HIV-infected mothers in Coˆd’Ivoire. PLOS Medicine, 4(1), 139–151. Boyatzis, R. E. (1998). Transforming qualitative information. Thousand Oaks, CA: Sage. Brinkman, H. J., de Pee, S., Sanogo, I., Subran, L., & Bloem, M. W. (2010). High food prices and the global financial crisis have reduced access to nutritious food and worsened nutritional status and health. Journal of Nutrition, 140(1), 61S–153S. Bukusuba, J., Kikafunda, J. K., & Whitehead, R. G. (2009). Nutrition status of children (6– 59 months) among HIV-positive mothers/caregivers living in an urban setting of Uganda’. African Journal of Food, Agriculture, Nutrition and Development, 9(6), 1346– 1364. Central Statistical Office Zambia (n.d.). Living conditions. Available online at: http:// www.zamstats.gov.zm/lcm.php#07. Accessed 25.05.10. Chandang’oma, K. P., Chabwela, P., & Banda, R. (2010). Zambian Country Report: Monitoring the declaration of commitment on HIV and AIDS and the Universal Access Biennial Report. Lusaka, Zambia: National HIV/AIDS/STI/TB Council. Chopra, M., Daviaud, E., Pattinson, R., Fonn, S., & Lawn, J. E. (2009). Health in South Africa 2: Saving the lives of South Africa’s mothers, babies, and children: Can the health system deliver. Lancet, 374(9692), 835–846. Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed method approach (3rd ed.). Thousand Oaks, CA: Sage Publications.

56

D. Sturtevant, J.S. Wimmer / Evaluation and Program Planning 42 (2014) 50–56

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2010). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, NJ: Prentice Hall. Glaser, B. G., & Strauss, A. L. (1967). Discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine. Gomez, F., Galvan, R. R., Cravioto, J., & Frenk, S. (1955). Malnutrition in infancy and childhood with special reference to Kwashiorkor. Advanced Pediatrics, 7, 131–169. Lombe, M., & Ochumbo, A. (2008). Sub-Saharan Africa’s orphans. International Social Work, 51(5), 682–698. Magezi, S. R., Kikafunda, J. K., & Whitehead, R. G. (2008). Feeding and nutritional characteristics of infants on PMTCT programs. Journal of Tropical Pediatrics, 55(1), 32–33. Merraim, S. B., & Associates, (2002). Qualitative research in practice: Examples for discussion and analysis. San Francisco, CA: Jossey-Bass. Naidoo, R., Rennert, W., Lung, A., Naidoo, K., & McKerrow, N. (2010). The influence of nutritional status on the response to HAART in HIV-infected children in South Africa. Pediatric Infectious Disease Journal, 29(6), 511–513. Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage. Peterson, K. (2009). Childhood undernutrition: A failing global priority. Journal of Public Health Policy, 30(4), 455–465. Sztam, K. A., Fawzi, W. W., & Duggan, C. (2010). Macronutrient supplementation and food prices in HIV treatment. Journal of Nutrition, 140(1), 23S–213S.

UNAIDS. (2008, 2009). Report on the global HIV/AIDS epidemic 2008. Geneva: Author. UNICEF (n.d.). WHO, UNAIDS, UNICEF, infant feeding guidelines. Geneva: Author. Available online at: http://www.unicef.org/nutrition/index_24811.html. Accessed 17.06.10. United Nations. (2012). The millennium development goals report 2012. New York: Author. World Health Organization, Guidelines on HIV and infant feeding 2010: Principles and recommendations for infant feeding in the context of HIV and a summary of evidence. Available online at: http://whqlibdoc.who.int/publications/2010/ 9789241599535_eng.pdf. Accessed 16.10.13.

Dr. Sturtevant received an MSW from Western Michigan University in 1984 and a Ph.D. from Michigan State University in 1997. Her areas of interest are child welfare, social policy, and nonprofit organizations. Dr. Sturtevant has been a social work professor at Hope College since 1988 and department chair since 2002.

Dr. Wimmer received an MSW from Rutgers University in 1969 and a Ph.D. from the University of Georgia in 2006. She pursued a career in US and international child welfare for many years. Dr. Wimmer has been a social work professor at Dalton State College since 2006.