AIDS Care, 2014 Vol. 26, No. 12, 1546–1549, http://dx.doi.org/10.1080/09540121.2014.936812
Monitoring retention in care: using multiple laboratory tests as an indicator for HIV medical care Caislin Leah Firtha*, Sean David Schafera and Käri Greeneb a
Oregon Public Health Division, HIV/STD/TB Program, Portland, OR, USA; bOregon Public Health Division, Program Design & Evaluation Services, Portland, OR, USA (Received 18 October 2013; accepted 11 June 2014) Retention in care is an important strategy for HIV prevention. Unfortunately, surveillance systems were not designed to capture face-to-face visits with HIV health care providers to assess retention in care. Instead, HIV-related laboratory tests are used as a surrogate measure. This study estimated the sensitivity (90%) and specificity (28%) of two HIV-related laboratory tests separated by at least 90 days for two face-to-face visits among people receiving HIV-related health care in Oregon. Overall accuracy of the surrogate was good but slightly overestimated the proportion of people living with HIV/AIDS actually retained in care.
Keywords: HIV surveillance; retention in care; CD4 lymphocyte count; viral load
Introduction Periodic health care visits by people living with HIV (PLWH) to health care providers with expertise in HIV treatment (“HIV-proficient provider”) reduce morbidity and new infections (Giordano et al., 2007; Mugavero, Amico, et al., 2012; Park et al., 2007). The logic is straightforward: (1) regular visits generally lead to antiretroviral treatment; (2) treatment leads to viral suppression; and (3) people with suppressed viral loads rarely transmit HIV infection (Mugavero, Amico, et al., 2012). Consequently, the National HIV/AIDS Strategy (NHAS), the US Health Resources and Services Administration (HRSA), and the Institute of Medicine (IOM) identify “retention in care” as an important strategy and a benchmark for HIV prevention (The White House Office of National AIDS Policy [WHONAP], 2010). No consensus gold standard exists on operationalizing retention in care or on a minimum frequency of visits. NHAS, HRSA, and IOM define retention in care as a minimum number of visits (typically two) with an HIV-proficient provider within a defined period (typically 12 months), where a minimum interval (typically, 90 days) has elapsed between at least two visits (Horberg et al., 2010; Ikard et al., 2005; Institute of Medicine [IOM], 2012; WHONAP, 2010). However, most public health agencies cannot directly measure the number and timing of health care visits. HIV surveillance systems were not designed to capture visits, leaving public health systems reliant on surrogate measures for visits, often a laboratory report of either a viral load or CD4-positive T-lymphocyte (CD4) count. *Corresponding author. Email: [email protected] © 2014 Taylor & Francis
Recently, defining “retention in care” as at least two visits with an HIV-proficient provider separated by at least 90 days and CD4 counts or viral load tests as surrogates for visits, the Centers for Disease Control and Prevention estimated that 45% of people in the USA with known HIV infection were retained in care during 2009 (Hall et al., 2012). Other estimates diverge widely (Marks, Gardner, Craw, & Crepaz, 2010; Yehia et al., 2012; Mugavero, Westfall et al., 2012). This study is aimed at assessing the accuracy of two HIV laboratory tests (CD4 count or viral load or both) at least 90 days apart as a surrogate measure for retention in care.
Methods Study population The Medical Monitoring Project (MMP) represents repeat annual cross-sections of PLWH receiving care in the USA. MMP is observational and does not include interventions or additional health care visits. Methods are explained in detail elsewhere (National Center for HIV/ AIDS, Viral Hepatitis, STD, and TB Prevention Center for Disease Control and Prevention, 2009). Seventeen US states and six cities with independent health departments currently participate. Study data were collected from a representative sample of 280 PLWH who received HIV-related outpatient medical care from an HIV-proficient provider during January 2010 through April 2010 in Oregon. Two-stage sampling begins with enumeration of HIV providers in the state who are then sampled randomly. Probability of provider sampling varies directly with the number of HIV patients in care.
AIDS Care Next, patients seen during January–April of each year are enumerated and sampled from selected practices. Retention in care definitions We defined a face-to-face visit as an in-person visit with an HIV-proficient provider. Health care encounters such as telephone, pharmacy, or nursing visits were not included in the count of face-to-face visits. Retention in care was defined as having at least two face-to-face clinic visits separated by at least 90 days and multiple laboratory tests as ≥2 HIV-related laboratory tests (CD4 count or viral load) separated by at least 90 days within the 12 months preceding the MMP interview (surveillance period). Data collection We conducted interviews in MMP offices, clinics, and participant homes. Two experienced abstractors reviewed medical charts to count face-to-face visits with an HIVproficient provider and recorded the dates of all HIV laboratory tests (CD4 counts and viral loads) during the surveillance period. They reported no difficulty determining whether an encounter comprised a face-toface visit. Statistical analysis We excluded 13 duplicate HIV laboratory test reports for 10 participants. We used Fisher’s exact tests to identify differences in behavioral and sociodemographic characteristics among participants’ groups that met and did not meet the retention in care definition. We used t-tests to compare numbers of HIV laboratory tests and face-toface health care visits. Normality was assumed, and we reported Pearson correlation. We calculated sensitivity and specificity, and positive and negative predictive values of multiple laboratory tests for retention in care and reported 95% Wald confidence intervals (CI) for each. All statistical analyses were conducted using SAS version 9.3.
Results Participant characteristics Participant characteristics were consistent with HIV case reports in Oregon (Table 1). Of the 280 participants, most were male (87%). Mean age was 47 years. Most (75%) participants reported race as white and resided in urban locales (75%). Nearly one-third (29%) lived in poverty, and one-third had an AIDS diagnosis (34%). We counted 880 HIV laboratory tests during the surveillance period. Participants had a mean of 3.0 CD4 counts and 3.1 viral loads and a mean sum of 3.2 HIV-related
1547
laboratory tests. All but five had ≥1 CD4 count and 271 had ≥1 viral load. Ninety-nine percent (276/280) had ≥1 HIV laboratory test. Almost all (278/280) had at least one face-to-face visit (mean 5.0) during the surveillance period. Association between multiple laboratory tests and retention in care We observed an association between laboratory tests and face-to-face visits (r = 0.41 p < 0.01). Sensitivity for retention in care of multiple laboratory tests was 90% (95% CI: 87–94%); specificity was 28% (95% CI: 14– 41%; Table 2). Positive predictive value was 88% (95% CI: 84–92%); negative predictive value was 32% (95% CI: 17–48%). Of 52 participants who were misclassified, 23 did not have multiple laboratory tests but met the retention in care definition, and 29 had multiple laboratory tests but did not meet the retention in care definition. None of the behavioral and sociodemographic characteristics varied significantly among the participants who met, and participants who did not meet, the retention in care definition (p > 0.05; Table 1). Discussion If face-to-face visits represent the essential element of medical care leading to sustained viral load suppression, then our findings suggest that at least two HIV-related laboratory tests separated by 90 days are a relatively good surrogate for estimating retention in care. In part, multiple tests accurately represent face-toface visits in this population precisely because the completion of at least one visit early in the year was a requirement of sampling. This is a substantial limitation. One could argue that these data provide support for the conclusion that two laboratory test results separated by at least 90 days predict a second visit in a year if the first is given. The predictive value positive or negative, or even the estimated sensitivity and specificity of using two tests to represent two visits, might be different if the sample included people who made no visits during January through April of a given year. On the other hand, despite the first visit being a de facto condition of participation, our sample did include people who failed to make a second visit and were therefore defined as not retained in care definition. This allowed us to estimate both sensitivity and specificity. In addition, we do not believe that the correlation we observed between numbers of tests and visits is necessarily a function of the sample having been drawn from those with at least one visit. Low specificity of multiple tests suggests that if the multiple surrogate tests were applied to a population where, unlike ours, many people are not regularly
1548
C.L. Firth et al.
Table 1. Characteristics of the MMP participants, Oregon, 2010 (n = 280). Participant characteristics
Total n (%)
Gender Male Female Transgender Age group at interview (years) 18 ≤ age ≤ 34 35 ≤ age ≤ 49 50 ≤ age Racea White, non-Hispanic Black Hispanic Other Urban residence Household poverty Education Less than high school High school diploma or equivalent Some college or technical school Bachelor’s degree or higher Injection drug use within last 12 months Homelessness Incarceration Ever received AIDS diagnosis
Retained in care n (%)
244 32 4
(87) (11) (1)
206 30 4
(84) (94) (100)
0.24
37 124 119
(13) (44) (43)
34 106 100
(92) (85) (84)
0.48
209 20 25 21 209 81
(76) (7) (9) (8) (75) (29)
176 19 24 18 183 71
(84) (95) (96) (86) (88) (88)
0.27 0.17 0.58
32 66 121 61 15 15 8 96
(11) (24) (43) (22) (5) (5) (3) (34)
30 56 106 48 12 14 8 86
(94) (85) (88) (79) (80) (93) (100) (90)
0.21 0.75 0.49 0.38 0.21
Retained in care n Mean (±SD)
Mean (±SD) +
a,b
Number of CD4 counts Number of HIV viral load testsb,c Total laboratory (CD4+ count or viral load) test datesb,d Visits with an HIV-proficient providerb,e
p value*
3.0 3.1 3.2 5.0
(1.2) (1.2) (1.3) (3.4)
236 233 236 240
3.2 3.2 3.3 5.5
p-value** (1.2) (1.2) (1.3) (3.4)
Monitoring retention in care: using multiple laboratory tests as an indicator for HIV medical care Caislin Leah Firtha*, Sean David Schafera and Käri Greeneb a
Oregon Public Health Division, HIV/STD/TB Program, Portland, OR, USA; bOregon Public Health Division, Program Design & Evaluation Services, Portland, OR, USA (Received 18 October 2013; accepted 11 June 2014) Retention in care is an important strategy for HIV prevention. Unfortunately, surveillance systems were not designed to capture face-to-face visits with HIV health care providers to assess retention in care. Instead, HIV-related laboratory tests are used as a surrogate measure. This study estimated the sensitivity (90%) and specificity (28%) of two HIV-related laboratory tests separated by at least 90 days for two face-to-face visits among people receiving HIV-related health care in Oregon. Overall accuracy of the surrogate was good but slightly overestimated the proportion of people living with HIV/AIDS actually retained in care.
Keywords: HIV surveillance; retention in care; CD4 lymphocyte count; viral load
Introduction Periodic health care visits by people living with HIV (PLWH) to health care providers with expertise in HIV treatment (“HIV-proficient provider”) reduce morbidity and new infections (Giordano et al., 2007; Mugavero, Amico, et al., 2012; Park et al., 2007). The logic is straightforward: (1) regular visits generally lead to antiretroviral treatment; (2) treatment leads to viral suppression; and (3) people with suppressed viral loads rarely transmit HIV infection (Mugavero, Amico, et al., 2012). Consequently, the National HIV/AIDS Strategy (NHAS), the US Health Resources and Services Administration (HRSA), and the Institute of Medicine (IOM) identify “retention in care” as an important strategy and a benchmark for HIV prevention (The White House Office of National AIDS Policy [WHONAP], 2010). No consensus gold standard exists on operationalizing retention in care or on a minimum frequency of visits. NHAS, HRSA, and IOM define retention in care as a minimum number of visits (typically two) with an HIV-proficient provider within a defined period (typically 12 months), where a minimum interval (typically, 90 days) has elapsed between at least two visits (Horberg et al., 2010; Ikard et al., 2005; Institute of Medicine [IOM], 2012; WHONAP, 2010). However, most public health agencies cannot directly measure the number and timing of health care visits. HIV surveillance systems were not designed to capture visits, leaving public health systems reliant on surrogate measures for visits, often a laboratory report of either a viral load or CD4-positive T-lymphocyte (CD4) count. *Corresponding author. Email: [email protected] © 2014 Taylor & Francis
Recently, defining “retention in care” as at least two visits with an HIV-proficient provider separated by at least 90 days and CD4 counts or viral load tests as surrogates for visits, the Centers for Disease Control and Prevention estimated that 45% of people in the USA with known HIV infection were retained in care during 2009 (Hall et al., 2012). Other estimates diverge widely (Marks, Gardner, Craw, & Crepaz, 2010; Yehia et al., 2012; Mugavero, Westfall et al., 2012). This study is aimed at assessing the accuracy of two HIV laboratory tests (CD4 count or viral load or both) at least 90 days apart as a surrogate measure for retention in care.
Methods Study population The Medical Monitoring Project (MMP) represents repeat annual cross-sections of PLWH receiving care in the USA. MMP is observational and does not include interventions or additional health care visits. Methods are explained in detail elsewhere (National Center for HIV/ AIDS, Viral Hepatitis, STD, and TB Prevention Center for Disease Control and Prevention, 2009). Seventeen US states and six cities with independent health departments currently participate. Study data were collected from a representative sample of 280 PLWH who received HIV-related outpatient medical care from an HIV-proficient provider during January 2010 through April 2010 in Oregon. Two-stage sampling begins with enumeration of HIV providers in the state who are then sampled randomly. Probability of provider sampling varies directly with the number of HIV patients in care.
AIDS Care Next, patients seen during January–April of each year are enumerated and sampled from selected practices. Retention in care definitions We defined a face-to-face visit as an in-person visit with an HIV-proficient provider. Health care encounters such as telephone, pharmacy, or nursing visits were not included in the count of face-to-face visits. Retention in care was defined as having at least two face-to-face clinic visits separated by at least 90 days and multiple laboratory tests as ≥2 HIV-related laboratory tests (CD4 count or viral load) separated by at least 90 days within the 12 months preceding the MMP interview (surveillance period). Data collection We conducted interviews in MMP offices, clinics, and participant homes. Two experienced abstractors reviewed medical charts to count face-to-face visits with an HIVproficient provider and recorded the dates of all HIV laboratory tests (CD4 counts and viral loads) during the surveillance period. They reported no difficulty determining whether an encounter comprised a face-toface visit. Statistical analysis We excluded 13 duplicate HIV laboratory test reports for 10 participants. We used Fisher’s exact tests to identify differences in behavioral and sociodemographic characteristics among participants’ groups that met and did not meet the retention in care definition. We used t-tests to compare numbers of HIV laboratory tests and face-toface health care visits. Normality was assumed, and we reported Pearson correlation. We calculated sensitivity and specificity, and positive and negative predictive values of multiple laboratory tests for retention in care and reported 95% Wald confidence intervals (CI) for each. All statistical analyses were conducted using SAS version 9.3.
Results Participant characteristics Participant characteristics were consistent with HIV case reports in Oregon (Table 1). Of the 280 participants, most were male (87%). Mean age was 47 years. Most (75%) participants reported race as white and resided in urban locales (75%). Nearly one-third (29%) lived in poverty, and one-third had an AIDS diagnosis (34%). We counted 880 HIV laboratory tests during the surveillance period. Participants had a mean of 3.0 CD4 counts and 3.1 viral loads and a mean sum of 3.2 HIV-related
1547
laboratory tests. All but five had ≥1 CD4 count and 271 had ≥1 viral load. Ninety-nine percent (276/280) had ≥1 HIV laboratory test. Almost all (278/280) had at least one face-to-face visit (mean 5.0) during the surveillance period. Association between multiple laboratory tests and retention in care We observed an association between laboratory tests and face-to-face visits (r = 0.41 p < 0.01). Sensitivity for retention in care of multiple laboratory tests was 90% (95% CI: 87–94%); specificity was 28% (95% CI: 14– 41%; Table 2). Positive predictive value was 88% (95% CI: 84–92%); negative predictive value was 32% (95% CI: 17–48%). Of 52 participants who were misclassified, 23 did not have multiple laboratory tests but met the retention in care definition, and 29 had multiple laboratory tests but did not meet the retention in care definition. None of the behavioral and sociodemographic characteristics varied significantly among the participants who met, and participants who did not meet, the retention in care definition (p > 0.05; Table 1). Discussion If face-to-face visits represent the essential element of medical care leading to sustained viral load suppression, then our findings suggest that at least two HIV-related laboratory tests separated by 90 days are a relatively good surrogate for estimating retention in care. In part, multiple tests accurately represent face-toface visits in this population precisely because the completion of at least one visit early in the year was a requirement of sampling. This is a substantial limitation. One could argue that these data provide support for the conclusion that two laboratory test results separated by at least 90 days predict a second visit in a year if the first is given. The predictive value positive or negative, or even the estimated sensitivity and specificity of using two tests to represent two visits, might be different if the sample included people who made no visits during January through April of a given year. On the other hand, despite the first visit being a de facto condition of participation, our sample did include people who failed to make a second visit and were therefore defined as not retained in care definition. This allowed us to estimate both sensitivity and specificity. In addition, we do not believe that the correlation we observed between numbers of tests and visits is necessarily a function of the sample having been drawn from those with at least one visit. Low specificity of multiple tests suggests that if the multiple surrogate tests were applied to a population where, unlike ours, many people are not regularly
1548
C.L. Firth et al.
Table 1. Characteristics of the MMP participants, Oregon, 2010 (n = 280). Participant characteristics
Total n (%)
Gender Male Female Transgender Age group at interview (years) 18 ≤ age ≤ 34 35 ≤ age ≤ 49 50 ≤ age Racea White, non-Hispanic Black Hispanic Other Urban residence Household poverty Education Less than high school High school diploma or equivalent Some college or technical school Bachelor’s degree or higher Injection drug use within last 12 months Homelessness Incarceration Ever received AIDS diagnosis
Retained in care n (%)
244 32 4
(87) (11) (1)
206 30 4
(84) (94) (100)
0.24
37 124 119
(13) (44) (43)
34 106 100
(92) (85) (84)
0.48
209 20 25 21 209 81
(76) (7) (9) (8) (75) (29)
176 19 24 18 183 71
(84) (95) (96) (86) (88) (88)
0.27 0.17 0.58
32 66 121 61 15 15 8 96
(11) (24) (43) (22) (5) (5) (3) (34)
30 56 106 48 12 14 8 86
(94) (85) (88) (79) (80) (93) (100) (90)
0.21 0.75 0.49 0.38 0.21
Retained in care n Mean (±SD)
Mean (±SD) +
a,b
Number of CD4 counts Number of HIV viral load testsb,c Total laboratory (CD4+ count or viral load) test datesb,d Visits with an HIV-proficient providerb,e
p value*
3.0 3.1 3.2 5.0
(1.2) (1.2) (1.3) (3.4)
236 233 236 240
3.2 3.2 3.3 5.5
p-value** (1.2) (1.2) (1.3) (3.4)
