Commentary
Data Collection for Monitoring Hepatitis Testing Programs: The HepTLC Data Management System
Andrew J. Hebert, MPHa Tamara L. Lamia, MPHa Ben T. Schoenbachler, MPHb Ayasha K. Richardson, MPHa
Viral hepatitis continues to be an important health disparity but an “underappreciated health concern,” as described in a 2010 Institute of Medicine report.1 The 2011 launch of the U.S. Department of Health and Human Services’ national action plan2 to combat viral hepatitis and the 2012 Centers for Disease Control and Prevention’s (CDC’s) Hepatitis Testing and Linkage to Care (HepTLC) initiative set the stage for a nationally coordinated effort to address viral hepatitis. The HepTLC initiative directly funded 34 U.S. organizations in a variety of settings to increase early identification, linkage to care, and treatment for people with undiagnosed chronic hepatitis C virus (HCV) and/or hepatitis B virus (HBV) infection, with a focus on disproportionately affected populations. CDC’s Division of Viral Hepatitis (DVH) required awardees to collect and report data on their testing activities and results to monitor implementation of the HepTLC initiative and track testing goals. DVH wanted to be able to track variables that would reflect the performance of the awardees’ activities for the two separate conditions, undiagnosed chronic HCV infection and undiagnosed chronic HBV infection. The tracking system needed to robustly, and in near real time: (1) capture data to monitor the project’s objectives, along with standard checks to ensure data quality and (2) support analysis to determine effective strategies for identifying undiagnosed cases. In addition, the system had to have the flexibility to accommodate the many awardee settings in which testing took place (e.g., HCV testing of people who inject drugs, HCV testing in community health centers, and HBV testing). CDC DVH, in collaboration with our team, created a customized version of Luther Consulting’s Internet-based data management and reporting system, EvaluationWeb®,3 to enable HepTLC awardees to comply with their reporting requirements. CDC’s Division of HIV/AIDS Prevention has used EvaluationWeb since 2010 to collect and house human immunodeficiency virus (HIV) testing and prevention intervention data. In this commentary, we, the designers of the new system, describe our steps to create and maintain this data management and reporting system.
ICF International, Inc., Atlanta, GA
a
Oak Ridge Institute for Science and Education, Oak Ridge, TN
b
Address correspondence to: Andrew J. Hebert, MPH, ICF International, Inc., 3 Corporate Blvd., Ste. 370, Atlanta, GA 30329; tel. 404-592-2285; fax 404-321-3688; e-mail . ©2016 Association of Schools and Programs of Public Health
Public Health Reports / 2016 Supplement 2 / Volume 131
41
42 Commentary
Our work comprised four main steps: 1. Design of the dataset and its variables. In starting this task, we aimed to create a dataset design that made the data maximally usable, while also minimizing the reporting burden for awardees. The data collected had to be useful for monitoring awardees’ performance based on the H epTLC cooperative agreements, and also to allow queries to address key evaluation questions. The selected variables included information about (1) the test sites that provided HepTLC services and the laboratories that provided the testing; (2) testing participants, including demographics, risk characteristics, and vaccination history; (3) diagnostic test results; and (4) posttest follow-up, including result notification, posttest counseling, referral to care, first medical appointment attendance, and surveillance reporting. 2. Setup of an Internet-based data reporting system. One of the requirements of the system was that it had to be user friendly for data entry staff members working in settings with limited resources and time. To meet this need, we modified an existing Internet-based data reporting system, EvaluationWeb, that allowed awardees to enter data either by direct data entry or by uploading a Microsoft® Excel® (.xlsx) or comma-delimited (.csv) flat file. To increase the speed by which awardees could enter data, we prepopulated certain data fields based on the user’s identity. To reduce data entry errors, we programmed data entry rules into the system. For either direct entry or uploaded files, the system automatically notified users of incorrect or incomplete entries at the time of data entry. 3. Establishment of a data review process. We created a data review process that was used by specialized data managers to review each awardee’s data for missing data or data anomalies. The findings from this process were summarized in a monthly data discrepancy report that was customized for each HepTLC awardee. The data manager would refer to this report in a monthly meeting with individual awardees to review and discuss any data entry issues and track awardees’ progression to their testing goals. These reports were also shared with the CDC DVH team, which tracked programmatic issues, followed up with individual awardees when necessary, and provided guidance based on awardee feedback. Additionally, awardees
were given continuous access to the data system so they could correct data issues, update testing data, and use built-in reporting features to track their testing progress in real time. 4. Creation of training and technical support functions. We offered technical assistance during special telephone conference calls, through annual trainings, and on an ad-hoc basis to individual awardees. At the beginning of the project, our team conducted trainings to provide instruction on the HepTLC reporting processes and tailored guidance on how to integrate data collection into awardees’ program testing activities. In the middle of the project, additional refresher trainings were held covering system updates, common data collection challenges, and peer-to-peer data sharing to report challenges and solutions by awardees. LESSONS LEARNED The HepTLC data management system ensured data validity and enabled CDC DVH to conduct special analyses during the course of the project, such as investigating the thoroughness of the linkage-to-care continuum. The system and data management processes provided awardees with data displaying their progress toward testing goals and the extent of missing data. HepTLC awardees had the reports and data needed to improve their program activities immediately, rather than months or years into the project. HepTLC awardees could immediately identify and address their issues with the help of data managers before they became larger, compounded problems. By adapting existing Internet-based software to fit the needs of the HepTLC initiative, our team, working with CDC DVH, updated the system in response to changing program needs. Each component of the HepTLC data management and reporting process worked in tandem to support awardees and ensure data quality throughout the project. The HepTLC initiative can serve as a model for other awardee programs that are collecting and analyzing test-level data for HCV, HBV, HIV, or any disease with a screening and/or continuumof-care component. Combining real-time data access with strong processes and training resources enabled awardees to ensure data quality and completeness and improve program management. All aspects of this project were considered part of a public health program and were determined to be exempt from institutional review board review.
Public Health Reports / 2016 Supplement 2 / Volume 131
Ensuring Data Quality in HepTLC Program Activities 43
REFERENCES 1. National Research Council (US). Hepatitis and liver cancer: a national strategy for prevention and control of hepatitis B and C. Washington: National Academies Press; 2010.
2. Department of Health and Human Services (US). Combating the silent epidemic of viral hepatitis: action plan for the prevention, care and treatment of viral hepatitis. Washington: HHS; 2011. 3. Luther Consulting, LLC. EvaluationWeb®: Version 5. Carmel (IN): Luther Consulting, LLC; 2013.
Public Health Reports / 2016 Supplement 2 / Volume 131
Data Collection for Monitoring Hepatitis Testing Programs: The HepTLC Data Management System
Andrew J. Hebert, MPHa Tamara L. Lamia, MPHa Ben T. Schoenbachler, MPHb Ayasha K. Richardson, MPHa
Viral hepatitis continues to be an important health disparity but an “underappreciated health concern,” as described in a 2010 Institute of Medicine report.1 The 2011 launch of the U.S. Department of Health and Human Services’ national action plan2 to combat viral hepatitis and the 2012 Centers for Disease Control and Prevention’s (CDC’s) Hepatitis Testing and Linkage to Care (HepTLC) initiative set the stage for a nationally coordinated effort to address viral hepatitis. The HepTLC initiative directly funded 34 U.S. organizations in a variety of settings to increase early identification, linkage to care, and treatment for people with undiagnosed chronic hepatitis C virus (HCV) and/or hepatitis B virus (HBV) infection, with a focus on disproportionately affected populations. CDC’s Division of Viral Hepatitis (DVH) required awardees to collect and report data on their testing activities and results to monitor implementation of the HepTLC initiative and track testing goals. DVH wanted to be able to track variables that would reflect the performance of the awardees’ activities for the two separate conditions, undiagnosed chronic HCV infection and undiagnosed chronic HBV infection. The tracking system needed to robustly, and in near real time: (1) capture data to monitor the project’s objectives, along with standard checks to ensure data quality and (2) support analysis to determine effective strategies for identifying undiagnosed cases. In addition, the system had to have the flexibility to accommodate the many awardee settings in which testing took place (e.g., HCV testing of people who inject drugs, HCV testing in community health centers, and HBV testing). CDC DVH, in collaboration with our team, created a customized version of Luther Consulting’s Internet-based data management and reporting system, EvaluationWeb®,3 to enable HepTLC awardees to comply with their reporting requirements. CDC’s Division of HIV/AIDS Prevention has used EvaluationWeb since 2010 to collect and house human immunodeficiency virus (HIV) testing and prevention intervention data. In this commentary, we, the designers of the new system, describe our steps to create and maintain this data management and reporting system.
ICF International, Inc., Atlanta, GA
a
Oak Ridge Institute for Science and Education, Oak Ridge, TN
b
Address correspondence to: Andrew J. Hebert, MPH, ICF International, Inc., 3 Corporate Blvd., Ste. 370, Atlanta, GA 30329; tel. 404-592-2285; fax 404-321-3688; e-mail . ©2016 Association of Schools and Programs of Public Health
Public Health Reports / 2016 Supplement 2 / Volume 131
41
42 Commentary
Our work comprised four main steps: 1. Design of the dataset and its variables. In starting this task, we aimed to create a dataset design that made the data maximally usable, while also minimizing the reporting burden for awardees. The data collected had to be useful for monitoring awardees’ performance based on the H epTLC cooperative agreements, and also to allow queries to address key evaluation questions. The selected variables included information about (1) the test sites that provided HepTLC services and the laboratories that provided the testing; (2) testing participants, including demographics, risk characteristics, and vaccination history; (3) diagnostic test results; and (4) posttest follow-up, including result notification, posttest counseling, referral to care, first medical appointment attendance, and surveillance reporting. 2. Setup of an Internet-based data reporting system. One of the requirements of the system was that it had to be user friendly for data entry staff members working in settings with limited resources and time. To meet this need, we modified an existing Internet-based data reporting system, EvaluationWeb, that allowed awardees to enter data either by direct data entry or by uploading a Microsoft® Excel® (.xlsx) or comma-delimited (.csv) flat file. To increase the speed by which awardees could enter data, we prepopulated certain data fields based on the user’s identity. To reduce data entry errors, we programmed data entry rules into the system. For either direct entry or uploaded files, the system automatically notified users of incorrect or incomplete entries at the time of data entry. 3. Establishment of a data review process. We created a data review process that was used by specialized data managers to review each awardee’s data for missing data or data anomalies. The findings from this process were summarized in a monthly data discrepancy report that was customized for each HepTLC awardee. The data manager would refer to this report in a monthly meeting with individual awardees to review and discuss any data entry issues and track awardees’ progression to their testing goals. These reports were also shared with the CDC DVH team, which tracked programmatic issues, followed up with individual awardees when necessary, and provided guidance based on awardee feedback. Additionally, awardees
were given continuous access to the data system so they could correct data issues, update testing data, and use built-in reporting features to track their testing progress in real time. 4. Creation of training and technical support functions. We offered technical assistance during special telephone conference calls, through annual trainings, and on an ad-hoc basis to individual awardees. At the beginning of the project, our team conducted trainings to provide instruction on the HepTLC reporting processes and tailored guidance on how to integrate data collection into awardees’ program testing activities. In the middle of the project, additional refresher trainings were held covering system updates, common data collection challenges, and peer-to-peer data sharing to report challenges and solutions by awardees. LESSONS LEARNED The HepTLC data management system ensured data validity and enabled CDC DVH to conduct special analyses during the course of the project, such as investigating the thoroughness of the linkage-to-care continuum. The system and data management processes provided awardees with data displaying their progress toward testing goals and the extent of missing data. HepTLC awardees had the reports and data needed to improve their program activities immediately, rather than months or years into the project. HepTLC awardees could immediately identify and address their issues with the help of data managers before they became larger, compounded problems. By adapting existing Internet-based software to fit the needs of the HepTLC initiative, our team, working with CDC DVH, updated the system in response to changing program needs. Each component of the HepTLC data management and reporting process worked in tandem to support awardees and ensure data quality throughout the project. The HepTLC initiative can serve as a model for other awardee programs that are collecting and analyzing test-level data for HCV, HBV, HIV, or any disease with a screening and/or continuumof-care component. Combining real-time data access with strong processes and training resources enabled awardees to ensure data quality and completeness and improve program management. All aspects of this project were considered part of a public health program and were determined to be exempt from institutional review board review.
Public Health Reports / 2016 Supplement 2 / Volume 131
Ensuring Data Quality in HepTLC Program Activities 43
REFERENCES 1. National Research Council (US). Hepatitis and liver cancer: a national strategy for prevention and control of hepatitis B and C. Washington: National Academies Press; 2010.
2. Department of Health and Human Services (US). Combating the silent epidemic of viral hepatitis: action plan for the prevention, care and treatment of viral hepatitis. Washington: HHS; 2011. 3. Luther Consulting, LLC. EvaluationWeb®: Version 5. Carmel (IN): Luther Consulting, LLC; 2013.
Public Health Reports / 2016 Supplement 2 / Volume 131
