U.S. Department of Veterans Affairs Public Access Author manuscript Clin Gerontol. Author manuscript; available in PMC 2017 June 09. Published in final edited form as: Clin Gerontol. 2017 ; 40(2): 75–76. doi:10.1080/07317115.2016.1268864.
VA Author Manuscript
Are We Doing a Good Job? In Praise of Program Evaluation Jennifer Moye VA Boston Healthcare System and Harvard Medical School, Boston, Massachusetts, USA Fellow Clinical Gerontologists, welcome to issue 2 of 2017. Regular readers of our journal may have noticed that we are introducing a new feature. Each article must now include a “clinical implications” statement in the abstract and again in the discussion. Soon every article should clearly indicate “so what does this mean for the clinician in the field?”
VA Author Manuscript
Most of the papers in our second issue of 2017 focus on program evaluation. I am very enthusiastic about program evaluation. Why? Well, for starters—the doing of clinical work is demanding enough—it takes extra energy to go to the extra step of program evaluation. Yet, systematically evaluating our programs forces us to characterize what we are doing and to investigate the extent and limits of our effectiveness. It is critical to join the knowledge we gain from program evaluation in the “real world” setting with controlled trials to arrive at a fully formed picture of good clinical care. Program evaluation maximizes external validity, typically examining utility, feasibility, and effectiveness. Clinical research maximizes internal validity, using tight controls to increase accuracy and precision. Program evaluation not only tells us “does this work?” but “who does it work for?” and “how can we improve it next time?” When we publish program evaluation we serve the community by being brave enough to open our work to scrutiny by others, and by being generous enough to share our ideas to help others.
VA Author Manuscript
As I understand it, in advancing evidence-based care we start with evaluation of a new program, move to clinical trials, and then shift to dissemination and implementation. However, I find dissemination and implementation frameworks useful for thinking about program evaluation on the front end as well. There are many frameworks available—one with freely available resources is the “RE-AIM” framework at http://re-aim.org. Their checklists, measures, and question sets will help you to identify variables to measure that might not have occurred to you. In this issue of our journal we have evaluations of programs that are clinical in nature, as well as those focusing on mentorship of professionals and education of community members. First up is Knight and Alarie’s evaluation of a geriatric mental health day treatment service (Knight & Alarie, 2017). Several things distinguish their evaluation including its large size (N = 255), its combination of pre-post measurement as well as focus groups, and its (somewhat rare) evaluation of psychiatric day treatment for older adults. I found their reflection on the challenges of “aftercare”—their clients desire to retain some connection after discharge—to be interesting and important. Next we have Emery-Tiburcio’s group presenting results from “BRIGHTEN” (Bridging Resources of a Geriatric Health
Moye
Page 2
Team via Electronic Networking) (Emery-Tiburcio et al., 2017). They demonstrate the ability to enroll participants into the program, and for the program to be equally effective across racial/ethnic groups and educational levels. Their innovative model uses treatment by linguistically and culturally sensitive staff combined with email consultation with an interdisciplinary team.
VA Author Manuscript
We are pleased to also share a program evaluation of a peer mentorship initiative with psychologists in home based primary care teams (Terry, Gordon, Steadman-Wood, & Karel, 2017). The authors describe the content of mentorship contacts (e.g., discussions of challenges with clinical care, professional communication, work-life balance), as well as participants’ high satisfaction with the program. Supporting new professionals working with older adults is one important piece of the puzzle of improving care overall.
VA Author Manuscript
We have two clinical comments that serve as small scale evaluations. Barrera and colleagues (Barrera et al., 2017) describe outcomes for three individuals treated with the “VA-HELPS” (the Veterans Affairs Home Based Emotional Learning with Practical Skills Program) for anxiety and depression via the telephone. They describe session content, modifications, and pre-post measures. Significant treatment gains were made by each individual despite—or perhaps because they altered the standard protocol to meet individual needs associated with comorbid health conditions. Given the considerable barrier that travel presents for mental health treatment for some older adults, it is wonderful to have information about applications of telephone based treatment. Cheung, a new and emerging professional with colleague Sundram (Cheung & Sundram, 2017) describes one individual treated for suicidal behavior following a cerebral vascular accident and loss of function. Addressing the interactions of health problems, associated functional change, and emotional health outcomes is one of the complexities of geriatric mental health care. Our final program evaluation paper focuses on an educational intervention. Grisby and colleagues (Grisby, Unger, Molina, & Baron, 2017) use an audio-visual novella to improves beliefs, attitudes, and knowledge about dementia among a primarily Hispanic participant sample. It is so helpful to learn about this innovative educational strategy. The article provides the YouTube link to see the novella in English or Spanish.
VA Author Manuscript
We also include in this issue an intriguing paper by Jaconelli and colleagues (Jaconelli et al., 2017). They discuss “subjective age”, which is how young or old you “feel”—and has been associated with physical and psychological function. In their cross-national study of 175 adults with and without dementia, Jaconelli and colleagues note equivalence in “subjective age” between adults with and without dementia, and across the US and French samples. The Clinical Gerontologist is increasingly inter-national—but few studies offer us cross-national comparisons; we need more like this. We thank you for sharing your program evaluation and look forward to hearing more from the field.
References Barrera TL, Cummings JP, Armento M, Cully JA, Amspoker AB, Wilson NL, … Stanley MA. Telephone-delivered cognitive-behavioral therapy for older, rural veterans with depression and
Clin Gerontol. Author manuscript; available in PMC 2017 June 09.
Moye
Page 3
VA Author Manuscript
anxiety in home-based primary care. Clinical Gerontologist. 2017; 40(2):114–123. [PubMed: 28452676] Cheung G, Sundram F. Understanding the progression from physical illness to suicidal behavior: A case study based on a newly developed conceptual model. Clinical Gerontologist. 2017; 40(2):124– 129. [PubMed: 28452674] Emery-Tiburcio EE, Mack L, Lattie EG, Lusarreta M, Marquine M, Vail M, Goden R. Managing depression among diverse older adults in primary care: The BRIGHTEN program. Clinical Gerontologist. 2017; 40(2):88–96. [PubMed: 28452672] Grisby TJ, Unger JB, Molina GB, Baron M. Evaluation of an audio-visual novella to improve beliefs, attitudes, and knowledge toward dementia: A mixed-methods approach. Clinical Gerontologist. 2017; 40(2):130–138. [PubMed: 28452671] Jaconelli A, Terracciano A, Sutin AR, Sarrazin P, Raffard S, Stephan Y. Subjective age and dementia. Clinical Gerontologist. 2017; 40(2):106–113. [PubMed: 28452675] Knight CA, Alarie RM. Improving Mental Health in the Community: Outcome Evaluation of a Geriatric Mental Health Day Treatment Service. Clinical Gerontologist. 2017; 40(2):77–87. [PubMed: 28452673] Terry DL, Gordon BH, Steadman-Wood P, Karel MJ. A peer mentorship program for mental health professionals in veterans health administration home-based primary care. Clinical Gerontologist. 2017; 40(2):97–105. [PubMed: 28452670]
VA Author Manuscript VA Author Manuscript Clin Gerontol. Author manuscript; available in PMC 2017 June 09.
VA Author Manuscript
Are We Doing a Good Job? In Praise of Program Evaluation Jennifer Moye VA Boston Healthcare System and Harvard Medical School, Boston, Massachusetts, USA Fellow Clinical Gerontologists, welcome to issue 2 of 2017. Regular readers of our journal may have noticed that we are introducing a new feature. Each article must now include a “clinical implications” statement in the abstract and again in the discussion. Soon every article should clearly indicate “so what does this mean for the clinician in the field?”
VA Author Manuscript
Most of the papers in our second issue of 2017 focus on program evaluation. I am very enthusiastic about program evaluation. Why? Well, for starters—the doing of clinical work is demanding enough—it takes extra energy to go to the extra step of program evaluation. Yet, systematically evaluating our programs forces us to characterize what we are doing and to investigate the extent and limits of our effectiveness. It is critical to join the knowledge we gain from program evaluation in the “real world” setting with controlled trials to arrive at a fully formed picture of good clinical care. Program evaluation maximizes external validity, typically examining utility, feasibility, and effectiveness. Clinical research maximizes internal validity, using tight controls to increase accuracy and precision. Program evaluation not only tells us “does this work?” but “who does it work for?” and “how can we improve it next time?” When we publish program evaluation we serve the community by being brave enough to open our work to scrutiny by others, and by being generous enough to share our ideas to help others.
VA Author Manuscript
As I understand it, in advancing evidence-based care we start with evaluation of a new program, move to clinical trials, and then shift to dissemination and implementation. However, I find dissemination and implementation frameworks useful for thinking about program evaluation on the front end as well. There are many frameworks available—one with freely available resources is the “RE-AIM” framework at http://re-aim.org. Their checklists, measures, and question sets will help you to identify variables to measure that might not have occurred to you. In this issue of our journal we have evaluations of programs that are clinical in nature, as well as those focusing on mentorship of professionals and education of community members. First up is Knight and Alarie’s evaluation of a geriatric mental health day treatment service (Knight & Alarie, 2017). Several things distinguish their evaluation including its large size (N = 255), its combination of pre-post measurement as well as focus groups, and its (somewhat rare) evaluation of psychiatric day treatment for older adults. I found their reflection on the challenges of “aftercare”—their clients desire to retain some connection after discharge—to be interesting and important. Next we have Emery-Tiburcio’s group presenting results from “BRIGHTEN” (Bridging Resources of a Geriatric Health
Moye
Page 2
Team via Electronic Networking) (Emery-Tiburcio et al., 2017). They demonstrate the ability to enroll participants into the program, and for the program to be equally effective across racial/ethnic groups and educational levels. Their innovative model uses treatment by linguistically and culturally sensitive staff combined with email consultation with an interdisciplinary team.
VA Author Manuscript
We are pleased to also share a program evaluation of a peer mentorship initiative with psychologists in home based primary care teams (Terry, Gordon, Steadman-Wood, & Karel, 2017). The authors describe the content of mentorship contacts (e.g., discussions of challenges with clinical care, professional communication, work-life balance), as well as participants’ high satisfaction with the program. Supporting new professionals working with older adults is one important piece of the puzzle of improving care overall.
VA Author Manuscript
We have two clinical comments that serve as small scale evaluations. Barrera and colleagues (Barrera et al., 2017) describe outcomes for three individuals treated with the “VA-HELPS” (the Veterans Affairs Home Based Emotional Learning with Practical Skills Program) for anxiety and depression via the telephone. They describe session content, modifications, and pre-post measures. Significant treatment gains were made by each individual despite—or perhaps because they altered the standard protocol to meet individual needs associated with comorbid health conditions. Given the considerable barrier that travel presents for mental health treatment for some older adults, it is wonderful to have information about applications of telephone based treatment. Cheung, a new and emerging professional with colleague Sundram (Cheung & Sundram, 2017) describes one individual treated for suicidal behavior following a cerebral vascular accident and loss of function. Addressing the interactions of health problems, associated functional change, and emotional health outcomes is one of the complexities of geriatric mental health care. Our final program evaluation paper focuses on an educational intervention. Grisby and colleagues (Grisby, Unger, Molina, & Baron, 2017) use an audio-visual novella to improves beliefs, attitudes, and knowledge about dementia among a primarily Hispanic participant sample. It is so helpful to learn about this innovative educational strategy. The article provides the YouTube link to see the novella in English or Spanish.
VA Author Manuscript
We also include in this issue an intriguing paper by Jaconelli and colleagues (Jaconelli et al., 2017). They discuss “subjective age”, which is how young or old you “feel”—and has been associated with physical and psychological function. In their cross-national study of 175 adults with and without dementia, Jaconelli and colleagues note equivalence in “subjective age” between adults with and without dementia, and across the US and French samples. The Clinical Gerontologist is increasingly inter-national—but few studies offer us cross-national comparisons; we need more like this. We thank you for sharing your program evaluation and look forward to hearing more from the field.
References Barrera TL, Cummings JP, Armento M, Cully JA, Amspoker AB, Wilson NL, … Stanley MA. Telephone-delivered cognitive-behavioral therapy for older, rural veterans with depression and
Clin Gerontol. Author manuscript; available in PMC 2017 June 09.
Moye
Page 3
VA Author Manuscript
anxiety in home-based primary care. Clinical Gerontologist. 2017; 40(2):114–123. [PubMed: 28452676] Cheung G, Sundram F. Understanding the progression from physical illness to suicidal behavior: A case study based on a newly developed conceptual model. Clinical Gerontologist. 2017; 40(2):124– 129. [PubMed: 28452674] Emery-Tiburcio EE, Mack L, Lattie EG, Lusarreta M, Marquine M, Vail M, Goden R. Managing depression among diverse older adults in primary care: The BRIGHTEN program. Clinical Gerontologist. 2017; 40(2):88–96. [PubMed: 28452672] Grisby TJ, Unger JB, Molina GB, Baron M. Evaluation of an audio-visual novella to improve beliefs, attitudes, and knowledge toward dementia: A mixed-methods approach. Clinical Gerontologist. 2017; 40(2):130–138. [PubMed: 28452671] Jaconelli A, Terracciano A, Sutin AR, Sarrazin P, Raffard S, Stephan Y. Subjective age and dementia. Clinical Gerontologist. 2017; 40(2):106–113. [PubMed: 28452675] Knight CA, Alarie RM. Improving Mental Health in the Community: Outcome Evaluation of a Geriatric Mental Health Day Treatment Service. Clinical Gerontologist. 2017; 40(2):77–87. [PubMed: 28452673] Terry DL, Gordon BH, Steadman-Wood P, Karel MJ. A peer mentorship program for mental health professionals in veterans health administration home-based primary care. Clinical Gerontologist. 2017; 40(2):97–105. [PubMed: 28452670]
VA Author Manuscript VA Author Manuscript Clin Gerontol. Author manuscript; available in PMC 2017 June 09.
