The effectiveness of consultation-liaison psychiatry in the general hospital setting: a systematic review.

Journal of Psychosomatic Research 76 (2014) 175–192

Contents lists available at ScienceDirect

Journal of Psychosomatic Research

Review

The effectiveness of consultation-liaison psychiatry in the general hospital setting: A systematic review Rebecca Wood a,b,⁎, Anne P.F. Wand b,c,d a

Sydney Local Health District, New South Wales, Australia Discipline of Psychiatry, Sydney Medical School, University of Sydney, New South Wales, Australia South East Sydney Local Health District, New South Wales, Australia d School of Psychiatry, Faculty of Medicine, University of New South Wales, Australia b c

a r t i c l e

i n f o

a b s t r a c t Objective: The aim of this study was to review how the effectiveness of consultation liaison psychiatry (CLP) services has been measured and to evaluate the strength of the evidence for effectiveness. Methods: Systematic review of medical databases using broad search terms as well as expert opinion was sought. The literature search was restricted to studies of general, whole-of-hospital inpatient CLP services. Results: Forty articles were found and grouped into five measurements of effectiveness: cost effectiveness including length of stay, concordance, staff and patient feedback, and follow-up outcome studies. All measurements contributed to the evaluation of CLP services, but no one measure in isolation could adequately cover the multifaceted roles of CLP. Concordance was the only measurement with an established, consistent approach for evaluation. Cost effectiveness and follow-up outcome studies were the only measures with levels of evidence above four, however the three follow-up outcome studies reported conflicting results. Subjective evidence derived from patient and staff feedback is important but presently lacking due to methodological problems. The effectiveness of CLP services was demonstrated by cost-effectiveness, earlier referrals to CLP predicting shorter length of stay, and concordance with some management recommendations. Conclusion: There is evidence that some CLP services are cost-effective and reduce length of stay when involved early and that referrers follow certain recommendations. However, many studies had disparate results and were methodologically flawed. Future research should focus on standardising patient and staff feedback, and shortterm patient outcomes. Crown Copyright © 2014 Published by Elsevier Inc. All rights reserved.

Article history: Received 15 November 2013 Received in revised form 2 January 2014 Accepted 3 January 2014 Keywords: Consultation-liaison psychiatry Effectiveness Evaluation Psychosomatic medicine Systematic review

Contents Introduction . . . . . . . . . . . Method . . . . . . . . . . . . . Search strategy . . . . . . . Inclusion and exclusion criteria Results . . . . . . . . . . . . . Literature search . . . . . . . Cost-effectiveness and length of stay Concordance . . . . . . . . . . . Staff feedback . . . . . . . . . . Patient feedback . . . . . . . . . Follow-up studies . . . . . . . . Discussion . . . . . . . . . . . . Limitations . . . . . . . . . Conclusion . . . . . . . . . . . . Competing interest statement . . . References . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

⁎ Corresponding author at: Concord Centre for Mental Health, Hospital Rd, Concord NSW 2138, Australia. Tel.: +61 2 97675000. E-mail address: [email protected] (R. Wood). 0022-3999/$ – see front matter. Crown Copyright © 2014 Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jpsychores.2014.01.002

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

176 177 177 177 177 177 177 187 187 187 187 187 191 191 192 192

176

R. Wood, A.P.F. Wand / Journal of Psychosomatic Research 76 (2014) 175–192

Introduction Consultation-liaison psychiatry (CLP) is the subspecialty of psychiatry concerned with medically and surgically ill patients in a general hospital setting [1]. There has been a consistent call for evidence to demonstrate the effectiveness of these services and to justify their ongoing funding [2]. The nature of psychiatric practice in this setting and the often intangible nature of the benefits have made this elusive. The increased sense of well-being that a patient gains from successful treatment of a psychiatric disorder may be as valuable as the “quantifiable aspects” of psychiatric interventions [3]. However, with an increasing focus upon evidence based practice and the need to justify resource

allocation, it is incumbent upon CLP to objectively demonstrate its value. Lipowski [4] suggested the following ways that CLP services should measure their effectiveness: reduction in the cost of medical care, staff and patient feedback, effects on length of stay, concordance with recommendations (that is whether the referring team adopts the advice of CLP), utilisation of laboratory tests and relief of patient symptoms and social functioning after discharge. The Netherlands Psychiatric Association Guideline for Consultation Psychiatry recently asked, among other questions, whether consultation psychiatry services in non-psychiatric settings are effective [5]. While this review concluded that these services were effective, the quality of the evidence for effectiveness was not extensively examined. Previous

1916 records identified

6 records identified through studies and reviews known to authors

through database searching

Records after duplicates removed (n=1659)

1659 records screened

28 full text articles assessed for eligibility

13 records excluded Irrelevant (n=3) Description/efficiency studies (n=5) Setting other than medical hospital (n=0) One diagnostic or population group of patients (n=1) Non-psychiatric staff only (n=2) Not original studies [reviews or discussion of issues) (n=2) Language other than English (n=0)

16 studies included in review

Manual search of reference lists of included studies (n=42)

29 studies added to 16 studies originally found = 45 studies included in review

1631 records excluded Irrelevant (n=1480) Description/efficiency studies (n=66) Setting other than medical hospital (n=15) One diagnostic or population group of patients (n=30) Non-psychiatric staff only (n=5) Not original studies (reviews or discussion papers) (n=28) Language other than English (n=7)

12 records excluded Irrelevant (n=2) Description/efficiency studies (n=7) Setting other than medical hospital (n=0) One diagnostic or population group of patients (n=1) Non-psychiatric staff only (n=0) Not original studies (reviews or discussion papers) (n=2) Language other than English (n=0)

5 duplicate studies removed *

40 studies included in review

Fig. 1. Results of literature search: PRISMA flow chart [56]. *Five duplicate studies [57–61].


reviews of one aspect of effectiveness such as cost effectiveness or adherence to recommendations have been conducted [6–8], however focussing on one outcome measure is insufficient to capture the full breadth of activities of CLP services. Similarly, reports of the effectiveness of specific liaison attachments or services to one ward or medical speciality, which are dedicated and expensive services less widespread than whole-of-hospital CLP, are widely variable in terms of design and goals, which limit their generalisability [9]. A review of studies from 1970–1981 broadly evaluated CLP services by describing and also reporting outcome measures [3]. This review identified the beginning of a shift towards conclusion orientated evaluative studies, encouraged the further development of studies of patient outcomes and identified the need for more exact definitions of the aspects of CLP that are most useful and valuable. Here, we review the CLP literature including more recent studies from the ensuing 30 years which have evaluated the effectiveness of general hospital CLP services using a range of measures. The purpose of the review was to ascertain the strength of the published evidence supporting the current and widespread use of inpatient based CLP services. Method Search strategy The databases Medline (from 1946), PsycINFO (from 1806) and All EBM reviews (Cochrane DSR, ACP Journal Club, DARE, CCTR, CMR, HTA and NHSEED) were searched using the same text words in October 2013. The text terms evaluat*, effic*, service delivery and effect* were used in combination with the following text terms: consultation liaison psychiatry, liaison psychiatry and psychosomatic medicine. PsycINFO was searched again using the subject headings “Psychosomatic Medicine” in place of all text terms pertaining to CLP, and “health delivery” in place of service delivery as well as the text terms evaluat*, effic* and effect*. In addition, articles from experts in CLP known to the authors were sourced. Inclusion and exclusion criteria Articles were selected if they were original studies which evaluated an entire hospital-based CLP service with at least one measure of effectiveness. The service needed to evaluate inpatients seen by the CLP service and it was not necessary for the CLP service to cover particular specialist departments. Publications were excluded if they only described a service in terms of the patients referred or other aspects of its operation. Measurements such as response time from referral to consultation or referral rates, which primarily indicate efficiency or performance of a service rather than its effectiveness, were not included. The CLP service needed to be available for all admissions to the hospital and not focus solely on one diagnostic group of patients or outpatients. The CLP service evaluated needed to include medically trained psychiatric staff such as psychiatrists, psychiatric residents or registrars, but could also comprise, but not be limited to, other mental health professionals as part of the team. Articles in languages other than English were excluded. Titles and abstracts from the database search were screened and then relevant full-text articles obtained and checked for eligibility independently by both authors. The reference lists of included articles were also screened to identify any additional relevant articles which were also checked in full text for eligibility independently by both authors. When there was a discrepancy between the authors, both checked again and discussed the issue until a consensus was reached. When all articles were identified, they were categorised according to the method of measuring effectiveness. The quality of each study was assessed using the Centre for Reviews and Dissemination's guidance for undertaking reviews in health care [10]. The level of evidence of each study was

177

determined using the Oxford Centre for Evidence-Based Medicine 2011 definition [11]. Results Literature search A total of 1659 publications were identified from the database search and other sources. Overall, 40 articles met the inclusion criteria (see Fig. 1). There were five distinct ways that effectiveness had been assessed. The categories identified were cost effectiveness (which included the closely related measure of length of stay), concordance, patient satisfaction, staff satisfaction, and outcomes at follow-up. Five studies used two measurements of effectiveness and four studies used three. Therefore, some of the studies were included in more than one category.

Cost-effectiveness and length of stay The eleven studies evaluating the cost-effectiveness of CLP services and length of stay are described in Table 1. The Rapid Assessment Interface and Discharge (RAID) study had within the goal of this CLP service to reduce length of stay and prevent readmissions, so it is perhaps unsurprising that the most convincing and impressive cost benefits were derived with this approach [12]. However, it was limited by the before and after design and a lack of description of psychosocial services available in the ‘control’ group prior to the intervention. The inability to control for other factors which may account for the results is another drawback of this design [13], and also applicable to other studies [14,15]. Billings [15] was the first to evaluate the cost effectiveness of a CLP service. Although quite dated, this study compared the admission costs of ‘psychiatric patients’, whom we assume to be patients referred to the CLP team, with all admitted general hospital patients to provide a comparison in terms of changes in length of stay and costs per patient. However, it is limited by a series of assumptions for which no evidence is provided. A Japanese study [16] compared the balance of salary and reimbursement of two CLP services in similar hospitals, one with a full time psychiatrist and the other part time, and found both services to be profitable on this basis, the full time service proportionally more so. An American study [17] also described how a CLP service may generate revenue which covered the cost of the service, but neither of these studies provided evidence of cost-savings through patient outcomes [16,17]. Two studies [14,18] assessed the effect of psychiatric consultation with medical patients with high screening scores for psychiatric symptoms and confusion, rather than consultation in response to physician request, which limited its extrapolation to typical CLP services where referrals are made by staff based upon clinical judgement. Five studies examined the association of length of stay with timing to referral, in order to determine whether earlier referral to CLP services was associated with a shorter length of stay and thereby demonstrate the effectiveness of CLP services on reducing length of stay and admission costs. There is an obvious confounder in making a direct comparison of timing to referral to overall length of stay as those who are in hospital longer will be more likely to receive psychiatric consults later [19]. Therefore, a new variable was calculated for the timing of the referral: the number of days prior to consultation divided by the length of stay [20]. A logarithmic transformation was used to account for the positive skew of the data. The first study to use this method found that the length of stay correlated with the timing to referral variable and it accounted for 12% of the variance of length of stay [20]. There were no statistical differences in terms of psychiatric diagnosis or whether they were medical or surgical inpatients referred in the first compared to the second half of their admission. However, it is possible that some consultations were earlier as the teams anticipated a shorter stay or that the patients who needed more urgent psychiatric input had shorter stays, such as those patients admitted following suicide attempts. Handrinos and colleagues [19] supported the use of timing to referral as a proportion of length of stay and found that this variable was independent of length of stay when this was greater than four days.

178

Table 1 Cost effectiveness and length of stay Setting and target population

Billings, 1941 [15]

–Reduction in length of stay was taken as 4 indicating reduced morbidity through assumed reductions in the time the patient was out of social or work roles, without any objective measurement provided to justify this conclusion. –No detail was provided on how costs/ patient were determined. –There was no evidence provided to conclude that CL involvement reduced ‘inappropriate’ investigations and therefore costs –Retrospective design cannot control for potential confounding factors –Timing variable used was the –There was a strong positive relationship –Other patient factors which affect length 4 General CLP service to the Retrospective comparison of General teaching of stay such as severity of disease or type of between the timing of referral and logarithm of the days prior to whole hospital, with medical the timing of referral to CLP hospital in the USA length of stay which indicates that the the consult divided by the and surgical patients with length of stay for 419 (900 beds) treatment may be confounding the logarithm of the length of stay. sooner the consultation, the shorter the relationship between the two variables consecutive inpatient referrals length of stay. –Length of stay to the CLP service –This association should not lead to the –The Pearson correlation coefficient assumption that earlier referral causes between the two transformed variables shorter length of stay as this causal was 0.342 (p b 0.001) relationship has not been proven by this –There were no statistical differences method. between those patients referred in the first or second halves of their admission in terms of medical versus surgical or psychiatric diagnosis. –Use of screening limits generalisability to 2 –Baseline (historical) controls 232 vs –Psychiatrist review of patients –Hospital resource use –Randomised controlled trial –General medical most real life CLP services. –Length of stay contemporaneous controls 253 vs (two control groups: historical randomised to intervention inpatients of a large –Possible dilution of effect by the inclusion experimental consultation group 256 group, usually within 24 h and contemporaneous) university hospital in of 77 patients randomised to the inter(although only 158 received the –All patients were screened for –Usual CLP consultations were the USA vention group in the analysis, but not reintervention all were included in the also available to treating teams, depression, anxiety, confusion ceiving intervention. analysis). and pain (The Medical Inpatient but not evaluated –Difficult to demonstrate reductions in –No significant difference in hospital Screening Test). If high scores length of stay due to overall changes within resource use or length of stay once then randomised to psychiatric the hospital system over the study period. severity of illness controlled. consultation or no intervention. –Reimbursement to the psychiatrist per –The authors assume that the CLP service is 4 Reimbursement per patient General hospital in the –Descriptive –CLP service (2 psychiatrists, beneficial and required. patient consultation ranges from seen (direct earning) and USA (746 beds) 3 nurses, 1 resident) $49–140 (from Medicaid to commercial –The aim is to describe how a CLP service indirect revenue (from –140 referrals/month could pay for itself rather than measuring insurance companies). This funds all of referring CL patients to inpa–Initial formal psychiatrist any direct benefits to patient's outcomes. the psychiatrists' consultation time. tient units) review then informal liaison –Indirect revenue to CL service is derived –US system of reimbursement for seeing nursing from their referrals of CL patients to inpatients is very different to other public –36% patients are aged N60; patient psychiatric units (~15.5 patients/ health services such as the NHS or 58% 20–40. month, which raises ~ $23,250/month) Australian system. –Assumptions of CLP service attributable –Estimation of cost savings from (unproven) reductions in length of stay: reductions in length of inpatient stay, based on other unrelated studies, not $1087 (Medicaid)–$2900/month (prievidence from this service. vately insured patients)

Lyons et al., 1986 [20]

Levenson et al., 1992 [14]

Schuster, 1992 [17]

General teaching Hospital in the USA (150 beds)

Study design

Retrospective comparison of annual statistics from baseline when the CLP service started to 5 years later

Service type

Outcome measures

–Length of stay General CLP service to the whole hospital including adults, –Cost/patient –‘Efficiency’ described in terms children and outpatients of limiting ‘futile’ (inappropriate) investigations

Conclusions

–Length of stay for ‘psychiatric patients’ reduced from 28.1 (baseline) to 14.8 days (after 5 years), which approximated the average cost for all admitted patients –Cost/patient improved to approximate the average cost for all general hospital patients –Improved efficiency through costsavings by avoiding ‘futile’ investigations in somatising patients

Limitations and comments

Level of evidence [11]


Reference

Reference Study design

Service type

Outcome measures

Ormont et al., 1997 [23]

–General teaching hospital in the USA (1,330 beds over two sites)

–Comparison of the timing of the referral with the number of days from referral to discharge for 145 consultations seen by the first author while working in the CLP service

–Hospital day of consultation General CLP service to the –Number of days from whole hospital, however only one psychiatric resident's initial consultation until discharge consultations were examined which was 20% of the whole service's consultations for the year.

Handrinos et al., 1998 [19]

–A low but significant correlation was found between the referral timing variable and length of stay, but when length of stay b4 days was excluded, there was no significant correlation. –Usage of the timing variable of the ratio of the time elapsed between admission and referral and length of stay was independent of the patient's length of stay when this was greater than 4 days and thus is a more useful way of measuring timing than is the time elapsed between admission and referral. –General teaching –Prospective randomised Semi-structured instrument de- –Total costs were significantly lower in General CLP service to whole hospital in the UK controlled trial the physician informed group compared hospital however the study was signed for the study to assess –All inpatients eligible screened limited to inpatients on medical economic impact of illness inwith the intervention group but neither for psychiatric disorder using was significantly different from the wards. Patients were excluded cluded contacts with medical the General Health control group. and other services, additional if they were already receiving Questionnaire (GHQ-28) psychiatric care, were admitted expenses and loss of productiv- –When analysis was confined to the –209 patients met threshold unemployed, there were no significant for self-harm, day patients and ity. and were randomly allocated to those unable to complete the differences between the three groups for –Both the patient and a family three groups: control group, any of the NHS component costs. member/friend were screening tool. “physician informed” (the –Including the additional costs to the interviewed. treating team were aware of Hospital records, GP's and social patient and loss of productivity for a screening results and could workers were also used to esti- grand total, neither of the intervention decide whether to refer to CLP groups were significantly different from mate costs. or not) and “intervention” the control group. group who received a psychiatric assessment. –159 patients were followed-up six months later and reassessed. –Two general hospitals –Prospective comparison of two –One hospital had a full time –Numbers of patients seen and –Monthly total number of patients in Japan (380 and 445 CLP services from implementa- psychiatrist and the other a part medical reimbursement increased to 505 in the full time service, tion of services for the first 10 - time (0.2) psychiatrist. beds) and in the other service plateaued to 30– earned. –Salary of the two psychiatrists 55 patients per month. –Both services accepted months. –The difference in the monthly total referrals from in- and outnumber of patients and reimbursement patients of the hospital but also could not be completely explained by the from patients outside.

Gater et al., 1998 [18]

Hosaka et al., 1999 [16]

General teaching hospital in Australia (300 beds)

Prospective comparison of timing of referral and length of stay for 712 consecutive inpatient referrals to the CLP service

General CLP service to the whole hospital (excluding referrals for selfharm)

Conclusions

–Number of days from admission to consultation –Timing variable of the days prior to the consultation divided by the length of stay. –Length of stay



–There is an association between earlier timing of consultations and shorter subsequent hospitalisation (p = 0.022)

–Only 20% of the consultations of the CLP 4 service were examined, and only those made to a psychiatric resident. This may have introduced bias in the patient population and reasons for referral. –The use of days from consultation to discharge removes the confounder of those referred later being more likely to have longer lengths of stay. However, this method is open to bias in the reason for referral particularly if the hospital culture is to refer to CLP late as part of discharge planning. –This association should not lead to the assumption that earlier referral causes shorter length of stay as this causal relationship has not been proven by this method –This study supports the use of the timing 4 variable of time from admission to consultation divided by length of stay by virtue of the lack of significant correlation for lengths of stay longer than 4 days, however this result could also be due to the fact that this CLP service had less effect on length of stay if referred earlier. –The exclusion of referrals for self-harm (typical CLP patients) was justified by authors because these referrals were based on hospital policy rather than clinical judgement, which may have confounded results. –The study is unique in its comprehensive 2 estimation of costs. –Use of screening limits generalisability to most real-life CLP services. –There was a significant difference in employment status between the groups at follow-up, which confounded results. –The exclusion criteria removed a sizeable population typical of CLP patients including those with self-harm, in psychiatric care, or unable to complete the screening tool (e.g. significant cognitive impairment), thus limiting generalisability and underestimating the effect of CLP.


Setting and target population

–The two services may have been different 4 in terms of the psychiatrist's style of working, referral trends and other hospital related factors which confounds comparison between the two services. –The simple balance of reimbursement and the psychiatrist's salary does not (continued on next page)

179

180

Table 1 (continued) Reference

Setting and target population

Study design

Service type

Outcome measures

–The full time service provided a liaison attachment and regular education.

Conclusions



account for the other costs associated with running the clinics. –This study provides evidence for economic sustainability of the services rather than measuring any direct benefits to patient's outcomes. –Other patient factors which affect length 4 –The referral timing variable correlated of stay such as severity of disease or type of with the logarithm of the length of stay treatment may be confounding the (r = 0.51, p b 0.001), accounting for 25.7% of the variation in the length of stay, relationship between the two variables –This association should not lead to the indicating that the earlier consultations assumption that earlier referral causes predicted a shorter length of stay. –When the medical condition, psychiatric shorter length of stay as this causal reason for referral and diagnosis, and the relationship has not been proven by this method. interventions recommended were controlled for, the timing of the psychiatric consultation still predicted length of stay. –Other patient factors which affect length 4 The referral timing variable correlated with the logarithm of the length of stay of stay such as severity of disease or type of treatment may be confounding the (95% CI, 0.64 to 0.75; p = 0.0001), acrelationship between the two variables counting for 22% of the variation, indi–This association should not lead to the cating that the earlier consultations assumption that earlier referral causes predicted a shorter length of stay. shorter length of stay as this causal relationship has not been proven by this method. –There may be additional costs of shorter 2 –Length of stay: RAID- direct managelength of stay such as increased burden on ment group (79 matched subjects) community beds or services or outpatient 0.9 days less than pre-RAID; RAIDmental health services influenced (359 matched subjects) 3.2 days less than pre-RAID. 93% of bed- –Non-randomised design, with historical cohort as control so cannot allow for all day savings were in people ≥65. –MAUb: 30% of pre-RAID group avoided potential biases such as changes in hospital practice affecting admissions or length of admission vs 33% in RAID intervention stay and characteristics of individual group i.e. avoided 160 admissions (full patients (only 79 matched RAID-direct unmatched sample) –Readmission 70% lower in RAID-direct management patients used in analysis and there was evidence of systemic changes in management group, than in pre-RAID control. No difference in RAID-influence hospital focused on facilitating discharge or avoiding admission in the study period) group. i.e. 1800 admissions saved. 86% –There may be additional economic benesavings were in people ≥65 –Discharge to own home 34% pre-RAID fits e.g. if RAID intervention reduces patient work absenteeism vs 67% RAID intervention group (i.e. 175 discharges to residential care prevented) number of working days for the part time service. –Both services covered the salary of the psychiatrist with the reimbursement funds.

General teaching hospital in the USA

Retrospective comparison of timing of referral and length of stay for 541 consecutive inpatient referrals to the CLP service

General CLP service to the whole hospital

–Timing variable used was the logarithm of the days prior to the consultation divided by the logarithm of the length of stay. –Length of stay

Alhuthail, 2009 [22]

General teaching hospital in Saudi Arabia (800 beds)

Prospective comparison of timing of referral and length of stay for 264 consecutive inpatient referrals to the CLP service

General CLP service to the whole hospital

–Timing variable used was the logarithm of the days prior to the consultation divided by the logarithm of the length of stay. –Length of stay

Parsonage and Fossey, 2011 [12]

–General teaching hospital in the UK. –All acute referred non-electively admitted patients N16 years old (77% 16–64; 23% N65) –Included self-harm, substance abuse, mental health problems associated with old age e.g. dementia

–Before (retrospective cohort) and after introduction of the service –Pair-matched control (gender, age, health resource group, diagnosis) pre and post. Two groups RAID-direct management and RAID-influenceda

–Rapid Assessment Interface and Discharge (RAID) CLP service. –Available 24 h a day, 7 days/ week. –Emphasis was rapid response, diversion and discharge from the Emergency Department and early, effective discharge from general wards. –250 referrals/month

–Length of inpatient stay –Avoidance of admission to general wards after initial admission to short-stay MAUb –reduced readmission rates after discharge –discharge destination

a RAID direct subgroup refers to patients in the intervention group who were referred to and directly managed by the RAID team. RAID-influenced subgroup were patients in the intervention group for whom the RAID team supported and trained hospital staff managing these inpatients (this included formal teaching sessions and informal hands-on training). b MAU — Medical admission unit.


Kishi et al., 2004 [21]

Table 2 Concordance Reference

Type of study

Moses and Barzilay, 1967 [24]

Type of CL advice measured for concordance

Method

Rate of concordance



Retrospective review of 173 medical records in a general teaching hospital in Israel Retrospective review of 242 diagnoses in 205 patients medical records in a paediatric teaching hospital in Canada

Accuracy of reporting diagnosis and management in discharge summary

Description of diagnosis and treatment in discharge summary rated as not mentioned, incidentally mentioned, appropriate or overly emphasised. Admitting and discharge diagnosis were classified as medical only, psychiatric and mixed. Changes in diagnosis were rated as no change, more psychiatric, less psychiatric.These diagnoses were compared with those made by the CLP team

46% of discharge summaries were appropriate on both diagnosis and treatment.

–No operational criteria for defining concordance. –No details regarding impact of follow up reviews on concordance rates. –Limited description of method, it was unclear how the CLP diagnosis was obtained. –No operational criteria for defining concordance. –There may be some aspects particular to psychiatric diagnoses in children which limit generalisability of these findings for all populations –No details regarding impact of follow up reviews on concordance rates.

4

Billowitz and Friedson, 1978 [26]

Retrospective review of 273 medical records in a general teaching hospital In the USA

All types of recommendations for management

Neill, 1979 [27]

Retrospective review of 100 medical records in a general teaching hospital in the USA

Recommendations related to medication

Taintor et al., 1979 [28]

Prospective record of consultations in a general hospital in the USA

975


Van Dyke et al., 1980 [29]

Retrospective study of consultations in a general teaching hospital in the USA

55


Popkin et al., 1982 [33]

Retrospective review of 306 medical records in a general teaching hospital in the USA

Froese, 1977 [25]

No of cases studied

Accuracy of reporting diagnosis in discharge summary

–No operational criteria for defining concordance. –A proportion of records were co-rated which provided some indication of reliability of the judgement of concordance. –Exclusion of burns unit as a liaison attachment removed this confounder. –No details regarding impact of followup reviews on concordance rates. –Limited description of criteria for defining concordance. –No details given regarding who was responsible for charting the medication. –No details regarding impact of followup reviews on concordance rates. Study primarily sought to describe the use of Diagnostic action 49%, –As this data was reported incidentally a new form for collecting data, which medication 78%, environmental while reporting on a form for gathering included concordance of recommendations management 75%, psychological clinical information, there was no 80% description of method used. Independent raters reviewed the medical 90% of all recommendations –Limited description of criteria used for record of consultation recommendations and were complied with. defining concordance. –Used independent rating and a assessed if actioned percentage were co-rated for reliability. –Exclusion of wards with liaison attachments removed this confounder. –No report regarding frequency of reinforcement of recommendations, apart from stating consultants verbally discussed with teams when possible. –Clear description of who instigated recommendations: always the treating team except when psychiatric admission advised. Review of initial consultation 51% concordance and from these, –Operational criteria for defining recommendations and if the action was 45% of results found were concordance. ordered in the following 96 h, using specific abnormal. –Only examined concordance with the criteria for concordance and then location of initial consultation, which ignores the results to determine if outside normal impact of reinforcement on concordance. parameters –The low yield of abnormal results may have influenced the concordance, i.e. teams may have been more likely to do

4

4

4

4

4


Diagnostic action e.g. investigations, other consults and yield of abnormal results

50% of diagnoses accurately recorded in discharge summaries: 24% of patients had an incorrect diagnosis, and 32% did have their psychiatric diagnosis recorded. 3 of the 59 patients with a change to a more psychiatric diagnosis were described as “normal”. None of the patients considered “normal” by CLP had a less psychiatric diagnosis 3 scores: actively, passively and not followed. Inter-judge agreement: 100%. 10% of records were scored by a second judge. Actively followed recommendations: Disposition 92%, Fitness for surgery 80%, Management at discharge 80%, Medications 78%, Other service involvement 66%, Ward management 68%, Diagnostic action 46%. Review of recommendations and medication 76% for all types: 90% chart for prescription of drug. neuroleptics, 65% benzodiazepines, 56% tricyclics

4

181

(continued on next page)

182

Table 2 (continued) Reference

Type of study

No of cases studied


Retrospective study of consultations in a general teaching hospital in the USA

Lanting and Hengeveld, 1984 [34]

Retrospective review of 271 medical records in a general teaching hospital in the Netherlands

Accuracy of reporting diagnosis in discharge summary and all types of recommendations for management

Wise et al., 1987 [35]

200 Retrospective review of medical records in a general teaching hospital in the USA

Huyse et al., 1990 [37]

Prospective review of 317 medical records, direct observation and interviews with ward staff and referring teams in a general teaching hospital in the Netherlands

394 cases for psychotropic medication, 381 for diagnostic action and 358 for psychiatric diagnosis reporting

Recommendations for psychotropic drugs and diagnostic actions, and accuracy of reporting the psychiatric diagnosis in discharge summary

Rate of concordance

Review of initial consultation recommendations. Rating of the medication chart and/or the diagnostic action for full, partial and no concordance using specific criteria — full concordance if ordered in the following 96 h. The discharge summary diagnosis was compared with the initial consultation diagnosis and rated using specific criteria for full, partial and no concordance with review of uncertainties by two independent raters. Review of initial consult recommendations and then medication chart in the following 96 hand discharge summary, using specific criteria for full, partial and no concordance

63% full concordance for psychotropic drug recommendations. 53% full concordance for diagnostic action recommendations. 43% accurate reporting of the psychiatric diagnosis in the discharge summary.

Accuracy of reporting diagnosis in discharge summary and all types of recommendations for management

Review of initial consultation recommendations and then medical record and medication chart in the following 96 h, and discharge summary, using specific criteria for full, partial and no concordance

Discharge summaries accurate for diagnosis: 45% (full), 19% (partial), diagnostic action performed: 71% (full), 16% (partial) and medication 79% (full), 7% (partial).


Review of initial consultation recommendations and then medical record and medication chart in the following 96 h and discharge summary, using specific criteria for full, partial and no concordance, with the addition of an independent rater, direct observation of the ward and interviews with referring team

Recommendations related to time of discharge (96%), disposition (95%), restraints (89%), non-medical consults (86%), increasing physical treatment (84%) had the highest concordance rates. Systematic orientation of patient (21%), collateral sought from PCPa (40%), providing objects for orientation (48%) had the lowest concordance rates.

80% of medications recommended were charted. 49% of discharge summaries accurate for diagnosis and advice.


diagnostic tests if they anticipated an abnormal result. –Some diagnostic actions may have been delayed or not done due to availability or other factors beyond the control of the treating team. –Operational criteria for defining concordance. –Only examined concordance with the initial consultation, which ignored the impact of reinforcement and didn't allow for changes to the diagnosis subsequent to this. –Some diagnostic actions may have been delayed or not done due to availability or other factors beyond the control of the treating team. –Operational criteria used for defining concordance. –Only examined concordance with initial consult, ignoring the impact of reinforcement. –“Very often” the consultant wrote in the medication chart which would have falsely increased ratings of compliance with medication recommendations (41% of all recommendations studied). –Operational criteria used for defining concordance. –Only examined concordance with initial consultation, ignoring the potential impact of reinforcement. –Examined rates of concordance with medication depending on whether team or CL psychiatry service (36/66) charted the medication. –Prospective design may have artificially increased concordance due to observation effect. –Operational criteria used for defining concordance. –Addition of direct observation and contact with staff covered recommendations that are not always documented in the file as performed such as change in style of interaction with patient. –Advantage: study not limited to initial consultation recommendations. –Recommendations effected by consultant were excluded. –Excluded patients discharged less than 3 days after initial consult, which may have positively biased results providing more time for concordance.


4

4

4

4


Popkin et al., 1983 [32]

Method

Reference Type of study

No of cases studied

Seward et al., 1991 [36]

Rate of concordance



Retrospective review of 270 medical records in a general teaching hospital in Australia

Recommendations related to medication and diagnostic action and accuracy of reporting diagnosis in discharge summary

Review of all medication and diagnostic action recommendations by psychiatry through the admission: start, stop, adjust, continue and don't give, and review of discharge summary for final psychiatric diagnosis

Medication 86%. No data reported for diagnostic action due to small numbers. 53% of discharge summaries accurate for diagnosis. No difference in concordance of teams with/without liaison attachment.


Randomised controlled trial (concordance measured for the experimental group only) in a general teaching hospital in the USA

119 for psychiatric diagnosis, 42 for medication and 64 for diagnostic action recommendations

Recommendations related to psychiatric diagnosis, medication and diagnostic action

“Modified version” of the method involving Psychiatric diagnosis: 44% review of initial consultation Medication: 52% Diagnostic recommendations and then of the action: 33% medication chart in the following 96 h and discharge summary, using specific criteria for full, partial and no concordance.

Clarke and Smith,1995 [30]

Retrospective review of prospectively recorded data in a general teaching hospital in Australia

165

Drug and non-drug types of recommendations for management

Review of the CLP clinical database records for concordance

Medications: 97% Non-drug recommendations: 95% when those carried out by the CLP service were excluded.

Gater et al., 1998 [18]

–Prospective randomised controlled trial (concordance measured for the intervention group only) in a general teaching hospital in the UK.

68 patients in the intervention group were assessed: 11 medication recommendations, 3 recommendations for medication subject to conditions, 8 recommendations for further assessment, 6 for social worker referral and 8 for a therapeutic discussion. 37 patients with psychiatric diagnoses were examined for communication of the psychiatric diagnosis and assessment in the discharge summary.

Recommendations related to psychiatric diagnosis in the discharge summary, prescription of medication and referral to other services.

The psychiatrists' recommendations were traced in the hospital and general practitioners' case notes.

Rigatelli et al., 2001 [31]

Retrospective review of medical records and direct interviews with patients and PCP's after discharge in a general teaching hospital in Italy

95

Inclusion of psychiatric assessment in discharge summary provided to the patient and PCP.

Review of medical records for letter by psychiatry service or inclusion of psychiatry opinion in discharge summary. Interview of patient regarding receipt of letter and passage to PCP, and interview with PCP regarding knowledge of psychiatric assessment 3–5 months after discharge.

–Medication: 91% during admission, 18% after discharge by the general practitioner. None of the patients received a therapeutic dosage for an adequate duration of time. –No psychotropic drugs were prescribed to the 3 conditional recommendations. –100% of recommendations for a therapeutic discussion and further assessment. –83% of social worker referrals. –Less than one third contained a reference to either a psychiatric disorder or to the psychiatrist's assessment. 98% attached the psychiatric letter or provided an inclusion to the discharge summary. 87% of patients recalled receiving a letter/discharge summary and 74% passed it on to PCP. 67% of PCP's reported knowledge of psychiatric assessment

–Operational criteria used for defining concordance and review of all psychiatry entries for the admission. –Advantage: study not limited to initial consultation recommendations. –Included recommendations to teams with liaison attachments but compared this with teams without liaison attachments to avoid this confounder. –Small numbers for some recommendations limited reporting of results. –Operational criteria used in modified version. Unclear how the psychiatric diagnosis was judged as concordant. –This was conducted as part of a randomised controlled trial and the patients were not referred by the team but rather identified by a screening tool, which may have influenced concordance as the team did not ask for the consultation. –No operational criteria described for defining concordance. –Only a low percentage (22%) of nondrug recommendations were carried out by the referring team only, and it is unclear what type of non-drug recommendations these were. –No operational criteria for defining concordance. –This was conducted as part of a randomised controlled trial and the patients were not referred by the team but rather identified by a screening tool, which may have influenced concordance as the team did not ask for the consultation. –Very small numbers of each recommendation. –Numbers were not confined to the main recommendation, but include every occasion upon which the treatment was recommended.

4

2

4

2


Method

a


–No operational criteria for an adequate 4 inclusion to discharge summary and lack of clarity in the patient section regarding the patients' memory of either the discharge summary or a specific letter –Recall bias for patients and PCP's. –It was not reported who conducted the interviews which may have introduced response bias. –The exclusion of patients who had been admitted to a psychiatric ward after their CLP referral and those with no psychiatric diagnosis from the CLP assessment may have biased the results as it was not a full sample of patients.

PCP — primary care provider. 183

184

Table 3 Staff feedback Type of study

Karasu et al., 1977 [38]

Medical physicians rated follow up reviews and advice on management on the wards as most important. Surgical physicians rated evaluation of competence, transfer to psychiatric care and advice on medication as most important. Both groups rated functions where staff were recipients of the function least important. Physicians in rehabilitation considered almost every function as important. Residents gave lower ratings overall than directors. 36% perceived service was always-usually 92 (39%) responded, 70 (76%) Questionnaire to 240 non psychiatry Two part questionnaire sent by helpful. Evaluation and direct services to who had used the service were interdepartmental mail, firstly asking the residents and house staff and 29 patients were rated the most important included. 21 (72%) of the usefulness of the service and the second psychiatric residents of a general functions. Activities where staff were repsychiatric residents responded part was the 17 item list of functions teaching hospital in the USA. cipients of the mediating function, such as developed by Karasu et al (1977)[38]. helping staff to deal with their reactions to patients were least valued. Significant difference in importance rating of 8 out of 9 mediating functions between psychiatric and non-psychiatric residents, with those from psychiatry placing more emphasis on these items. Doctors: 32% very- 36% somewhat satisfied 400 cases were included, 297 Study primarily sought to describe use of Prospective record of 975 (total 68%), 5% not satisfied with consultadoctors and 188 nurses were new form for collecting data. Data was psychiatric consultations in a tion process, 27% unknown. included, no clear inclusion collected by the CLP staff conducting the general hospital in the USA Nurses: 32% very, 32% somewhat (total consult but the method was not clearly de- criteria. 64%), and 2% not satisfied, 37% unknown scribed. –Two independent raters conducted semi- 206 interviews were conducted. Telephone interview of the 60–70% of recommendations had a positive structured open ended interviews after the This included 48 physicians and attending physician, house officer, effect on patient care, 1–2% had a negative patient's discharge about the effect of rec- 48 house officers of the 55 head nurse and psychiatric effect. Co-raters agreed on 94% of evaluainvolved in the patient's cases. All tions. ommendations made in psychiatric conconsultant involved in the care of the 55 patients seen by the CLP ser- sults. 55 of the nurses and psychiatric vice in a general teaching hospital in –A random sample of 13% were co-rated consultants were interviewed. the USA.

Sasser and Kinzie, 1978 [9]


Van Dyke et al., 1980 [29]

Cohen-Cole and Friedman, 1982 [39]

Questionnaire to 250 physicians on inpatient services at two related general hospitals served by the one CLP service in the USA.

Method

Participation rate

97 (40%) responded. Questionnaire sent via inter-office mail to all physicians, including residents, attending physicians and directors. Questionnaire included a 17 item list of functions in 3 groups: evaluation, direct services to patients and mediating functions. These functions were rated as unimportant, important or very important.

Conclusions



–Low response rate may have biased results 4 as those who didn't respond may not consider the service favourably or not know about it. –Ratings for different functions give an indication of relative importance however, there was no comparison to other services or how effective the physicians perceived the service to be in these functions.

–Low response rate may have biased results. –Smaller group of psychiatric residents for comparison with larger group of nonpsychiatry staff. –Usefulness was only asked with one question asking for a general impression, and the functions part focussed on importance rather than effectiveness.

4

4 –No description of method, so it was not possible to assess the validity of this data. –High percentage of ‘unknown’ feedback, with no clear reason for why it was unknown. 4 –Recall bias –Excluded staff opinion on recommendations not complied with, which implies that the staff would have been biased that these recommendations would be likely to have a positive effect. –For some recommendations, there was no way of the interviewee being able to judge the effect a recommendation had on the patient, as the effect on the patient is a subjective one. –Excluded consultations to wards with liaison attachments which avoided this confounder, but would be useful to be reported separately and compared. –House staff were excluded on the basis of 4 Only 34% of house staff responded 68% were satisfied with the service Mailed 37 item questionnaire covering Questionnaire to all 407 attending low numbers which may represent a general psychological issues, importance of and therefore were excluded. 200 frequently or always. Patient disposition physicians and house staff at a difference in this group's perception of the was most valued. Formal psychiatric general teaching hospital in the USA. 18 consultation activities on a 4 point scale (70%) of attending physicians service, as they may have more day to day and satisfaction with service, including how responded and these results were evaluation and services to medical and ward staff were least valued. 72% reported contact with patients than attending analysed. satisfaction varied with individual physicians. considerable variation in satisfaction with consultants. individual consultants and 52% felt this was –Satisfaction described in general terms of satisfaction, help understanding the patient particularly common in psychiatry. and with the general contribution.


Reference

Reference Type of study

Method

Participation rate

Conclusions

Nishiura et al., 1983 [46]

Two phases of study: 1. Questionnaire sent to 87 referring physicians in a year of standard service. 2. In the following year interviews conducted of referring physicians and nurses looking after 95 referred patients in two treatment groups in a general teaching hospital in Japan.

1. Mailed questionnaires six months after consultation about useful functions of the consultation, and the patient's physical and psychiatric outcome. 2. All referred patients divided alternately into either active treatment group who were seen more than 4 times in admission, or a single contact group. Referring physicians contacted one month later and interviewed by the author regarding the perception of function of consultations.

Ozbayrak and Coskun, 1993 [40]

Questionnaire to all paediatricians and residents in paediatrics at two children's hospitals in Turkey

Paper questionnaire with 26 items including satisfaction with consultations, their confidence with dealing with patients with psychological disturbance, percentage of patient load with the need for psychiatric services and opinion of further contact with child psychiatry. Also, it included a list of 19 consultation functions derived from Karasu et al (1977)[38] for rating of importance.

Phillips et al., 1996 [41]

Questionnaire to all 45 medical staff and a slightly different questionnaire to 7 in charge nurses in an obstetric-gynaecology teaching hospital in Australia.

Paper questionnaires with 9–11 items including frequency of use of the service, how helpful it was in the patient's management, opportunities for learning, whether it was important to know more about psychological factors, difficulties in using the service and suggestions. Nurses were asked about how suggestions to refer are received by teams.

Ito et al., 1999 [42]

Questionnaire sent to all psychiatric and non-psychiatric staff in four general hospitals with a psychiatric unit in Japan.

Mailed questionnaires with questions about characteristic of responders, understanding of psychiatric services, overall satisfaction and expectation of future services.

4 –Use of the term “primary physician” and the mentioning of “walk-ins” to an outpatient service raises uncertainty regarding the referral base for this study and whether all patients were inpatients of the general hospital where the study was based. –Recall bias in both parts –Response bias as interviewer in phase 2 is part of the CLP service. –Broad description of functions of consultation as either evaluation or management. –Some patients in the single contact group required more than one review due to their clinical requirements, which meant that the two groups were not as different as initially intended. –Confounding factor of differences in the two patient groups, such as severity of illness, which may have affected the value of evaluation/management by the physicians and nurses. 4 –Unknown response rate suggests the 21% found the psychiatry service No information provided possibility of a non-representative sample satisfactory and 74% reported positive regarding how many feelings towards child psychiatry. 15% were of staff questionnaires were originally –Satisfaction was asked with one broad sent out. 121 questionnaires were not aware a psychiatry service was question rather separating this into aspects available. completed and returned: 70 Working with the patient's family, helping of care. residents and 51 paediatricians. the physician understand the psychological –There was a greater focus on the opinion of how a service could be improved and the aspects of the patient's illness, priorities of the staff rather than on evalumanagement suggestions and follow up visits were considered the most important ating the aspects of the current service. functions. Arranging transfers to psychiatry and medication advice were least important. 75% response rate for doctors, 62% of doctors had used the service in the –Small numbers surveyed, particularly as 4 100% response rate for nurses past 6 months. 86% (n = 21) of doctors follow on questions were only completed who had used the service found it very– by those who had used the service. –“Helpfulness” asked about with one broad quite helpful, and 46% (n = 24) had question, rather than asking about learned from the consultation, 83% satisfaction with different aspects of the (n = 30) felt it was important to know service. more. 6 / 7 nurses found it quite helpful, one did not answer; 3 / 7 found it difficult to suggest a referral. 4 102 (79%) of non-psychiatric staff and 154 –This study included staff perceptions of 129 non psychiatric staff (31 (56%) of psychiatric staff were either satis- overall psychiatry services including CLP. physicians, 74 nurses and 24 The results about the CLP service specifified or very satisfied with the CLP service, administration staff) 64% and 86% respectively thought it would cally were limited. 274 psychiatric staff –The lack of information about the rebe an important service in the future. AlNo information provided about though 86% of psychiatric staff felt they had sponse rates limits comment about potenresponse rates. tial bias. supported non psychiatric staff, only 54% –It is probable that administration staff were satisfied with their support. would have a different perspective on services in the hospital and so it is difficult to interpret results from a group with such different roles in the hospital. 1. 55 (63%) response rate 2. 88 (93%) physicians were available and participated in the interview, and the corresponding 88 nurses (93%) of these patients



1. No marked difference in importance of evaluation or management functions. Management was more important if patient had improved outcomes. 2. Evaluation was the most important function in the active contact group. For the single contact group, evaluation and management were considered equally important. The nurses involved in both groups considered evaluation to be most important. The authors discussed the possibility that those in the active contact group felt more confident with management due to the frequent contact with the CLP team.


(continued on next page) 185

186

Table 3 (continued) Type of study

Method

Participation rate

Conclusions

Rigatelli et al., 2001 [31]

Direct telephone interviews with PCPa's 3–5 months after patient discharge from a general teaching hospital in Italy

Ninety five PCP's were contacted by phone and asked about their global impression, objective improvement in patient's psychiatric condition and if they agreed with the psychiatric diagnosis

100% of PCP's were contacted for the 95 included patients in this aspect of the study.

Gonzalez et al., 2006 [43]

219 (74%) responded to the Questionnaire sent to 298 non- psy- Questionnaires sent via internal e-mail with 4 questions evaluating the C-L service questionnaire chiatric physicians in a general based on need, promptness and adequacy. teaching hospital in Chile.

Alhamad et al., 2006 [44]

Questionnaire sent to all 115 referring teams over a four month period in a general hospital in Saudi Arabia.

88(77%) responded to the Questionnaires with 10 parts about questionnaire knowledge and attitude towards psychiatry, usefulness of psychiatric consultation and likelihood of concordance with recommendations.

Solomons et al., 2011 [45]

Semi structured face to face interviews with consultants and senior nurses of a general teaching hospital in the UK.

Interviewers had worked in the service 2 years before. Interviews were taped, transcribed and analysed manually using the framework method of qualitative analysis. Issues raised by participants were fed iteratively in subsequent interviews until no new themes were raised.

–Excellent response rate –Recall bias. –It was not reported who conducted the interviews which may have introduced response bias. –The exclusion of patients who had been admitted to a psychiatric ward after their CLP referral and those with no psychiatric diagnosis from the CLP assessment may have biased the results as it was not a full sample of patients. –Good response rate and large numbers 80% considered the service to be highly necessary, 19% thought although necessary, surveyed. –Not all data gathered (as per methods) the service was not indispensable. was reported. 2 / 3 thought the service was adequate. –Good response rate. 62% felt psychiatric consultation was important. 56% felt psychiatric consultation –Limited description of specific questions. –Unclear how the referring teams found was almost always helpful in patient the service helpful or in which aspects of management. the service. 75% reported they would always take recommendations seriously. –Qualitative method provides richer Unanimous agreement of benefit to staff, information regarding perception of the patient and to service delivery. Described benefits included management of complex service, but this approach is not appropriate patients, use of involuntary treatment and for objective outcome measurement. –Potential for response bias due to the negotiating the psychiatry system. interviewers' previous association with service.

a

PCP — primary care provider.

25 interviews conducted: 14 senior nurses and 11 consultants

83% had a positive global impression, 60% noticed an improvement in the patient's condition and 81% agreed with the psychiatric diagnosis


Level of evidence [11] 4

4

4

N/A


Reference


187

Recently, two studies using the same variable found that earlier referral time predicted a shorter length of stay, with a greater percentage (25.7% and 22%, respectively) of the variance in length of stay attributed to timing to referral [21,22]. The timing of referral has also been compared to the time subsequent to the consultation until discharge, instead of the overall length of stay [23]. A significant association between early consultation and shorter hospitalisation was observed. However, this comparison does not eliminate the difficulty of the potential influence of anticipated length of stay by the team on timing of consultation or other confounders.

did not have a representative sample as strict exclusion criteria, such as no psychiatric diagnosis or a diagnosis of a psychotic disorder, were applied limiting the generalisability of results [31,49,50]. There was considerable variability in the type of questions asked of patients. Frequently questions lacked attention to specific aspects of the patient's care. Overall, 30–88% of patients were satisfied or found the service helpful [9,24,28,31,41,47–51]. This wide range of responses likely reflects the variability in the way this data was collected.

Concordance

Three studies followed-up patients seen by CLP services after discharge and objectively measured outcomes, as distinct from subjective questioning regarding satisfaction [14,18,52], and are described in Table 5. All of these studies used screening tools as part of their quite different methods and provide higher levels of evidence than seen in the other studies of effectiveness. There were low participation rates at follow-up for two studies (41% [14] and 42% [52]) compared with a higher rate of 72% for the third study [18]. However, the results were contradictory with one study reporting a benefit in outcome measures in patients who were seen by CLP compared with those who hadn't been referred [52] and the other two studies reporting no difference between those who had seen CLP and the control groups [14,18].

Sixteen studies evaluated concordance with advice from CLP services (see Table 2). Several studies provided only brief accounts of how concordance was measured, which limited interpretation due to the lack of transparency and objective data required to replicate the measurement [18,24–31]. This limitation was overcome when a standard operationalized method was developed [32,33]. This method was replicated in further studies which greatly improved the reliability of comparison [14,34,35]. These authors who used this method found a full concordance rate for medication recommendations of 52–80% [14,32,34,35], diagnostic action, such as investigations or psychological testing, of 33–71% [14,32,33,35] and 43–49% for accuracy of the psychiatric diagnosis recorded in the discharge summary [14,32,34,35]. This operationalized method was enhanced by applying it to all CLP reviews of the patient, with similar findings [36]. Measurements of concordance were improved further by Huyse and colleagues [37] who used a prospective design, direct observation and interviews with staff in addition to the same operationalized method for all CLP reviews. Importantly, this study measured concordance with different items, such as non-pharmacological management on the ward and organisation of staff, which other researchers had not reported. Staff feedback Thirteen studies which included staff feedback on CLP services are compared in Table 3. Staff was defined broadly to include any employee of the studied hospital as well as primary care providers, and was not limited to referring providers. Eight studies used a paper based questionnaire [9,38–44], three directly interviewed staff by telephone or face to face [29,31,45], one study used both questionnaire and interview methods [46] and another did not stipulate how the feedback was obtained [28]. Response rates were highly variable (39–100%) [9,29,31,38,39,41,43,44,46]. In the majority of studies, general satisfaction including perception of usefulness and necessity, ranged from 56–86%, [28,31,39,41–44]. A unanimous perception of benefit was reported in one study, but this may have been influenced by the use of an interviewer who had previously worked in the CLP service [45]. In contrast, much lower satisfaction rates of 21% and 36% were found in two studies, perhaps influenced by an unknown [40] and low (39%, [9]) response rate. A frequent limitation noted was the use of broad questions about satisfaction and helpfulness instead of asking about satisfaction with specific aspects of the service. Patient feedback Eleven studies reported patient feedback on CLP services (see Table 4). Five studies used interviews [31,47–50], five used questionnaires [9,24,41,44,51], and one study did not describe the method of determining patient satisfaction [28]. Participation and response rates varied from 28–100%, with lower response rates from studies using mailed questionnaires [9,24,31,41,44,47–51]. Some studies were retrospective and conducted many months after contact with the CLP service and so were subject to recall bias, but had the advantage of providing information regarding long term satisfaction [24,31,48,51]. Other studies

Follow-up studies

Discussion The literature evaluating the effectiveness of CLP services was systematically reviewed. Studies were categorised according to several of Lipowski's [4] suggested measures of effectiveness, namely, cost effectiveness and length of stay, concordance, staff feedback, patient feedback and outcomes at follow-up. The studies of cost effectiveness and length of stay included three of the four studies with a level of evidence greater than four [11]. However, the best designed was still limited by the complexity of the natural environment of hospitals and the conundrum of distilling out the effect of the CLP service without altering the natural setting with all the variables that may confound the picture [12]. If the setting is altered, for instance by screening all patients [14,18], the study is flawed for this reason, and if it is done without alteration, the potential influence of other variables in the system remains. The majority of studies of the influence of CLP services on length of stay suggest that a smaller proportion of time of the whole admission before referral to CLP services predicts a shorter length of stay. However, it is questionable whether this actually reflects CLP services influencing length of stay through earlier involvement during the admission or other factors. Unfortunately, any measurement of length of stay is limited as it would be unethical to run a randomised controlled trial of patients seen by CLP and those that are referred but not seen. Overall, the current evidence suggests that CLP services are cost effective. Despite the methodological difficulties, and the argument that an overly concentrated view on cost effectiveness overlooks the global effect of CLP services for patients, families and staff [45], this financial aspect is vital to measure given the focus on optimising resource allocation in funding general hospitals and psychiatric services. Concordance is the best developed measure of the effectiveness of CLP, having an accepted and replicable method of measurement. It is a logical starting point as the effectiveness of CLP is dependent on the referrer implementing recommendations [36], however there are likely to be other factors which influence the implementation of recommendations. This particularly relates to the inclusion of assessments in the discharge summary, which have been shown to be inaccurate tools of communication overall [53]. Most studies suggest good concordance with management recommendations, but less so for requests for investigations and communication of psychiatric opinion at discharge. Future research should extend from measuring concordance with recommendations quantifiable in the medical record [37] to measuring

188

Table 4 Patient feedback Reference

Type of study

Method

Schwab, 1967 [48]

Retrospective questionnaire of 88 consecutive patients seen over 3 months in a general teaching hospital in the USA Moses and Barzilay, Retrospective questionnaire of 1967 [24] 173 consecutive patients over a year in a general teaching hospital in Israel

Questionnaires mailed to patients with twelve questions and five point Likert scale. Two further attempts made to send questionnaire if not returned Questionnaires mailed to patients within 8– 18 months of consultation. Two further attempts made to send questionnaire if not returned. Only the third attempt guaranteed protected anonymity.

Hughson and Lyons, 1973 [49]

Patients' reactions were rated by the consultants at initial interview. A follow-up interview was conducted by an independent rater, about the relevance of consultation, whether it had been helpful and whether there was symptom relief.

Hale and Abram, 1967 [51]

Interview of 72 consecutive patients over 2 months in a general hospital, (country not specified).

Conclusions

Comments


4 –Patients were rated by observers on several measures rather than selfreporting anonymously, which introduces a strong potential for bias, particularly at the initial consultation by the CLP consultant. –There was no clear basis for the patients to rate their report of satisfaction. 4 –No clear description of who conducted –The referred group went to the doctors All 100 patients in the original this interview or how it was conducted group and the control group were significantly more often prior to approached 43 patients from each hospitalisation but not after. 35% of referred consistently, e.g. either using a questionnaire, structured questions etc, group were available and willing to patients changed doctors in the year after which limits reliability. hospitalisation, whereas 16% of the control participate (overall participation –Use of services before and after group changed doctors. rate of 87%). –Two-thirds of the referred group did not feel hospitalisation is by self-report and may the consultation had effected change in their be unreliable lives. Thirty percent were positive about the –There is no clear basis for how the satisfaction responses were rated, either by the consultation, 50% were ambivalent and 20% interviewer or directly by the respondent. regarded the consultation unfavourably, –Recall bias. which represented little change from –Doubtful validity of using a hypothetical feedback given during hospitalisation. consultation as a control. 4 –Low response rate for mailed out 48 (55%) responded to the mail out 42% reported deriving either a great deal or some benefit from the CLP service. questionnaires may have biased results. but only 34 (39%) completed the –Recall bias. survey in full.

All 50 patients participated. There was no record of consent or if patients were referred in the time frame and refused to participate.

Subjectively according to the consultant: 33% were accepting initially, which increased to 44% at end. By patient report: 29% were accepting of the consultation initially increasing to 52% at end. In the follow up review: 36% had positive feelings and 44% were satisfied.

4 –Recall bias –Low response rate may have been influenced by delayed guarantee of anonymity. –Broad reports of positive or negative responses but not clear in which aspects of their health care. 4 –Patients were rated by observers based 46% of patients were rated as initially 57 (79%) patients were followed up, 15 were excluded due to being accepting of consult. 65% felt the consult was on an interview with the initial rating discharged, significant intellectual relevant, 61% felt that it was helpful and 44% incorporated into the consultation which introduces strong potential for bias. reported symptom relief they attributed to impairment or no psychiatric –Exclusion of patients without a the consult. diagnosis psychiatric diagnosis would have biased results as these patients may have had a different sense of the service's usefulness in their care, and limits generalisability. 99 (57%) returned questionnaires, 96 (55%) had fully completed the questionnaires.

37% indicated a positive response to the consultation, 46% felt it had no influence, 2% felt it had a negative influence.


Patient's response to the consultation was rated by consultant initially and at the end of the consultation. Three categories were used to rate their impression and the patients reported attitude to the consultation. Within 24 h an independent rater conducted an interview to determine attitude and reaction to consultation. Prospective follow-up interview Patients from both the referred and control of 43 of the 50 referred patients groups were interviewed one year after hospitalisation in their homes. The control in the original study [47] compared with 43 of 50 control group were matched patients who were patients from a general teaching admitted at the same time and were contacted about a hypothetical consultation hospital in the USA in hospital.

Schwab et al., 1966 Prospective interview of 50 [47] patients referred over 6 months in a general teaching hospital in the USA

Participation rate

Reference Method

Sasser and Kinzie, 1978 [9]

Questionnaires mailed to patients with ten Retrospective questionnaire to 97 consecutive patients over a 3 questions, based on that by Hale and Abram (1967)[51]. month period in a general teaching hospital in the USA.

Koran et al., 1979 [50]

Interview of all patients referred to the service over 5 months in a general teaching hospital in the USA.


Prospective record of 400 consultations for which followup data was known in a general hospital in the USA.

Phillips et al., 1996 [41]

Retrospective questionnaire to 100 consecutive patients referred in a obstetric gynaecology teaching hospital in Australia

Rigatelli et al., 2001 Retrospective interview of 95 [31] patients out of 318 referred over 6 months to a general teaching hospital in Italy

Alhamad et al., 2006 [44]

Questionnaire given to 188 consecutive patients referred over 4 months in a general hospital in Saudi Arabia

Participation rate

Conclusions

Comments


–Low response rate. –No indication of delay time from consultation to questionnaire. –Recall bias. –No objective measure for compliance with recommendations. Patients' attitude and view of the purpose of 118 patients were referred but 58 72% were initially accepting of consultation. –Patients were rated by observers based on an interview with the initial rating were excluded due to being unable 63% felt it had been very helpful, 48% felt it the consultation were rated by the incorporated into the consultation. had been necessary. Of those who were to provide consent due to young consultants at initial interview. A follow-up dissatisfied, no suggestions for improvement –Large number of exclusions due to age or psychosis (blanket interview was conducted 24 h later, asking perceived inability to consent or refusal about the value of the consultation, the need exclusion), discharged or refusing. were made. due to psychosis may have biased results 60 (51%) were included. for the service; whether it helped staff proas these patients may be more likely to vide better care, points for improvement and provide a negative evaluation. This also attitude to the referrer. limits generalisability. –No description of method, so it was not 29% were very and 33% somewhat satisfied Study primarily sought to describe the use of 220 (55%) of patients were (total 62%) with the service, 10% not satisfied possible to assess the validity of this data. included in the data, no clear a new form for collecting data. Data was –High percentage of unknown data with and 22% unknown. reason for not including all collected by the psychiatry staff conducting no clear reason for why it was unknown. patients. the consultation but the method was not clearly described. –Moderate response rate. 44 (44%) responded. 65% felt the consultation was necessary in Mailed questionnaires with stamped self–Difficult to interpret change in attitude as their care. addressed envelopes with 9–11 questions no clear data on what the attitudes were 88% found it very or quite helpful. 52% about satisfaction with being informed of the changed their attitude towards psychiatrists earlier. consultation, need for the consultation, their –Recall bias after the consultation. attitudes towards psychiatrists before and after, how helpful it was and suggestions for service improvement. Telephone interview of patients regarding –Low response rate. 95 (30%) were able to be contacted 66% were highly satisfied, 22% were level of satisfaction with service, 3–5 months and responded. indifferent and 12% were negative about the –Use of interview may have caused positive bias due to lack of anonymity. service. after discharge –Recall bias 60% stated that they had improved in their –It was not reported who conducted the “psychic subjective state”, 33% had not interviews, which may have introduced changed and 7% had worsened however, of these 7%, only 57% reported compliance with response bias. –The exclusion of patients who had been recommendations. admitted to a psychiatric ward after their CLP referral and those with no psychiatric diagnosis from the CLP assessment may have biased the results as it was not a full sample of patients. –Very high response rate, most likely due 34% felt there was a need for a CLP service, Questionnaire given to all patients with 9 yes/ 184 (98%) responded and to the questionnaire being given instead of 38% thought their complaint might be psycompleted the questionnaire. 18 no questions covering knowledge and mailed. However, there may have been patients referred in the time frame chological. attitude towards consultation. pressure to complete if it was handed out were excluded due to time by a member of the team. constraints. –Limited questions regarding evaluation of the service, this study focussed more on attitude to psychiatry. 27 (28%) of patients responded, however only 73 questionnaires were deliverable.

38% felt hospital care was improved by the service. 38% were positive about psychiatry prior to consultation, and then 55% afterwards. 65% reported following recommendations given.

4

4

4

4

4

4


Type of study

189

190

Table 5 Follow-up outcomes Type of study

Method

Outcome measures

Participation rate

Conclusions

Limitations

Payson and Davis, 1967 [52]

Prospective case control design (observational study with dramatic effect [11])

Occupational status: occupational scale used by Hollingshead in his Two-Factor Index of Social Position. Psychiatric adjustment: three equally weighted factors — no. of psychiatric diagnoses, psychiatric incapacity and classification of psychotic/non- psychotic. Both measures equally combined for “Psycho-occupational status”.

42%

–Patients for whom referral was indicated and were referred had better levels of functioning than those for whom referral was indicated but were not referred. –Some beneficial relationship with referred patients' psychosocial adjustment at follow-up.

–There was a low participation rate 3 for the follow-up evaluations, which were sometimes delayed up to 15 months after discharge creating a potential for significant recall bias and confounding factors. –Using a psychiatric opinion to determine appropriateness for referral limits generalisability. –“Psychiatric incapacity” is not a clearly explained outcome measure


Randomised controlled trial conducted in a general teaching hospital in the USA

–Every sixth admission to hospital was screened including a psychiatric screening interview to determine mental status and assess need for referral. If recommended for referral, the treating team could choose to make the referral or not. –Patients who were naturalistically referred to the CLP service during the study were then subsequently screened in the same way and included in the study. –Six months later, all screened patients were followed-up and assessed regarding psychosocial and psychological functioning. –Experimental group and contemporaneous control group. –All patients were screened for depression, anxiety, confusion and pain (The Medical Inpatient Screening Test). If high scores then randomised to psychiatric consultation or no intervention (control group). –Three months after discharge, all patients were targeted for a survey conducted by separate research assistants with the assistance of computers

Physical functioning: activities of daily living, perceived health status, disability and work loss days, post hospital physician and drug use, financial burden and current depression and anxiety (SCL-90-R questions)

197 patients were able to be contacted from original study (41%). 150 (76%) of these patients completed the interview.

2 –Use of screening limits generalisability to most real-life CLP services. –Possible dilution of effect by the inclusion of 77 patients randomised to the intervention group in the analysis, but not receiving intervention.

Gater et al., 1998 [18]

–Prospective randomised controlled trial conducted in a general teaching hospital in the UK

Mental health was assessed using the Psychiatric Assessment Schedule. Subjective health status was assessed using the Nottingham Health Profile for both patients and their family/friend. Quality of life was assessed using the Rosser index.

159 (72%) patients participated in the follow-up assessment.

–No differences between groups in subsequent rehospitalisations, physician visits, activities of daily living, cumulative costs and charges, or anxiety and depression scores. –Experimental subjects were likely to have their next hospitalisation sooner, but this was not significant when disease severity was controlled. –No significant differences between the three groups in mental health status, subjective health status or quality of life.

–All inpatients eligible screened for psychiatric disorder using the General Health Questionnaire (GHQ-28). –209 patients met threshold and were randomly allocated to three groups: control group, “physician informed” group (treating team were made aware of screening results and could choose to refer or not) and “intervention” group who received a psychiatric assessment. –Patients were followed-up six months later and reassessed.


–Use of screening limits 2 generalisability to most real life CLP services. –There was a significant difference in employment status between the groups at follow-up, which may have confounded results. –The exclusion of typical CLP patients (those patients already receiving psychiatric care and those admitted for self-harm) would have limited generalisability.


Reference


concordance of the more qualitative aspects of the CLP service interventions such as education and staff support. The effect of the frequency of reviews and reinforcement of recommendations, which may improve concordance, would be important to study in order to quantify the optimal input from CLP services which should be factored into the cost–benefit analysis of achieved outcomes [3]. Evaluating consumer satisfaction (both staff and patients) is another important aspect of the measurement of the effectiveness of CLP as they are key stakeholders in the service and also a target of intervention. The involvement of consumers in mental health service planning is a recognised priority [54]. Although such subjective evidence is often considered less robust than objective measurement, it may provide valuable data with which to develop services [43]. The majority of studies of staff satisfaction found support for CLP services. Unfortunately, no clear method of evaluating satisfaction in this group has emerged in the literature and all studies used different approaches, often with general lines of questioning. Additionally, there is no benchmark for satisfaction and so it is difficult to determine if any services have managed to demonstrate effectiveness with this measure. Several studies examined perceptions of the importance of different CLP functions, but this is not equivalent to effectiveness in their delivery of such functions. This area requires the development of a replicable approach to assessing satisfaction, which considers specific functions of the service. The studies evaluating patient satisfaction demonstrated variable support and satisfaction with CLP services. Patient satisfaction studies were similarly limited in terms of a lack of detail regarding specific aspects of the service, an accepted method or benchmark for evaluation and the subjective nature of the data. The time point chosen for collecting feedback is important. We question whether it is meaningful to measure the effectiveness of a CLP service, which is by definition confined to the general hospital setting, several months after discharge. The planning of an integrated strategy that will be implemented by community services after discharge is the core business of a CLP service [31]. This role may be evaluated by examining outcomes of CL-referred patients at a follow-up point proximal to discharge. Objective measurements of psychosocial functioning are most valuable in determining effectiveness, as they reduce bias related to subjective reports from patients or those involved in their care [52]. However, it is questionable whether the three studies identified in the domain of follow-up outcomes in fact measured the effectiveness of other services engaged by the CLP service or enlisted subsequently (and perhaps independently), rather than the impact of the CLP service itself. All three studies examining follow-up outcomes were the only studies included which used screening as part of the method [14,18,52], which may limit generalisability and potentially affect outcomes as screening is not routinely used in CLP services. CLP services generally receive referrals from the treating team based upon perceived current individual need rather than generic screening, which may detect mental health problems not necessarily relevant to the present admission. The two patient populations may therefore be quite different, with implications for clinical course, patient outcomes and the relationship between CLP and the treating team. The pathway of screening to CLP involvement, rather than naturalistic consultations, may therefore account for the negative findings in two of the studies [14,18]. The other study, which reported positive findings, used a psychiatrist's opinion to determine the need for referral, which is also not reflective of usual practice [52]. There were nine studies which examined more than one measurement of effectiveness [9,14,18,24,28,29,31,41,44]. Most overlapped in concordance, patient and staff satisfaction [9,24,28,29,31,41,44], except for two studies which measured cost effectiveness, concordance and follow-up outcomes [14,18]. Overall, results of these studies in each category tended to track together directionally, however it would be interesting to design a study which looked at cost effectiveness and a satisfaction measure, as these could be considered competing priorities.

191

The results of the literature search over a 70 year period are telling in terms of the deficits in research in this field. Search terms were deliberately kept broad in this review in order to encompass the variety of ways that the effectiveness of CLP services may be measured. The numbers of articles in each area correlated with the ease with which these measurements are obtained, with follow-up studies being most methodologically difficult. There was a notable lack of recent articles. This may possibly reflect the fact that CLP as a discipline has presumed itself to be validated by incumbency; however this assumption does not appear to be supported by a solid evidence base, as demonstrated in this systematic review. A large proportion of articles were found by searching through references of other articles and through sources known to the authors. This is not entirely surprising given previous research which has found that the use of electronic databases may only find half of all relevant studies [55]. The broad nature of the text terms used in the literature search may have influenced the search results, however this was necessitated by the lack of appropriate subject headings in two of the three databases used and the aim of the paper which was to discover all of the ways that effectiveness in CLP may have been interpreted and measured. Once some articles had been captured, the reference lists pertaining to the relevant measure of effectiveness were reviewed and additional articles found along similar lines of enquiry. Thus, we can be fairly confident that the search was comprehensive. This observation also demonstrates the fragmented nature of the literature on effectiveness and the importance of bringing these measurements together. Limitations This review was limited by the inclusion criteria that the CLP service studied must cover an entire general hospital. This requirement was set in order to ensure that effectiveness measured pertained to a service in totality rather than one aspect or to a dedicated liaison attachment. It is possible that studies which examined a more limited area of the service, such as particular diagnostic groups or departments, can better control for confounding factors and produce more robust findings. However, the extrapolation of these findings is questionable as CLP services generally do cover whole hospitals and a wide variety of patients and need to demonstrate effectiveness in such real-world settings. There are several directions for future research. In all of the types of effectiveness measures identified in this review, there is a need for further development, particularly regarding establishing accepted methods and standardised data collection to provide firstly a body of evidence that supports effectiveness in this area and secondly benchmarks for future comparison. There is also further discussion needed about the duration of the effect of CLP and the place of follow-up patient outcome measurements and timing of patient satisfaction studies. Finally, services would ideally use more than one measure of effectiveness in order to provide a comprehensive review of various aspects of the service. Conclusion There are multiple ways to measure the effectiveness of CLP services. Each of these measures alone is insufficient to evaluate the full effect of a CLP service but all have merits, with the arguable exception of long term follow-up studies, to support an important contribution to an evaluation of effectiveness. This systematic review found a body of literature that is notably disparate and variable in measurements used to gauge the effectiveness of CLP services. Some CLP services appear to be cost effective, with earlier referral generally being associated with reduced length of stay. Concordance with management recommendations is variable. While subjective evidence from patient and staff feedback is a valuable addition, the level of evidence is currently inadequate in isolation

192


to demonstrate effectiveness. The overall impression is that many methodological flaws and a lack of consistency limit conclusions regarding the effectiveness of the discipline of CLP and comparison across services. The task of establishing accepted methods of evaluation from which to build a substantial evidence base to support the effectiveness of CLP services is far from finished. Competing interest statement The authors have no competing interests to report. References [1] Bronheim H, Fulop G, Kunkel E, Muskin PR, Schindler BA, Yates WR, et al. The Academy of Psychosomatic Medicine practice guidelines for psychiatric consultation in the general medical setting. Psychosomatics 1998;39:S8–S30. [2] Ilchef R. Diamonds in the coalface: new research in consultation-liaison psychiatry. Curr Opin Psychiatry 2006;19:175–9. [3] McKegney FP, Beckhardt RM. Evaluative research in consultation-liaison psychiatry: review of the literature: 1970–1981. Gen Hosp Psychiatry 1982;4:197–218. [4] Lipowski ZJ. Psychiatric consultation: concepts and controversies. Am J Psychiatry 1977;134:523–8. [5] Leentjens AFG, Boenink AD, Sno HN, van Schijndelb RJM Strack, van Croonenborgd JJ, van Everdingend JJE, et al. The guideline “consultation psychiatry” of the Netherlands Psychiatric Association. J Psych Res 2009;66:531–5. [6] Leentjens AFG, Boenink AD, van der Feltz-Cornelis CM. Can we increase adherence to treatment recommendations of the consultation psychiatrist working in a general hospital? A systematic review. J Psychosom Res 2010;68:303–9. [7] Andreoli PBDA, Citero VDA, Mari JDJ. A systematic review of studies of the costeffectiveness of mental health consultation-liaison interventions in general hospitals. Psychosomatics 2003;44:499–507. [8] Strain JJ, Hammer JS, Fulop G. APM task force on psychosocial interventions in the general hospital inpatient setting: a review of cost-offset studies. Psychosomatics 1994;35:253–62. [9] Sasser M, Kinzie JD. Evaluation of medical-psychiatric consultation. Int J Psychiatry Med 1978;9:123–34. [10] Centre for Reviews and Dissemination, University of York. Systematic reviews: CRD's guidance for undertaking reviews in health care. York: York Publishing Services; 2009. [11] OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evidence. Oxford Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653. [12] Parsonage M, Fossey M. Economic evaluation of a liaison psychiatry service. London UK: Centre for Mental Health; 2011. [13] Strain JJ, Lyons JS, Hammer JS. Cost offset from a psychiatric consultation-liaison intervention with elderly hip fracture patients. Am J Psychiatry 1991;148:1044–9. [14] Levenson JL, Hamer RM, Rossiter LF. A randomized controlled study of psychiatric consultation guided by screening in general medical inpatients. Am J Psychiatry 1992;149:631–7. [15] Billings EG. Value of psychiatry to the general hospital. Hospitals 1941;15:30–4. [16] Hosaka T, Aoki T, Watanabe T, Okuyama T, Kurosawa H. General hospital psychiatry from the perspective of medical economics. Psychiatry Clin Neurosci 1999;53:449–53. [17] Schuster JM. A cost-effective model of consultation-liaison psychiatry. Hosp Community Psychiatry 1992;43:330–2. [18] Gater RA, Goldberg DP, Evanson JM, Lowson K, McGrath G, Tantam D, et al. Detection and treatment of psychiatric illness in a general medical ward: a modified cost–benefit analysis. J Psychosom Res 1998;45:437–48. [19] Handrinos D, McKenzie D, Smith GC. Timing of referral to a consultation-liaison psychiatry unit. Psychosomatics 1998;39:311–7. [20] Lyons JS, Hammer JS, Strain JJ, Fulop G. The timing of psychiatric consultation in the general hospital and length of hospital stay. Gen Hosp Psychiatry 1986;8:159–62. [21] Kishi Y, Meller WH, Kathol RG, Swigart SE. Factors affecting the relationship between the timing of psychiatric consultation and general hospital length of stay. Psychosomatics 2004;45:470–6. [22] Alhuthail YR. Timing of referral to consultation-liaison psychiatry. Int J Health Sci 2009;3:165–70. [23] Ormont MA, Weisman HW, Heller SS, Najara JE, Shindledecker RD. The timing of psychiatric consultation requests: utilization, liaison, and diagnostic considerations. Psychosomatics 1997;38:38–44. [24] Moses R, Barzilay S. The influence of psychiatric consultation on the course of illness of the general hospital patient. Compr Psychiatry 1967;8:17–26. [25] Froese AP. Pediatric referrals to psychiatry: III. Is the psychiatrist's opinion heard? Int J Psychiatry Med 1977;8:295–301. [26] Billowitz A, Friedson W. Are psychiatric consultants' recommendations followed? Int J Psychiatry Med 1978;9:179–89. [27] Neill JR. Consultation evaluation: I. Psychotropic drug recommendations. Gen Hosp Psychiatry 1979;1:62–5.

[28] Taintor Z, Spikes J, Gise LH, Strain JJ. Recording psychiatric consultations: a preliminary report. Gen Hosp Psychiatry 1979;1:139–49. [29] Van Dyke C, Rice D, Pallett P, Leigh H. Psychiatric consultation: compliance and level of satisfaction with recommendations. Psychother Psychosom 1980;33:14–24. [30] Clarke DM, Smith GC. Consultation-liaison psychiatry in general medical units. Aust N Z J Psychiatry 1995;29:424–32. [31] Rigatelli M, Casolari L, Massari I, Ferrari S. A follow-up study of psychiatric consultations in the general hospital: what happens to patients after discharge? Psychother Psychosom 2001;70:276–82. [32] Popkin MK, Mackenzie TB, Callies AL. Consultation-liaison outcome evaluation system: I. Consultant-consultee interaction. Arch Gen Psychiatry 1983;40:215–9. [33] Popkin MK, Mackenzie TB, Callies AL, Hall RCW. Yield of psychiatric consultants' recommendations for diagnostic action. Arch Gen Psychiatry 1982;39:843–5. [34] Lanting RHH, Hengeveld MW. Outcome of psychiatric consultation in a Dutch university hospital. Psychosomatics 1984;25:619–22 [625]. [35] Wise TN, Mann LS, Silverstein R, Steg J. Consultation-liaison outcome evaluation system (CLOES): resident or private attending physicians' concordance with consultants' recommendations. Compr Psychiatry 1987;28:430–6. [36] Seward LN, Smith GC, Stuart GW. Concordance with recommendations in a consultation-liaison psychiatry service. Aust N Z J Psychiatry 1991;23:243–54. [37] Huyse FJ, Strain JJ, Hammer JS. Interventions in consultation/liaison psychiatry. Part II: concordance. Gen Hosp Psychiatry 1990;12:221–31. [38] Karasu TB, Plutchik R, Conte H, Siegel B, Steinmuller R, Rosenbaum M. What do physicians want from a psychiatric consultation service? Compr Psychiatry 1977;18:73–81. [39] Cohen-Cole SA, Friedman CP. Attitudes of nonpsychiatric physicians toward psychiatric consultation. Hosp Community Psychiatry 1982;33:1002–5. [40] Ozbayrak KR, Coskun A. Attitudes of paediatricians toward psychiatric consultations. Gen Hosp Psychiatry 1993;15:334–8. [41] Phillips N, Dennerstein L, Farish S. Progress and evaluation of a consultation-liaison psychiatry service to an obstetric-gynaecology hospital. Aust N Z J Psychiatry 1996;30:82–9. [42] Ito H, Kishi Y, Kurosawa H. A preliminary study of staff perception of psychiatric services in general hospitals. Gen Hosp Psychiatry 1999;21:57–61. [43] González M, Calderón J, Olguín P. In the general hospital: a doctors' perception survey. Eur J Psychiatr 2006;20:224–30. [44] Alhamad AM, Al-Sawaf MH, Osman AA, Ibrahim IS. Differential aspects of consultation-liaison psychiatry in a Saudi hospital. II: knowledge and attitudes of physicians and patients. East Mediterr Health J 2006;12:324–30. [45] Solomons LC, Thachil A, Burgess C, Hopper A, Glen-Day V, Ranjith G, et al. Quality of psychiatric care in the general hospital: referrer perceptions of an inpatient liaison psychiatry service. Gen Hosp Psychiatry 2011;33:260–6. [46] Nishiura K, Araki S, Matsushima M, Morimoto N, Ogata R, Shibata S, et al. Evaluation and management in consultation-liaison psychiatry. Folia Psychiatr Neurol Jpn 1983;37:419–34. [47] Schwab JJ, Clemmons RS, Valder MJ, Raulerson JD. Medical inpatients' reactions to psychiatric consultations. J Nerv Ment Dis 1966;142:215–22. [48] Schwab JJ. Evaluating psychiatric consultation work. Psychosomatics 1967;8:309–17. [49] Hughson B, Lyons R. Patient response to psychiatric consultation in a general hospital. Aust N Z J Psychiatry 1973;7:279–82. [50] Koran LM, Van Natta J, Stephens JR, Pascualy R. Patients' reactions to psychiatric consultation. JAMA 1979;241:1603–5. [51] Hale ML, Abram HS. Patients' attitudes toward psychiatric consultations in the general hospital. Va Med Mon 1967;94:342–7. [52] Payson HE, Davis JM. The psychosocial adjustment of medical inpatients after discharge: a follow-up study. Am J Psychiatry 1967;123:1220–5. [53] Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA 2007;297:831–41. [54] Thornicroft G, Tansella M. Growing recognition of the importance of service user involvement in mental health service planning and evaluation. Epidemiol Psichiatr Soc 2005;14:1–3. [55] Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Metaanalysis of observational studies in epidemiology: a proposal for reporting. JAMA 2000;283:2008–12. [56] Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA. Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2007;6:e1000097. [57] Popkin MK, Mackenzie TB, Hall RCW, Garrard J. Physicians' concordance with consultants' recommendations for psychotropic medication. Arch Gen Psychiatry 1979;36:386–9. [58] Popkin MK, Mackenzie TB, Hall RCW, Callies AL. Consultees' concordance with consultants' psychotropic drug recommendations: related variables. Arch Gen Psychiatry 1980;37:1017–21. [59] Popkin MK, Mackenzie TB, Callies AL. Consultees' concordance with consultants' recommendations for diagnostic action. J Nerv Ment Dis 1980;168:9–12. [60] Callies AL, Popkin MK, Mackenzie TB, Mitchell J. Consultees' representations of consultants' psychiatric diagnoses. Am J Psychiatry 1980;137:1250–3. [61] Huyse FJ, Lyons JS, Strain JJ. Evaluating psychiatric consultations in the general hospital: multivariate prediction of concordance. Gen Hosp Psychiatry 1992;14:363–9.

Evaluating the effectiveness of self-administration of medication (SAM) schemes in the hospital setting: a systematic review of the literature.

Patient-controlled hospital admission in psychiatry: A systematic review.

The phenomenology of self-mutilation in a general hospital setting.

General nurses' experiences of end-of-life care in the acute hospital setting: a literature review.

General hospital inpatient psychiatry in the 1990s: problems and possibilities.

A survey of ECT in a general hospital psychiatry unit.

Further comment on the history of general hospital psychiatry.

Economic impact of electronic prescribing in the hospital setting: A systematic review.

Fecal Carriage of Staphylococcus aureus in the Hospital and Community Setting: A Systematic Review.

Risk Factors Associated with the Requirement for Pharmaceutical Intervention in the Hospital Setting: A Systematic Review of the Literature.

Patient engagement in the inpatient setting: a systematic review.

Clinical Effectiveness of Liraglutide in Type 2 Diabetes Treatment in the Real-World Setting: A Systematic Literature Review.

Consultation-Liaison Psychiatry in the General Hospital: the Experience of UK, Italy, and Japan.

A BEME systematic review of UK undergraduate medical education in the general practice setting: BEME Guide No. 32.

The Future of Psychiatry in General Practice.

The Training of General Practitioners in Psychiatry.

The Effectiveness of Aromatherapy for Depressive Symptoms: A Systematic Review.

The cost-effectiveness of patient decision aids: A systematic review.

Psychiatry and the General Practitioner.

Psychiatry and the general internist.

Severe cutaneous reactions to drugs in the setting of a general hospital.

Complex cardiac pacing in the setting of a district general hospital: procedural success and complications.

A systematic literature review on the effectiveness of eurythmy therapy.

The effectiveness of compulsory drug treatment: A systematic review.