916
Examining the examiners SiR,—Your interest in postgraduate medical examinations (Feb 24, p 443) deserves repayment by an outline of what this college is doing to ensure
the competence of its fellows. The examination system is
only one component; others are the inspection and recognition of hospitals for training and the organisation of postgraduate educational activities. The College of Anaesthetists has been trying for some time to "de-mystify" its examinations, and is about to publish a comprehensive training and assessment guide for candidates. It has also done a survey of all candidates presenting in 1988 for the final part of the examination in an attempt to identify of failure, this will be published in the British Journal of Anaesthesia. Although some criticisms emerged, notably in relation to the method of announcing the results, an average of 92% of candidates who responded were satisfied with the arrangements for the four elements of the examination. The College uses multiple choice questions (MCQ), written answers, clinical examinations (with or without real patients), and oral tests. Their use is constantly refined, increasingly structured formats being used to improve standardisation and reliability, but the overall mix of the methods of assessment is, we feel, basically correct. The MCQ is probably the most objectively reliable test, but has poor validity in terms of the assessment of competence to practise anaesthesia, intensive care, and pain relief. It is a good way of placing candidates in rank order, but the temptation to use norm referencing has to be resisted. Discriminating questions, whose previous performance is known, are used to compare candidates with previous cohorts and to draw grading bands. Written answers, clinicals, and orals are probably the more valid for assessing competence. Although they are less objective they can be made acceptably reliable by careful structuring and training of examiners. The performances of all parts of the examination and of individual examiners are carefully audited. The objectively structured clinical examination, increasingly used at undergraduate level, is being actively considered by this College. Your suggestion that the improving quality of medical school entrants should be reflected in improved pass rates fails to take into account doctors who may not be intellectually able to pass our fellowship examination but who sit it repeatedly. The College of Anaesthetists does not limit the number of attempts and some candidates have sat our final examination twenty times. In 1989 we introduced a maximum of six attempts at the part I, with compulsory counselling after four. Pass rates, which are published, are rising but are probably still too low, and they can best be improved by attempting to match training to assessment. It is important to ensure that criteria are closely related to clinical competence; selection processes must be adequate and clinical experience must be satisfactory; trainers must be trained and given as much information as possible about the standard of competence required. common causes
College of Anaesthetists, 35-43 Lincoln’s Inn Fields, London WC2A 3NP, UK
D.
J. HATCH,
Chairman, Examinations Committee
Unsigned editorials SiR,—Your Feb 24 editorial on Royal College examinations has attracted attention, in part because of the apparent paradox that while calling for greater openness from the colleges, the editorial itself was unsigned. The whiff of hypocrisy makes an easy target for superficial criticism, and merits a rehearsal of the arguments for unsigned editorials. The principle of not signing editorials is well recognised in newspaper journalism; do critics who read The Times, Guardian, or Daily Telegraph complain to the editors of those papers about anonymous editorials? The President and chief examiner of the Royal College of Psychiatrists (March 24, p 730) confuse an editorial with a signed paper. These beasts are of different species. Papers are unsolicited, whereas editorials are commissioned. A paper carries the personal views of the authors, and its substantive content is not edited. An editorial represents the persona of the journal, and the original draft may be substantially altered by the editor. Editorials and papers
also differ for the writer. Signed papers are attributed to the author, who receives credit in forms such as the citation rate; the principal reward for an editorialist for what can be a large amount of work is the private knowledge of a job well done. Anonymity may sometimes result in abuse-but that is not unknown in signed articles. Nor should it be assumed that when reading an unsigned piece the staff of The Lancet immediately take leave of their critical faculties and publish anything that flops through their letterbox. Indeed since the journal’s reputation partly depends on its editorials they are perhaps considered more carefully than signed papers. Why do those wishing to know the identity of editorialists need that information? The truth or falsity of a argument does not depend upon the person stating it. Knowledge of identity only deflects from the important questions and allows potential ad hominem criticism. "Hunt the author" is a popular party game, but must not be confused with serious discussion. When Richard Wakeford, who has a special interest in medical education, criticised in a signed piece the quality of the Society of Apothecaries’ examinations" the Master noted "Criticism of our examination is particularly resented when it comes from a medically unqualified and uninformed source"2 while another correspondent began his letter with "Who is Richard Wakeford ... ?.3 Academic Department of Psychiatry, St Mary’s Hospital Medical School London W2 1NY, UK
I. C. MCMANUS
1. Wakeford RE. LMSSA: a back door entry into medicine? Br Med J 1987; 294: 890-91. 2. Southwood WFW. LMSSA: a back door entry into medicine? Br MedJ 1987; 294: 1035. 3. Whimster WF. LMSSA: a back door entry into medicine? Br Med J 1987; 294: 1285.
Homeless and mentally ill SiR,—The high prevalence of mental illness amongst the homeless is well established.",2 However, the debate on how best to respond seems to have become sidetracked into a re-evaluation of community care.",3,4 This misses the point: the homeless mentally ill constitute a large group lacking access to a full range of psychiatric services flexible enough to respond to their needs. These services receive little or no specific funding for the homeless because resources are allocated on the basis of census information. Under the new provisions of the government white-paper every district health authority will receive an allocation to provide a service for its own residents.5 Residency will be determined by the patient’s own perception of where he or she is currently living or, if homeless, was last resident. A purely subjective definition will thus carry
significant funding implications.
We have done an audit of an acute inpatient service on one day last month. All acute psychiatry wards in an inner city district were visited by N. R. F. who collected data on age, sex, working diagnosis, mode of referral, and residential status. Of the 87 inpatients 33 (38%) had no permanent UK home. Interviews revealed that on the new residency criteria, 10 of these 33 would become the financial responsibility of another health authority, II would continue under the current authority, 3 were foreign residents, and 9 could not give a specific reply, "London in general"
being a typical response. The homeless group tended to present with severe psychotic illness as emergencies-via the accident-and-emergency department, admissions under the Mental Health Act, or from the criminal justice system (table). These modes of referral for severe illness imply that the services providing early intervention and aftercare for the resident population have little impact on the homeless group. Nor is there much incentive to improve the service to the homeless for to do so might uncover even greater need and place yet more strain on services to the local resident population. Psychiatric services to the homeless are ignored in most funding calculations. The white-paper proposals go some way towards finding a solution but the rules may be hard to implement and vulnerable patients may find themselves under pressure to give the "right" answers. Moreover, a significant group of long-term homeless mentally ill cannot recall a place of last residence. The
917
AUDIT OF ACUTE INPATIENT PSYCHIATRIC SERVICE
government-supported documentation centre on alternative to provide scientific information and data on alternative and complementary medicine to the professions, to scientists, and to the general public.
medicine) aims
Whatever the area of medicine, Schoones’ view that more than source of information should be used is sensible. The application of information technology" to information services has many advantages but these do not include simplification of information retrieval. Effective searching is more complex than many suppose. one
Medical Information Service, British Library Document Supply Centre, Boston Spa, Wetherby, West Yorkshire LS23 7BQ, UK
1. Pentelow GM. New
*Refers to admission under Mental Health Act sections.
homeless originate nationwide but tend to gather in specific areas." The logical solution would be to provide central funding for the homeless so that they can have full access to comprehensive psychiatric services in the inner cities. Clinicians and managers alike could then turn their attention to the needs of the homeless mentally ill without the fear that by so doing they will jeopardise the service they can provide to the resident
population. Department of Psychiatry, Middlesex Hospital, London W1 N 8AA, UK
NIGEL R. FISHER STUART W. TURNER ROBERT PUGH
M, Tobiansky RI, Hollander D, Ibrahami S. Psychosis and destitution at Christmas 1985-88. Lancet 1989; ii: 1509-11. Marshall M. Collected and neglected: are Oxford’s hostels for the homeless filling up with disabled psychiatric patients? Br Med J 1989; 299: 706-09. Whitehead T. Closure of psychiatric hospitals. Lancet 1990; i: 172-73. Lethem KR, Pugh CR. Psychiatric illness among the homeless. Br Med J 1989; 299: 1101. Department of Health. Working for patients: working party no 11 (annex 12). London: HM Stationery Office, 1989: 33-35.
1. Weller 2. 3. 4. 5.
Information data bases SiR,—Dr Schoones (Feb 24, p 481) comments on the retrieval of journal literature. The apparently not uncommon supposition that a search of ’Medline’ provides more or less a
medical
complete search of the medical literature is mistaken. About 3000 journals are indexed for Medline. The coverage is international, though there may be some bias towards US journals. Of the titles currently received by the British Library Document Supply Centre about 10 000 are relevant in some way to medicine. Acquisition policy in the UK means that all probably contain substantive material. The US National Library of Medicine takes about 22 000 periodicals but some will be of local interest or of a type with little or no substantive content. These figures suggest that Medline may include up to 40% of the world’s medical literature. This may be the most important 40% but significant material is likely to be excluded. ’Embase’ covers about 5000 journal titles but many are on Medline and a greater proportion are indexed selectively (ie, only items judged relevant by
indexers are taken). These considerations
not only reinforce Schoones’ recommendation about using more than one source but also imply that to create a single comprehensive database for medical journal literature would be a formidable task. Complementary medicine is neglected by both the above two data bases. The British Library’s medical information service has been trying to bridge this gap by compiling a database in these areas and producing a monthly index, listing journal publications many of which are not covered by other information services. This service is provided in cooperation with the Research Council for Complementary Medicine. In the Netherlands, IDAG (a
DAVID ROBERTS
technology in medical libraries. Br MedJ 1989; 298: 907-08.
Scottish health minister breaks the law SIR,-Dr Crawford (March 10, p 609) has been carefully "disinformed". I myself have encountered remarkable coyness on the part of officialdom with respect to the ultra vires position; to be told that "there’s nothing in the Act to say how often SHSPC [Scottish Health Service Planning Council] should meet" verges on the fatuous. I do not believe that there would be no objection to a statutory body charged with "keeping the NHS [National Health in Scotland under review" never meeting. There would be "no difficulty", SHSPC was advised, about it having its last meeting in June, 1989. After that meeting a circular letter signed by Mr Forsyth stated that he had abolished SHSPC. A Scottish Office news release on Oct 4, 1989, expressly indicated that SHSPC had been replaced by the Scottish Health Service Advisory Council. This new body, having been made subservient (in its remit) to the will of the Minister, is not a creation of Parliament but a mere "creature of the Department" that can be killed off at any moment suitable to the Minister and his officials. It has no statutory authority and certainly no statutory right "to advise on its own initiative". After consulting colleagues at the law faculty at Edinburgh University, I criticised these unlawful acts in the Scotsman newspaper and called for a judicial review. Since then (post hoc, ergo propter hoc) SHSPC and the national consultative committees have been "suspended"; but suspension sine die is simply constructive abolition. Unfortunately, there has been no Harry Keen in Scotland. With the connivance of officialdom (which had its own shabby reasons" for wanting to rid itself of SHSPC) a Minister of the Crown, now busy lecturing the Scottish people and some fellow MPs about the need to obey the law and pay their poll tax, has himself without any real opposition from those concerned driven a coach and horses through the British Constitution, in order to neutralise at a crucial moment professional opposition to policies for the NHS which are
Service]
as
half-baked as they are irresponsible?
Mr Forsyth knew only too well what he was about; but Scotland’s doctors, nurses, paramedical staff and others have all been hoodwinked into surrendering during those months when it most mattered their statutory right (won not without a heroic struggle 20 years ago) to advise on their own initiative. In these squalid times, no governmental manoeuvre is too disreputable to be ruled out; and sometimes discretion is the better part of valour. On this occasion, however, in view of opinion poll findings, valour migh have been the better part of discretion. Gogarburn Hospital Voluntary Association, Gogaburn Hospital, Edinburgh EH12 9BJ, UK
DRUMMOND HUNTER, Chairman
1. Hunter TD. Close encounters of the bureaucratic kind. Political
Quart 1987;
April-June. RB, von Otter C. Public competition 1989; 11: 43-55.
2. Saltman
versus
mixed markets. Health
Policy
Examining the examiners SiR,—Your interest in postgraduate medical examinations (Feb 24, p 443) deserves repayment by an outline of what this college is doing to ensure
the competence of its fellows. The examination system is
only one component; others are the inspection and recognition of hospitals for training and the organisation of postgraduate educational activities. The College of Anaesthetists has been trying for some time to "de-mystify" its examinations, and is about to publish a comprehensive training and assessment guide for candidates. It has also done a survey of all candidates presenting in 1988 for the final part of the examination in an attempt to identify of failure, this will be published in the British Journal of Anaesthesia. Although some criticisms emerged, notably in relation to the method of announcing the results, an average of 92% of candidates who responded were satisfied with the arrangements for the four elements of the examination. The College uses multiple choice questions (MCQ), written answers, clinical examinations (with or without real patients), and oral tests. Their use is constantly refined, increasingly structured formats being used to improve standardisation and reliability, but the overall mix of the methods of assessment is, we feel, basically correct. The MCQ is probably the most objectively reliable test, but has poor validity in terms of the assessment of competence to practise anaesthesia, intensive care, and pain relief. It is a good way of placing candidates in rank order, but the temptation to use norm referencing has to be resisted. Discriminating questions, whose previous performance is known, are used to compare candidates with previous cohorts and to draw grading bands. Written answers, clinicals, and orals are probably the more valid for assessing competence. Although they are less objective they can be made acceptably reliable by careful structuring and training of examiners. The performances of all parts of the examination and of individual examiners are carefully audited. The objectively structured clinical examination, increasingly used at undergraduate level, is being actively considered by this College. Your suggestion that the improving quality of medical school entrants should be reflected in improved pass rates fails to take into account doctors who may not be intellectually able to pass our fellowship examination but who sit it repeatedly. The College of Anaesthetists does not limit the number of attempts and some candidates have sat our final examination twenty times. In 1989 we introduced a maximum of six attempts at the part I, with compulsory counselling after four. Pass rates, which are published, are rising but are probably still too low, and they can best be improved by attempting to match training to assessment. It is important to ensure that criteria are closely related to clinical competence; selection processes must be adequate and clinical experience must be satisfactory; trainers must be trained and given as much information as possible about the standard of competence required. common causes
College of Anaesthetists, 35-43 Lincoln’s Inn Fields, London WC2A 3NP, UK
D.
J. HATCH,
Chairman, Examinations Committee
Unsigned editorials SiR,—Your Feb 24 editorial on Royal College examinations has attracted attention, in part because of the apparent paradox that while calling for greater openness from the colleges, the editorial itself was unsigned. The whiff of hypocrisy makes an easy target for superficial criticism, and merits a rehearsal of the arguments for unsigned editorials. The principle of not signing editorials is well recognised in newspaper journalism; do critics who read The Times, Guardian, or Daily Telegraph complain to the editors of those papers about anonymous editorials? The President and chief examiner of the Royal College of Psychiatrists (March 24, p 730) confuse an editorial with a signed paper. These beasts are of different species. Papers are unsolicited, whereas editorials are commissioned. A paper carries the personal views of the authors, and its substantive content is not edited. An editorial represents the persona of the journal, and the original draft may be substantially altered by the editor. Editorials and papers
also differ for the writer. Signed papers are attributed to the author, who receives credit in forms such as the citation rate; the principal reward for an editorialist for what can be a large amount of work is the private knowledge of a job well done. Anonymity may sometimes result in abuse-but that is not unknown in signed articles. Nor should it be assumed that when reading an unsigned piece the staff of The Lancet immediately take leave of their critical faculties and publish anything that flops through their letterbox. Indeed since the journal’s reputation partly depends on its editorials they are perhaps considered more carefully than signed papers. Why do those wishing to know the identity of editorialists need that information? The truth or falsity of a argument does not depend upon the person stating it. Knowledge of identity only deflects from the important questions and allows potential ad hominem criticism. "Hunt the author" is a popular party game, but must not be confused with serious discussion. When Richard Wakeford, who has a special interest in medical education, criticised in a signed piece the quality of the Society of Apothecaries’ examinations" the Master noted "Criticism of our examination is particularly resented when it comes from a medically unqualified and uninformed source"2 while another correspondent began his letter with "Who is Richard Wakeford ... ?.3 Academic Department of Psychiatry, St Mary’s Hospital Medical School London W2 1NY, UK
I. C. MCMANUS
1. Wakeford RE. LMSSA: a back door entry into medicine? Br Med J 1987; 294: 890-91. 2. Southwood WFW. LMSSA: a back door entry into medicine? Br MedJ 1987; 294: 1035. 3. Whimster WF. LMSSA: a back door entry into medicine? Br Med J 1987; 294: 1285.
Homeless and mentally ill SiR,—The high prevalence of mental illness amongst the homeless is well established.",2 However, the debate on how best to respond seems to have become sidetracked into a re-evaluation of community care.",3,4 This misses the point: the homeless mentally ill constitute a large group lacking access to a full range of psychiatric services flexible enough to respond to their needs. These services receive little or no specific funding for the homeless because resources are allocated on the basis of census information. Under the new provisions of the government white-paper every district health authority will receive an allocation to provide a service for its own residents.5 Residency will be determined by the patient’s own perception of where he or she is currently living or, if homeless, was last resident. A purely subjective definition will thus carry
significant funding implications.
We have done an audit of an acute inpatient service on one day last month. All acute psychiatry wards in an inner city district were visited by N. R. F. who collected data on age, sex, working diagnosis, mode of referral, and residential status. Of the 87 inpatients 33 (38%) had no permanent UK home. Interviews revealed that on the new residency criteria, 10 of these 33 would become the financial responsibility of another health authority, II would continue under the current authority, 3 were foreign residents, and 9 could not give a specific reply, "London in general"
being a typical response. The homeless group tended to present with severe psychotic illness as emergencies-via the accident-and-emergency department, admissions under the Mental Health Act, or from the criminal justice system (table). These modes of referral for severe illness imply that the services providing early intervention and aftercare for the resident population have little impact on the homeless group. Nor is there much incentive to improve the service to the homeless for to do so might uncover even greater need and place yet more strain on services to the local resident population. Psychiatric services to the homeless are ignored in most funding calculations. The white-paper proposals go some way towards finding a solution but the rules may be hard to implement and vulnerable patients may find themselves under pressure to give the "right" answers. Moreover, a significant group of long-term homeless mentally ill cannot recall a place of last residence. The
917
AUDIT OF ACUTE INPATIENT PSYCHIATRIC SERVICE
government-supported documentation centre on alternative to provide scientific information and data on alternative and complementary medicine to the professions, to scientists, and to the general public.
medicine) aims
Whatever the area of medicine, Schoones’ view that more than source of information should be used is sensible. The application of information technology" to information services has many advantages but these do not include simplification of information retrieval. Effective searching is more complex than many suppose. one
Medical Information Service, British Library Document Supply Centre, Boston Spa, Wetherby, West Yorkshire LS23 7BQ, UK
1. Pentelow GM. New
*Refers to admission under Mental Health Act sections.
homeless originate nationwide but tend to gather in specific areas." The logical solution would be to provide central funding for the homeless so that they can have full access to comprehensive psychiatric services in the inner cities. Clinicians and managers alike could then turn their attention to the needs of the homeless mentally ill without the fear that by so doing they will jeopardise the service they can provide to the resident
population. Department of Psychiatry, Middlesex Hospital, London W1 N 8AA, UK
NIGEL R. FISHER STUART W. TURNER ROBERT PUGH
M, Tobiansky RI, Hollander D, Ibrahami S. Psychosis and destitution at Christmas 1985-88. Lancet 1989; ii: 1509-11. Marshall M. Collected and neglected: are Oxford’s hostels for the homeless filling up with disabled psychiatric patients? Br Med J 1989; 299: 706-09. Whitehead T. Closure of psychiatric hospitals. Lancet 1990; i: 172-73. Lethem KR, Pugh CR. Psychiatric illness among the homeless. Br Med J 1989; 299: 1101. Department of Health. Working for patients: working party no 11 (annex 12). London: HM Stationery Office, 1989: 33-35.
1. Weller 2. 3. 4. 5.
Information data bases SiR,—Dr Schoones (Feb 24, p 481) comments on the retrieval of journal literature. The apparently not uncommon supposition that a search of ’Medline’ provides more or less a
medical
complete search of the medical literature is mistaken. About 3000 journals are indexed for Medline. The coverage is international, though there may be some bias towards US journals. Of the titles currently received by the British Library Document Supply Centre about 10 000 are relevant in some way to medicine. Acquisition policy in the UK means that all probably contain substantive material. The US National Library of Medicine takes about 22 000 periodicals but some will be of local interest or of a type with little or no substantive content. These figures suggest that Medline may include up to 40% of the world’s medical literature. This may be the most important 40% but significant material is likely to be excluded. ’Embase’ covers about 5000 journal titles but many are on Medline and a greater proportion are indexed selectively (ie, only items judged relevant by
indexers are taken). These considerations
not only reinforce Schoones’ recommendation about using more than one source but also imply that to create a single comprehensive database for medical journal literature would be a formidable task. Complementary medicine is neglected by both the above two data bases. The British Library’s medical information service has been trying to bridge this gap by compiling a database in these areas and producing a monthly index, listing journal publications many of which are not covered by other information services. This service is provided in cooperation with the Research Council for Complementary Medicine. In the Netherlands, IDAG (a
DAVID ROBERTS
technology in medical libraries. Br MedJ 1989; 298: 907-08.
Scottish health minister breaks the law SIR,-Dr Crawford (March 10, p 609) has been carefully "disinformed". I myself have encountered remarkable coyness on the part of officialdom with respect to the ultra vires position; to be told that "there’s nothing in the Act to say how often SHSPC [Scottish Health Service Planning Council] should meet" verges on the fatuous. I do not believe that there would be no objection to a statutory body charged with "keeping the NHS [National Health in Scotland under review" never meeting. There would be "no difficulty", SHSPC was advised, about it having its last meeting in June, 1989. After that meeting a circular letter signed by Mr Forsyth stated that he had abolished SHSPC. A Scottish Office news release on Oct 4, 1989, expressly indicated that SHSPC had been replaced by the Scottish Health Service Advisory Council. This new body, having been made subservient (in its remit) to the will of the Minister, is not a creation of Parliament but a mere "creature of the Department" that can be killed off at any moment suitable to the Minister and his officials. It has no statutory authority and certainly no statutory right "to advise on its own initiative". After consulting colleagues at the law faculty at Edinburgh University, I criticised these unlawful acts in the Scotsman newspaper and called for a judicial review. Since then (post hoc, ergo propter hoc) SHSPC and the national consultative committees have been "suspended"; but suspension sine die is simply constructive abolition. Unfortunately, there has been no Harry Keen in Scotland. With the connivance of officialdom (which had its own shabby reasons" for wanting to rid itself of SHSPC) a Minister of the Crown, now busy lecturing the Scottish people and some fellow MPs about the need to obey the law and pay their poll tax, has himself without any real opposition from those concerned driven a coach and horses through the British Constitution, in order to neutralise at a crucial moment professional opposition to policies for the NHS which are
Service]
as
half-baked as they are irresponsible?
Mr Forsyth knew only too well what he was about; but Scotland’s doctors, nurses, paramedical staff and others have all been hoodwinked into surrendering during those months when it most mattered their statutory right (won not without a heroic struggle 20 years ago) to advise on their own initiative. In these squalid times, no governmental manoeuvre is too disreputable to be ruled out; and sometimes discretion is the better part of valour. On this occasion, however, in view of opinion poll findings, valour migh have been the better part of discretion. Gogarburn Hospital Voluntary Association, Gogaburn Hospital, Edinburgh EH12 9BJ, UK
DRUMMOND HUNTER, Chairman
1. Hunter TD. Close encounters of the bureaucratic kind. Political
Quart 1987;
April-June. RB, von Otter C. Public competition 1989; 11: 43-55.
2. Saltman
versus
mixed markets. Health
Policy
