Hospital standardised mortality ratios--their use and misuse.

Papers Presented to the Society

Hospital standardised mortality ratios – their use and misuse

Medico-Legal Journal 2015, Vol. 83(2) 72–92 ! The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/0025817215583211 mlj.sagepub.com

Professor Sir Brian Jarman OBE United Kingdom

Keywords Mortality ratios, Hospital Standardised Mortality Ratio, mortality rates, Standardised Mortality Ratio

The President: Welcome everybody. Tonight, Professor Sir Brian Jarman, OBE, has very kindly agreed to speak to us. Brian has enjoyed a brilliant career and has the unusual mix of skills being both an academic and a man who can find practical solutions to problem. After completing his first degree at Cambridge, his National Service with the Royal Artillery gave him the opportunity to assist in planning the Allied Nuclear Defence Strategy in Western Europe. Fortunately for us, after gaining PhD in geophysics at Imperial and working for Shell, at 31 he decided to change career and study medicine. He obtained a degree in medicine from Imperial College, where of course he achieved first class honours, went to Harvard as a clinical fellow and then returned to the UK and worked as a GP for 28 years. He became a Professor of Primary Health Care and is a former President of the British Medical Association. He is an Emeritus Professor at Imperial and is a Director of the Dr Foster Unit. An international expert on the development of primary healthcare services, he has deservedly received many honours and awards, too numerous to list, and we are most fortunate that he is joining us here tonight. He is well known for developing the Hospital Standardised Mortality Ratio, which was devised to explain the differences in English hospital death rates using routinely collected data. It is now some 15 years since the publication of that paper and he is going to give us his view as to the use and misuse of the ratios. Brian. Professor Jarman: Thank you very much. (Applause)

A meeting of the Society was held at the Medical Society of London, 11 Chandos Street, Cavendish Square, London, W1G 9EB, on Thursday, 8 January 2015. The President, Ms Linda Lee, was in the Chair.

So, to add a little to Linda’s introduction, at age 31 I started on a medical career in hospitals, initially at St Mary’s and then at Harvard, and then actually I came back to the UK because our son developed an illness, acute leukaemia, when we were in Boston. We didn’t think he would survive, so we came back to look after him, and I thought I would change to general practice, which is what I did and spent 28 years in practice fairly near here. Then I was put on to the Bristol Inquiry (1999–2001) as an Inquiry panel member. I gave evidence at the Mid Staffs (Francis) Inquiry (2009–11) and was in 2000 involved in what is now the Dr Foster Unit at Imperial College.

This Hospital Standardised Mortality Ratio thing was developed for an entirely different reason initially, which was that in the ’70s and ’80s I was sort of into RAWP (Resource Allocation Working Party) and worked out various things to do with that. Then in

Downloaded from mlj.sagepub.com at Mount Royal University on July 17, 2015

Jarman

73

1988 a new set of data, the Hospital Episode Statistics, had come through, and I and others were given a sort of tabula rasa to set the resource allocation formula – the formula is similar to the one that is used now to distribute £100 million – and I wanted to try to see whether we could identify hospitals that had particular problems. So I was trying to work out a mechanism and decided that we would have to use death rates.

which was looking at a set of, in his case, 30,000 admissions in 51 hospitals, actually in New York State, and he found that in 3.7% of admissions there was an adverse medical event occurring, and that was repeated, with fewer hospitals, in England and it was found that the figure was 10.8%, which sort of tallied with us having more problems.

Now, if you actually look at death rates in English hospitals and American hospitals where I have been, you can see actually that our death rates are very much higher than the ones in American hospitals, and in fact over the age of 70/75 the death rates in English hospitals are about two and a half times the death rates in American hospitals, though they were of course paying about twice as much for the cost of their health care. That fascinated me. The data were based on about eight million admissions in each country, day cases not included. These are the actual data that it is based on. So when I was over there my professor, Howard Hiatt, had done this Harvard Medical Practice Study,

If you actually look at all of that data (and I looked at seven countries) and if you total all the deaths and relate it to diagnosis, the top diagnoses causing most deaths, as you would expect, is pneumonia No. 1 – these are the primary diagnoses – stroke No. 2, and adverse medical events is actually No. 3 in order, about half of which are preventable. So that is an important issue. When you look at that study which was published you can see the claims for negligence, which are the black ones, and the actual ones in which there was considered to be an adverse event, and it is only about a quarter of cases. A higher proportion of cases, say, in vascular surgery are actually thought to have been negligent and adverse events, whereas in neonatology there is a very small proportion. Of course, the payment is very high for those.


74

Medico-Legal Journal 83(2) And they came out in round about 1999 and 2001 and they have affected the thoughts on this subject both in England and in the States.

So when this information started to come out there were two reports from the Institute of Medicine in the United States, which a chap named Don Berwick, whom I know personally from America, was very much involved with. One was ‘‘To Err Is Human’’

Of course, looking at death rates has been important right from the time of Florence Nightingale, way back in 1863, who was quite a pioneer in this subject. So the Bristol Inquiry was one on paediatric cardiac surgery, which it considered to have been bad for 10 years, that is up until 1999, and its mortality was much higher than the national figure. It was found to be under-funded, and once somebody came in and had a look at the problem the mortality dropped within three years from 29 to 3%. It was just not properly funded; they didn’t have the proper resources.

Another is called ‘‘Crossing The Quality Chasm’’

We were three years on this inquiry and people came and gave evidence and nobody really seemed to be responsible for the quality of care in the NHS. In the end it turned out that actually it was probably the Department of Health; they admitted it on the last day of the inquiry. It was chaired by Ian Kennedy and our conclusion was that the Department of Health didn’t know what to do. We couldn’t say that the external system was inadequate; there actually wasn’t a system; so what we said was that it would


Jarman

75

really be reassuring if there were a system, because the problems that happened at Bristol could very well occur again in the NHS. This is a sort of summary of the concerns, and you can see in 1986 the Chief Medical Officer of Wales telling the CMO of England there was a problem; there was a television programme; there was an actual report by Dame Deirdre Hine about the problems, which was more or less the same as our inquiry, but nothing was done. An anaesthetist, Dr Bolsin, expressed concerns; and the first time anything happened was when Private Eye

wrote six articles, because they got hold of the information from Dr Bolsin and published it and said it was accurate. We followed this up. Our unit at Imperial College was responsible for doing it; indeed, I was one of the panel members; and you can see here that the mortality at Bristol is high in that period – Oxford is borderline and we did write a letter to them then – and then after improvements Bristol dropped to one of the lowest in the country. So obviously we decided that if you actually find a problem, correct the problem – and they were many; they didn’t do intra-operative echocardiograms, and so on, and were very understaffed, and many other things – then you can do something.

Now, when we published that follow-up and showed that Oxford had become even higher, we worked with the Oxford people for some while, for about, I think, a year or two, and their response was that 16 cardiologists and cardiac surgeons wrote to the GMC to have us struck off.

The GMC took seven months to decide, but it concluded that we had done a good thing for the populace and we had acted correctly, and so on, but it was a slightly stressful period waiting for them to decide. So we did a Clinical Case Note Review, and this was actually quite important. We had a room, twice the size


76

Medico-Legal Journal 83(2) And what we pointed out was that in fact this dataset that I had used way back to do resource allocation was the one which we used for the Bristol Inquiry; it was there and it could be used.

of this, full of the 2000 sets of notes from Bristol, and we took a random sample of 80 cases (100 operations) and we got seven panels of specialists to go through these. I think there were 49 people involved; it was a big study. Just to go through it very quickly, more or less the conclusion was that the results from the cardiac and Case Note Review were similar to the results that we got from the mortality data.

Liam Donaldson, who was CMO at the time, said that there seemed to be no system working.

One of the points made was that up until about 1990 if a doctor reported another for possibly poor care it was the doctor who reported to the GMC who would be struck off for serious professional negligence, as you know.

So what we emphasised was that really doctors are not bad people, or anything like that, but things go wrong. Often, bad outcomes are due to poor quality of services, as they thought, and these facts should be used for learning, to improve quality and safety, and so on, and that was not being done if these errors were denied. What was also important was that parents who took their young children for operations at Bristol could have been told that a unit which was one-hour drive up the motorway had a quarter of the mortality, and that would have driven up these issues. Healthcare professionals, we thought, should have a duty of candour and they should have a safety non-ex. on the Board of every Trust. This is actually an extract from the records of the hearings and the Department of Health admitted that it was their responsibility to have a system.


Jarman

77

These two external specialists who came in, Hunter and de Leval, found immense problems and it was the result of just one month’s inspection that led to the mortality dropping over three years from 29 to 3%.

You can see the drop going down on this line here, the red one, and also a general drop in the country as a whole. In fact, this is a current one which I did recently for lecturing and you can see that Bristol is now – it’s called a ‘‘funnel plot’’ – one of the lowest mortalities in the country as a result of this improvement.

So we started a unit at Imperial College. The aim was that Bristol should not occur again. It was clear that the Department of Health was not going to do the analyses, so we thought Imperial College could do them. We used mortality because, uniquely, it is recorded by law and it is not present on admission, which is important, and morbidity is present on admission quite often; you can’t measure that accurately. As it is a sensitive subject, you have got to be pretty strong on proof and ready to defend what you are saying. We


78

Medico-Legal Journal 83(2)

tried to get permission to publish the data, which was recommended in the Bristol Inquiry, but we were told that we couldn’t do so, and then eventually I got to see Simon Stevens, who was the Prime Minister’s health advisor, and he gave me permission to publish the data.

The paper was first published in 1999, as has been referred to, and it is really what has been going on in other sectors of society. The car industry uses similar things to what we do. This is an example, for example, of one hospital. This is the age-related national death rates per area; this is the number of admissions, in green, at this particular hospital; and so if you multiply them together you get the blue, which is the number of deaths you would get at each age group if the hospital had the national death rate. All you are doing is saying what would the mortality be if it had the national death rate, nothing more than that. So, in this instance, you have got a rate of 600 in 10,000, you multiply it by 1600 admissions and you get 98 expected deaths.

The HSMR is just like an SMR, the ratio of the observed to the expected deaths, the Standardised Mortality Ratio, and it is the same thing but in this instance it is not for one disease, it is for the entire hospital. We don’t say that it identifies poor quality of care; we say it is a trigger to look further; that is all we are saying.

In fact, we adjust for various other things, age, sex, admission status, and so on, so that is important. We do check every month the quality status of hospitals over the month, a whole range of factors. And, when it came to this Mid Staffs Inquiry, both the first and the second, it published this table, which I prepared with a friend, and we said that in the last three years there were more deaths than expected, about 500 more, and the range of accuracy was between 400 and 600. Now, the papers got the figure of 1000 or


Jarman

79

1200 and that was published a lot in the newspapers, but in fact it was incorrect.

At the Mid Staffs Inquiry it was mentioned that it was an incorrect figure, but during the inquiry the HSMR was mentioned about 1000 times and there were about 400 mentions of ‘‘with hindsight, we would . . .’’ etc We did this in 12 countries. This is one for the United States, plotting vertically the HSMRs, but horizontally they have data on cost, so you can see how cost is related to the adjusted death rates (HSMR).

Now, these are the mortality alerts which we send every month. This is a graph. If a patient dies it goes up a long way if it was low risk and a small distance if it was high risk, and eventually reaches a point where the number of deaths is much more and it triggers an alert, and those are the things which you have had, say, in Toyota cars since the 1970s, when Deming went there and told them how to pick up things going wrong on the production line and changing them. That is what we are trying to do with the mortality rates. This is one of the alerts which we sent to Mid Staffs. This is another one. And the False Alarm Rate is less than one in 1000. So that means that for this particular diagnosis, which is pulmonary heart disease, the hospital or Trust has got double the death rate in the last month or three months and the chance of it being a false alarm is less than one in 1000. Afterwards my colleague Paul Aylin and myself sent a letter out, and note we sent it in April 2007. We copied it to the Care Quality Commission and they actually published something or other indicating that it was an example of a mortality alert, but actually it was one which we had sent them but they’ve added their name at the top as if they had produced it. In fact, it was one which we had sent them.


80


So we followed up these ‘‘outliers’’ and this assessment by the Healthcare Commission was that it led to improvement in clinical care. And the Head of Investigations at the Healthcare Commission stated that it was these alerts that forced him to go in and start investigating the Trust. So I think we have actually seen about four eras in regulation. After the Bristol Inquiry lots of things were done. They found a problem at Mid Staffs. Ian Kennedy said that Gordon Brown’s ‘‘bonfire of the

regulators’’ affected healthcare and a lot of things were abolished and replaced by others which were not so rigorous. Then in the period when things started going wrong at Mid Staffs, when they started looking into it there were a few years of investigations, and then there was the report by Francis in 2013, and things from then have started to get quite considerably better. These are the things that happened after the Bristol Inquiry. It is all very fine and actually led to good results, but unfortunately they were disbanded. At Mid Staffs there was an awkward situation in 2007 because Mid Staffs had been put forward as a


Jarman

81

they were told the HSMR was not 127 but 101, an under-estimate of what it really was. The Healthcare Commission then decided, partly because of a patient group and the alerts, to investigate, and they found major problems.

Foundation Trust: there was Department of Health encouragement to get more Foundation Trusts. That was in March, and we published our HSMR of 127 (that is 27% above what it should be) in April, and that was inconvenient. We also sent them several alerts. The SHA employed Birmingham University to look at our methodology. When they went to the Department of Health they didn’t actually tell them they had had a high HSMR or that there was any problem with quality of care, and when it went to Monitor

Some of the evidence to the Mid Staffs Inquiry has shown that a company, CHKS, was asked to advise about coding. It was commissioned for Medway and this is the letter which CHKS wrote saying that they thought not enough cases were being coded as palliative care.


82


A year later a similar situation occurred at Mid Staffs in which their coding went from virtually 0 to 34% and the HSMR dropped, and you can see that actually in the ‘‘Trend’’.

So Mid Staffs actually had had a high HSMR from 1997, apart from one year on the borderline, and when they changed the coding, here, the HSMR went down.

If you look at the figures, you see average coding as palliative is about 3 or 4% nationally, and it had been like this for many years. The figures at Medway were very low and they suddenly increased it after this advice they had received from about 0 to 40%, the national figure being about 3%. We let it be known that we had seen this, because the HSMR dropped over that period. Some cases where patients died were effectively removed if they were coded as palliative care. After three or four years later they changed their coding back to normal and their HSMR went up.

In a report to the Cabinet, just before the Mid Staff Inquiry came out, there was a note that the Trust had received an HSMR of 127 and that it had been reduced


Jarman

83

to 101 within a short time. In fact, this was not correct. The true figure was higher. But the Cabinet were actually discussing HSMRs and they had misleading information. What was quoted was misleading.

So you can see that this is the publication with the data in three groups, the low, the average and the high groups, and then they are in alphabetical order within those groups. The Francis Report was very large, with quite a lot of pages, but the Bristol Inquiry Report was longer; I think it was about 12,000 pages.

A summary of it is that many people were harmed, and at the time of the report, on 6 February 2013, after the PMQs (Prime Minister’s Questions) on the Wednesday, instead of just being half an hour, the Prime Minister stayed on longer in the House of Commons answering questions about the Mid Staffs Inquiry, and he did two things. One was that he asked Sir Bruce Keogh to investigate the high death rate Trusts, and secondly, he said they would have Don Berwick give recommendations. Don Berwick, as I said, I knew well in the States. I was lecturing with him last month in Orlando. I worked with him. He was at Harvard. He was a paediatrician, but he then went on to be head of Medicare, which has a budget of $840 billion.

The Permanent Secretary said they’d been drawn too far into the issue of poor coding rather than seeing the mortality information as a basis for further investigation. At Mid Staffs Dr Coates, Consultant Physician in charge of Clinical Governance, said it was absolutely wrong to focus on coding as a sole cause for the abnormal HSMR. University of Birmingham: a report from independent advisors said that they used the Mid Staffordshire issue as a context for discrediting the Dr Foster methodology . . ..


84


review the question from the barrister was ‘‘Do you recognise that for a member of the public listening to that, that is to say unacceptable, to put it mildly?’’ and he agreed that he thought that probably it should have been published.

. . . and that it would have been ‘‘completely irresponsible not to aggressively investigate further’’. That is a copy from the first Mid Staffs Report. They were misled by coding issues. The College of Surgeons went in and did an invited review and, in a very good and clear report, showed major problems, but, being an invited review which had been paid for by the Trust, the Trust were not required to publish it, unfortunately. And when John Black, who was the President, gave evidence on the fact that they had not published the

The Healthcare Commission report came along and said a lot of the same things we had seen. Bruce Keogh, who is the Medical Director of the NHS, said that had they looked at HMSRs, with patient and staff surveys, they would probably have picked up the problems earlier.


Jarman

85

Actually, when the inquiry started it listed a range of organisations whose job it was to look at aspects of quality of care in the NHS. There are actually 20 of them, and basically more or less none said that Mid Staffs was of poor quality. Actually problems were found by the Commission for Health Improvement. There were other indicators, not very strong ones. Another one is that you can look at ways they deal with patients. This is the waiting times, and you see in the 10 minutes before the 4-hour waiting time the numbers of people waiting increased enormously and then dropped very low. What actually happened, it seemed from the inquiry, is that some of those people were put to a side room, and apparently sometimes left there for days. One of the points that we make is that HSMRs don’t identify unnecessary deaths; it is not possible to give a figure for unnecessary deaths; it just shows if the death rate is higher than you would expect nationally: have a look and see if there is a problem. Bruce Keogh and the statistician David Spiegelhalter said that they didn’t think that it was correct to publish league tables of HSMRs. The slides of the 2007 HSMR publication show that that they are not as league tables but in high, medium and low HSMR groups and in

alphabetical order within those groups. Some have said that, with hindsight, they might have picked up Mid Staffs from the high HSMRs.

One of the problems is that whistleblowers are not able to draw attention: also patient complaints. Of the roughly 14,000 written hospital complaints to the NHS the Parliamentary and Health Service Ombudsman only fully investigated 0.27% in 2011/12; that is 1 in 375. Whistleblowers: this is one of the things that hasn’t been changed, although there is an investigation going


86


on at the moment to review it, but the situation is roughly put that if you whistleblow you will be dismissed by the monopoly employer, the NHS. If you are found to have been wrongly dismissed and go to an Employment Tribunal and are awarded something to cover your, say, £1 million costs, you may have to sign a gagging clause to say that you will not say what went wrong. Those are not good things. There were reports commissioned by the Chief Medical Officer to Ara Darzi about the NHS and they were very critical, but they were not published. That was in 2008. Liam Donaldson then wrote an overview of those reports and it was completely damning about quality of care in the NHS. I didn’t actually know about that until a year ago, when I asked a friendly MP to ask a question and she asked a Parliamentary question and the report was denied her. She then asked under freedom of information and it was denied her again, and she only got it via the media, which is the way things work these days.

and that the Department of Health was not interested in quality. Barbara Young, who was the Chair of the Care Quality Commission said that the reason the Government didn’t like tough reports was because they were running the services that were being reported upon, and the Chair of Monitor said: ‘‘The culture of the NHS, particularly the hospital sector, I would say, is not to embarrass the minister.’’ The Minister himself said: ‘‘The impression of us all was that we would just, you know, constantly do what was meant to be the

Now, why were there these problems, people were asking the inquiry. The Chairman of the Healthcare Commission, who chaired the Bristol Inquiry, mentioned the problem of the ‘‘bonfire of the regulators’’


Jarman

87

thing that Number 10 wanted or that we were all, you know, unthinkingly piling this stuff through. We weren’t.’’ And he gave two examples when they were not. He did, however, point out in the House of Commons that they were intending to use the HSMRs as an indicator of possible problems.

The Francis final report said ‘‘the principles by which the HSE has sought to decide whether or not to involve itself in healthcare cases has led to a particularly unsatisfactory situation when placed alongside the CQC’s refusal to investigate individual cases’’. The Care Quality Commission had said that it does not investigate individual cases of quality of care, and at that stage, in 2009, we had no investigator of clinical quality complaints, and as Francis said, ‘‘This has led to a regulatory gap which must be closed’’, and attempts are being made now to close it, but we are not completely there yet.

There were other concerns about reports being suppressed, and things like that. Since then, there have been a lot of improvements. Since the Mid Staffs report came out there have been a number of improvements in quality of care.

Sir Bruce Keogh has been looking at high death rate trusts.

He had got a very detailed method of looking at it. This has now been copied by the Care Quality Commission. The whole of the Care Quality Commission board, with one exception, has been changed. These particular Trusts are the ones with high death rates. They were put into ‘‘special measures’’, most of them, by the Secretary of State. We had published, in


88


2007, the high HSMRs of most of the trusts put into special measures. One of the big things in the Mid Staffs Inquiry report was a ‘‘Duty of Candour’’ for organisations and for individuals. The Department of Health accepted the ‘‘organisations’’, not the ‘‘individuals’’.

However, there are some quite strong recommendations that have come out since November 27 last year for making quite large improvements in the regulation and quality of care by the Secretary of State. So

there is an intention in many cases and some things have been done.

With regard to whistleblowers, there is a thing currently called the ‘‘Freedom to Speak Up’’ Francis review to see whether anything can be done to help people who are seeing problems at their work. And Francis himself has said that any Chief Exec who is found guilty of suppressing whistleblowers ‘‘should be sacked; I put it bluntly like that’’. This was, I think, in the Health Select Committee.


Jarman

89

More investigations have been going on. General improvements at the Care Quality Commission, changes to the Board. And Don Berwick’s report on what is the way forward is called ‘‘A promise to learn – a commitment to act . . .’’ and it made a number of recommendations, all of which you probably would agree with.

Clear warnings; many cases; problems exist throughout the NHS. You need a support system. But fear of doing wrong is toxic to both safety and improvement. Things going wrong should be used as

an indicator for improvement being made, not to punish people. So a range of solutions he gave related to those factors, which I won’t go into now, and a number of recommendations also related to them. The final slide is a thing I picked up when Don and I were lecturing last December, and he says ‘‘Culture will trump rules, standards, and control strategies every. . . time’’. ‘‘A safer NHS will depend far more on major cultural change than on a new regulatory


90


regime’’, with which I agree. Thank you very much. (Applause) The President: Thank you very much. First question?

Discussion Mr Baird: Peter Baird, orthopaedic surgeon. Thank you very much. I remember very well how you were pilloried over the years when you first brought out this report. I worked in the same health area for some years and you were always being pilloried by medical people and other statisticians and I congratulate you from the bottom of my heart for going against them all in spite of the blitz which you have had. The problem is we are stuck in this blame culture; we’re still there. I have four children who are doctors. There is a blame culture where they work. If you whistleblow, you are gagged. Meirion Thomas, just recently in the Daily Mail, who has just been gagged, he has got to go back to work. He has to say nowt. It strikes me that is because the lawyers got involved, and the legal system thrives on blame; it can’t exist without blame. How can you separate the culture in the NHS from the problems of the legal system and the framework within which it works? Professor Jarman: That is a very good question. It is not easy to answer with a solution, but the problem, I think, is related to a political thing in which the NHS is more or less universal, it is a monopoly employer, and Bill Moyes, Chairman of Monitor said ‘‘The culture of the NHS, particularly the hospital sector, I would say, is not to embarrass the minister.’’ All three major parties have said that this current election is going to be based on the NHS: I sometimes think that all politicians want all hospitals to be better than average. It is not easy to achieve that statistically and that is a major factor, and so the managers can just be dismissed at the drop of a hat. I wrote an editorial for the BMJ in December 2012 and pointed out that the Department of

Health has many levers, of funding, of patronage and everything, over the medical profession. We have been (and I used the word) ‘‘emasculated’’, in my view. Supposing, for instance, a manager wants to get rid of a doctor; all they have to do is make a false complaint to the GMC. Even if it is found to be false, that doctor may not work again in the NHS. Maybe nothing happens to the manager. There is a whole range of factors. Now, there is currently the review by Robert Francis, ‘‘Freedom to Speak Up’’ review, and my own view of it is that it should be a public inquiry rather than just a review. I think that, by definition, these people who have been gagged, if they were to say to the review things that really happened, they could possibly be fired, but if it was a public inquiry they are required by law to speak up . . . People were reminded at the beginning of the Mid Staffs Inquiry that you have to give evidence on oath, you must give evidence if you know it’s relevant and you can go to gaol for up to 51 weeks if you prevent evidence being given to the Inquiry. It is a strong way of making sure that anyone who actually says there is a problem is able to speak out. For instance, an ex-CQC employee witness to the Mid Staffs Inquiry, who signed a compromise agreement when he left the CQC that he considered prevented him from saying anything disparaging about the CQC, did give evidence pursuant to a direction from the inquiry. After I gave this lecture, or a similar thing, a New Zealander came up to me and he said, ‘‘You know, we don’t have that problem in New Zealand because we have a lot of private practice and you just go off and do private work. We give up the NHS side of things and we work in the private sector.’’ So they are actually trying to maintain this. Now, obviously, as a GP, I don’t do private practice, but I am told there is quite a difference for a doctor working in the NHS who has trouble and goes to the private sector. In fact, the head of the NHS, Sir David Nicholson, said he would have ‘‘Stalinist’’ control from the centre, and I think he did, and in fact there was a campaign last year to force him to resign and he did resign. That to a certain extent is an indication of the power of democracy, because of the patient groups and data and social media. But I think you are absolutely right. The review by Francis was meant to come out last November. He has had so much information that it is not going to come out for a while now, and even then it won’t probably be able to cover the really deep things we really do require. So there is a movement in the right direction. The two very difficult things that we have are whistleblowing – and don’t forget more than half the doctors in the BMA say they have seen things that they would like to report, and a smaller proportion say that they have


Jarman

91

been told – this is in a BMA survey of 2000 people – that if they were to whistleblow it would be dangerous to their jobs. We are in that situation. I mean, if I report that a mortality is higher, I might have on Twitter many attacks. Twitter is a place where people can attack you. I have taken the policy that I give the evidence and you can make what you like of it. That is the evidence. If you have got better evidence, then show it to me. A very good question. The President: Bertie. Mr Bertie Leigh: I think we would all agree with your quotation from Don Berwick that culture beats standards. It necessarily follows from that that our efforts should be directed towards enhancing the culture so as to have a culture that is inclined to increase standards. Would you agree that the most important thing that we could do to our culture is to reinforce the professionalism of the clinical staff and empower the clinical staff to do better and to support them? That is the first proposition I would like you to comment on. The second one is the role of the HSMR data itself. It is common ground that the population is not going to increase exponentially, 100% of us are going to die. Whether we die in hospital or outside is presumably a function of choices which we make or are made for us, which again would depend upon alternative places where we may be cared for, such as hospices, home support, and so on. If the first thing that happens to us when we fall ill is that people shift us into hospital, then the death rate data in that hospital is going to appear to be higher than it would otherwise, and I would like to know how you can deal with that. The third proposition I would like you to comment on is that all HSMR data is going to be vulnerable to poor quality data appearing to be submitted by hospitals and other places, or data that appears to be poor quality, as we saw with the case of the Leeds Paediatric Cardiac Service, which was suspended because not your outfit but NICOR leaked data that was crude and unrefined and appeared to show a high mortality which was in fact artifactual, but it was only found out after the service had been closed. How do you think that you can improve the submission of data from the centre and avoid your service being vulnerable to poor quality data being submitted to it? Professor Jarman: Thank you. Two very good questions again. In regard to the first one about clinicians having a greater role, I pointed out in my editorial on the 19 December 2012 in the BMJ that I felt that the medical profession had been emasculated. I cannot think of many people who really have much influence with the Department of Health. The greatest influence I have had actually is via Members of Parliament and the media. The media has the problem that you can’t control what the headlines say. It is quite difficult to do

that. I haven’t given you a solution, I realise that, but I did try to put some in my editorial, which would take a long time to go through now. The second one is if you are going into hospital and you are ill, you know, that will make your HSMR higher. Well, two things. One is if you just take the case of the children, paediatric cardiac surgery: they had major difficulties with almost everything at Bristol at the time, and one inspection run by two external specialists (that is very important) lasting one month saw the problems and was able to make recommendations and after they were implemented the mortality reduced to one-tenth, more or less, or one-eighth within three years. That I think is something that we should do, and I think also that we ought to let the parents of children taking them to that hospital know that if they took them, say, to Birmingham they could have got 6% instead of 29% mortality. With regard to, say, a hospital that . . . Let’s take Walsall, which was one of the highest HSMR hospitals when we first published them in 2001. It has, I think, the highest level of admissions for alcoholic liver disease and, interestingly, its SMR (Standardised Mortality Ratio) for that diagnosis is low, even though it has the highest number, because the adjustment for the number of people it has and the mortality for that diagnosis is such that, if it is treating those patients well, its SMR is low, because the number of expected deaths is high and the number of observed deaths is low. In fact, I remember when I was in the States, I looked around New York and Manhattan at Bellevue hospital, in the south-east corner, a deprived area; it does a very good job with the patients it has and it has one of the lowest HSMRs in Manhattan, and in New York in fact. If you look at the lowest HSMRs in England they often are in the inner city deprived areas, which have very ill people going in, but they have managed and learned over the years to deal with them. So it isn’t actually true that someone very ill going into hospital may necessarily make the HSMR higher. If you have got alcoholic liver disease and you go into a hospital that is normally very good because people are dealing with it that will lower your HSMR, not raise it, and people have often said to me ‘‘We have a very elderly group of people’’ and I sometimes say ‘‘Look, if I were to remove all the elderly from your admissions and the rest of the country your HSMR would go up because you are no longer dealing with the elderly’’, or they might go down, it depends. The third question was about paediatric cardiac surgery and the NICOR report and the fact that in Leeds, but not in the other nine paediatric cardiac surgery units in England, they had not recorded weight correctly in 37½% of cases. Now, when we did the Bristol Inquiry we had 4 databases. One was the Cardiac Surgical Register actually, which was started


92


by the Master at my College at Cambridge, Sir Terence English, in 1977, and that is one we used. We also had the surgeons’ logs and we had the South-West Regional Cardiac Database, but the most important one was the Hospital Episode Statistics, in which only less than 1% of cases were not included, and the other ones, including the one that is used by NICOR, which is CCAD (Central Cardiac Audit Database), is one which we decided we would not use, and we did not use it because we knew that factors such as the one you mentioned were likely to happen. In the event, it is unreliable if there are missing data, but the Department of Health likes to use it. Now, that data is held by HQIP, as it is called, which has a number of clinical databases, and I read somewhere that there have been 52 applications for data, of which 50 have been granted. We, I think, probably are some of the experts on paediatric cardiac surgery and the Bristol Inquiry certainly chose Imperial College to do the Bristol Inquiry. We have applied twice to NICOR/HQIP to get that data and we have been refused twice. The first time it was said they didn’t think it would be good for patients’ families. The second time we were told we didn’t have permission. We actually had written permission, which we pointed out. They were not going to let us have it, so at the beginning of last year I actually said ‘‘The reason you are not letting us have it is because you will not publish the NICOR data . . .’’ – they refused to publish it – ‘‘. . . but I’ve got our data and what it shows is that you are proposing closing the hospitals with the low death rates and keeping open Alder Hey, which has a high death rate, and I feared that might be for political reasons and not the death rates’’. I totally agree with what you are saying. The bad data, which is based on things like measurement of weight in this instance, which was missing in 37½% of cases in one unit, is not reliable, but actually when it was checked with our overall data, which actually, incidentally, also is one year more up to date, it showed that it was a very similar result. We are tied up with politics in the NHS, unfortunately, and it has come out in all these inquiries. One of the biggest recommendations from the Bristol Inquiry is that people should be honest. We think they should. Three very excellent questions. I have had four very excellent questions, in fact, tonight. Dr Daniel Haines: Sir Brian, you have spoken most eloquently about SMRs in National Health Service hospitals. Have you been able to look at the private health industry and its SMRs to see how they compare?

And a secondary small question: I was delighted at your reference to Private Eye. Are you monitoring what the M.D. section of Private Eye writes on a regular basis? Professor Jarman: Well, the person who actually got them up to date is Andrew Bousfield but Phil Hammond [‘‘M.D.’’ at Private Eye] writes the thing and one of them contacted me and they published one or two reports referring to things I’d written. Actually I think, if anyone should be getting honours, they’re the ones who should be getting the honours rather than some of the others on the Honours List. But I try to just say it like it is. I haven’t completely answered your question, have I, the first part of which was . . .. Dr Daniel Haines: The private sector. Professor Jarman:. . . the private sector, yes. We have tried to analyse it certainly. It is legally, I think, required that private hospitals do record the same key data as the NHS, and when I have gone on to Twitter the people responsible for it have claimed that they do record the data, but in fact, when we have looked at it, it hasn’t been of sufficiently high quality in all the private hospitals for us to use it. I know we did do one with fractured neck of femur once [for private patients in NHS hospitals] and they came out well with fractured neck of femur. We didn’t publish because we weren’t sure of the quality of that data. So I don’t actually think that if we were to get their data they would show up badly. They tend to do a little bit of the ‘‘slick and quick’’ procedures, with surgical planned admissions, and so on, which have a relatively low number of deaths, luckily, so that the data will probably have wide confidence intervals. I think there may be only one private hospital with more than 200 beds in this country, which is the Wellington; I think it is the only one. So they won’t necessarily have data which is like you get in a large NHS hospital with very narrow confidence intervals of accuracy, but I am sure if they had major problems or were a particularly good hospital we would be able to pick that up. But they are required to have it, though nobody has actually insisted that they do so, as far as I know. The President: Thank you. I would like to thank you on behalf of the Society for a most excellent talk. In the talk you concentrated on where the alerts weren’t acted on, but the counter side would have been, say, where alerts have been acted on, and we would like to thank you for that also. Professor Jarman: Thank you.


Wound swab use and misuse at a regional general hospital.

Including post-discharge mortality in calculation of hospital standardised mortality ratios: retrospective analysis of hospital episode statistics.

Diuretics. Use and misuse.

Life years saved, standardised mortality rates and causes of death after hospital discharge in out-of-hospital cardiac arrest survivors.

Hospital standardised mortality rates should not be used to make interhospital comparisons.

Multivitamins : Use or Misuse?

Use and misuse of condoms.

Use and misuse: Antidepressants in general practice.

Letter: Use and misuse of cryoprecipitate.

Lipid Use and Misuse by the Heart.

[The hospital service misuse problem in orthopedics].

#Schizophrenia: Use and misuse on Twitter.

Personality disorders, alcohol use, and alcohol misuse.

Narcotic use and misuse in Crohn's disease.

The use and misuse of psychometric models.

Improving hospital weekend handover: a user-centered, standardised approach.

Hospital doctors' career structure and misuse of medical womanpower.

The Use and Misuse of Sedatives and Hypnotic Drugs.

Prescription opioid use: Patient characteristics and misuse in community pharmacy.

Editorial: Use and misuse of metal chelation therapy.

The use (and misuse) of English in urological papers.

New teaching in pathology: use and misuse of IPALS.

Use and misuse of statistical significance in survival analyses.

Use and misuse of automobile child restraint devices.