Brifish Jottrnal of Medical Editcarion, 1975, 9, 91-97
How should w e select? - a sociologist‘s view FETER SHELDRAKE’ E d i n b u r g h University
C e n t r e for Research in t h e E d u c a t i o n a I S c i e n c e s ,
K e y words *COLLEGE ADMISSION TEST ‘SCHOOLS, MEDICAL EDUCATION, MEDICAL, UNDERGRADUATEINTELLIGENCE TESTS PERSONALITSCURRICULUM STUDENT DROPOUTS ENGLANDSCOTLANDUNITEDSTATES
especially earlier on in the medical school course). A good and recent example of a study of this kind comes from the work of Jones and McPherson, looking at school record and student performance in the University of Edinburgh (Jones and McPherson, 1974). When other factors are added to school grades, correlations can be squeezed up to values of 0.5 or 0.6 (and even higher in the case of the Jones/ Selection and prediction McPherson study). Abercrombie’s; study of Selection is concerned with the process of identiselection and performance of architecture stufying successful applicants - a process that has dents offers comparable data of this kind three main concerns. (Abercrombie, Hunt, and Stringer, 1969). and 1. The nature of the pool of applicants from Gough’s paper in Coombs and Vincent’s inwhich the selection is to be made. valuable collection is an excellent summary of 2. The likely performance of the people con- research on medical students themselves cerned if they enter the medical school. (Coombs and Vincent, 1971). 3. The type of person that the medical school Personality factors appear to be somewhat would like to attract. less useful in prediction, with correlations tendSe!ection for medicine is an area that has ing to be in the region of 0.2 to 0.3. Again, received much interest and research, especially considerable research has been carried out in in relation to the prediction of performance. In this area, and a good example is a recently regeneral, three types of predictors that have been ported study by Savage (1972). He concluded, studied can be identified : a. intellectual factors; on the basis of his study, that an effective selecb. personality factors; and c. ‘sociological’ tion framework needed to consider intelligence (he found a correlation of 0.36 between A levels factors. On the first of these, intellectual factors, and medical school performance - and 0.7 where considerable research has to be carried between AH5: scores and medical school perout, the consistent findings are of correlations formance); study habits (where the correlation in the order of 0-3 to 0.4 between A level per- was 0.37); and various personality variables. formance and subsequent medical school results Savage’s results led him to end with the sugges(with the correlations tending to be a little tion that students with certain characteristics higher when science subjects are considered. should be given careful consideration before any decision was made to admit them : included
This brief paper looks at the recurring issue of selection for entry into the medical school. In part it is based on data collected and analysed by the author; in part it is based on deliberately controversial views. It is intended to stimulate thought and argument, not to act as a definitive statement.
‘Requests for reprints to Dr Peter Sheldrake, Centre in the Educational Sciences, University of Edinburgh, 23 /24 Buccleuch Place, Edinburgh EH8 9JT. for Research
T h e AH5 intelligence test: A. Hein, A H 5 Test. National Foundation for Educational Research, 1956. 91
92
Prier Sheldrake
were those who are unconventional; independent or strongly extrovert; showing a large amount of self-conflict; or being schizoid. This recommendation was based on correlations that included - 0.23 on ‘independence’,- 0.20 on ‘psychoticism’,- 0.19 on ‘easily upset’, - all from the DP1, factors accounting for just over 5 % of the variance at best. Research of my own on entry data and subsequent performance of students at the Edinburgh Medical School has found that A levels correlate at 0.3 with medical school examinations, and various head teachers’ comments in the region of 0.2 to 0.3. Various combinations of variables can actually give multiple correlations as high as 0.6 to 0.9. In one study, together with Lynne Alexander, I looked at the school and medical school records of all those who graduated from Edinburgh in 1971, together with all those who might have graduated had they not withdrawn or been excluded at some earlier point. We also collected background information on these students. Our interest was in examining how far, singly or in combination, information about an applicant’s background and school record could be of use in predicting his eventual performance in medicine, or indeed his likelihood of completing the course. The results of this study will be referred to at various points in this paper, but our major finding was that there was no consistent means by which this could be done. Of course, questions raised about the value of predictors also raise the questions of what the student is being selected for. This is a question that has received far less consideration in the literature, but one suggestion is that what is most important is to select people who will do well in the initial medical school examinations. This appears to be based on the belief that correlations are fairly high between early degree examination results and later ones, and that selection for the earlier examinations will also select for the later ones. In fact, in Edinburgh at least, I found that correlations between earlier and later examinations were not particularly high (of the order of 0.3 to 0.4). Of course, there are differences in the importance ascribed to examinations as the criteria by which students are assessed as competent to move from preclinical to clinical medicine.
The ‘contest’ model I have just been referring to is one favoured by a number of medical schools, but in others ‘sponsored mobility’ is at least as important. When sponsored mobility is more characteristic of the situation, selection for preclinical performance becomes a less important factor. Finally, the search for methods and information about ‘how’ to select leads to consideration of the pool of applicants, as a whole rather than just the entrants. Again this has received scant consideration, though Johnson (1971) tells us that a substantial proportion of the people left in the ‘pool’ and not selected by the medical school he was studying were academically as good as those selected, both in terms of level of achievement in these A and 0 levels. He also tells us that pupils from state schools seem to be at a disadvantage compared to those from direct grant and public schools, as are those from some geographical areas. Most strikingly, he found that lower class children tended to be self-selected out from the pool -very few even apply to read medicine. Finally, he noted various background factors that apparently contribute to having a successful interview - without providing any statistical data on this. Other research recently reported has shown that there are interesting regional variations that affect the pool of applicants. On the one hand, it is clearly an advantage to live in certain parts of the country if you want to get in to a medical school; at present it seems that living in East Anglia is a severe disadvantage for the would-be doctor. On the other hand, it also appears that regions ‘lose’ their students : while for some parts of the country the numbers going to read medicine are proportionately coqparable with other regions, once graduated these students do not want to return. There is a distinct preference for doctors to work in the south-east and the south-west and in or around major urban areas-once again East Anglia, and also the west of Yorkshire, appear to suffer from a relative shortage of qualified practitioners. A further study of my own, on applicants to Edinburgh for entry in 1972, confirms the picture given by Johnson.Many unsuccessful applicants are well qualified, and some clearly fail
How should u e select?
to get into other schools. Equally Edinburgh does take a high proportion of its Scottish applicants (but it is important to note here that these students find it difficult to gain admission into English medical schools, having experienced a different school and examination system). Bloom, in a recent paper as a contribution to a working group on the selection of students for medical education organised by the World Health Organization (1972), has summarized the work done on medical school selection in looking at the selection criteria in the following way :
93
high qualifications were not sufficient to keep them in, despite the lower entry standard of other schools.
Alternatives in selection Savage clearly had little doubt about the value of the kind of study he had done. In his paper he suggests that high correlations between variables can provide the pattern on which a selection programme can be based. However, one only needs to look at the results we obtained in our very precise study of Edinburgh students to wonder whether or .not the selection frameworks that they imply are at all feasible.
‘ The pattern of results . . . has been consistently negative: that is, virtually no satisfactory method of evaluating the effectiveness of current selection procedures has been found. Within this pattern, objective criteria, such as college grades,and the medical college aptitude test (MCAT) scores, have predictive value only for those students who perform poorly in the first year of medical school. To a lesser extent, these variables also predict performance in the second year. In the clinical years, however, the correlation of the performance with these prediction variables has not been notably successful. Moreover, correlations between performance in the first two years of medical school (the basic science or preclinical years) and the last two years (the clinical years) tend to be zero, suggesting that performance criteria are so different in these two parts of medical school as to render such prediction studies highly questionable, particularly with reference to the presumed major objective of medical education, performance as a doctor ’.
In looking at the performance of the 1971 graduates and failures in the examinations they took at various stages of their medical school careers, we found, according to the examination being considered, a different combination of variables (from a group which included a variety of head teacher’s comments, A level performance, and the student’s expressed interest in studying medicine) would combine to give the best prediction of their actual marks. The data gave two sorts of problem: first, that the particular variables concerned were different for each examination considered; and, second, that even when the same variable appeared as a prediction for different examinations, they were sometimes related to performance in quite different and even contradictory ways. Thus, we found that a negative recommendation from a head teacher on an applicant’s suitability for studying medicine would be correlated with performance on another, and so on.
Bloom himself is a pioneer in the recent trend in the study of selection to look at more sociological factors, in his case of how students select medical schools and the implications of their views for their subsequent reactions. Bloom, in a study of SUNY Downstate Medical Center, found that the medical school was regarded as a prestigious institution and that this had drawn students to it. As a result the entry standard was very high as measured by MCAT scores. However, these students then became very disillusioned on entering the school to find they continued to be tested and ‘flunked’, and their
Simply, there is no consistent, or even meaningful, framework that such results indicate. More generally, many studies, such as the one I have carried out in Edinburgh, have shown that only very few single ‘entry’ variables show any significant correlation with subsequent performance, and that these tend to be low but (as Abercrombie points out) moderately consistent correlations; Apart from measures of academic ability, especially A level grades, little else is consistent, however. When we go on to consider combinations of variables, the predictability of performance does go up, but the consistency and the meaning of these combinations becomes
94
Peter Sheldrake
more and more tenuous. Of course, the correlations that A levels show are important - though it must be emphasized they only account for something like 10 to 15% of the variance in most cases. Clearly, the data do argue strongly for making some ‘cut-off‘ point on the basis of academic ability when admitting students to a medical school, but little else. It is very instructive to make a small aside here. Perry, who studied Edinburgh selection 10 years before my own study, produced some fascinating data (Perry, 1966). He showed that the failure rate then (the proportion who were excluded and those who withdrew as well) went down as the number of direct entrants (into second year) increased; and proportionately, women did better than men (fewer failed to complete the course, and did so with less resits, and repeated years). Yet, in recommending the selection policy to be followed in Edinburgh, Perry, taking necessary account of the constraints of reality, actually recommended keeping down both the number of direct entrants, and the number of women. He did so because Scottish students find it difficult to gain places in English medical schools, as they do not usually take A levels, and so Scottish medical schools have a special responsibility. Women, he pointed out, tend on average, to work half the working life of men, and are a poorer investment in economic terms. Perry’s views point to the first of two important areas that we now need to consider: first, the question of policy, especially in relation to quotas made up on the basis of ‘political’ factors; and second, the notion of fairness in selection. We first of all need to distinguish between two types of variables: those that I will label ‘political’- that is variables concerned with things like sex. place of birth, type of school attended (and even, possibly, father’s occupation); and, those I will label ‘background’ variables -including the more common variables looked at, including such things as interests, academic record at school, health, personality, etc. I hope that the basis of the distinction between the two types of variables is quite clear. Political variables are those that refer to characteristics of people which are both outside their control, and which refer to attributes not consequent on their own development
or innate potentialities. Further, as I hope is clear from the labelling, any selection method based on such variables will be one that is taking account of political considerations. Similarly, those variables I have labelled as ‘background’ variables include attributes of people that are, to a certain extent, under their own control, or at least consequent to their own individual abilities and development. A selection system based on these kinds of variables would be of a rather different kmd, though it too has very important implications. I would like to suggest that selection should operate with a clear recognition of this distinction - that, in the first instance, there should be explicit recognition of the political implications of differences between candidates in terms of features such as sex, birth, and schooling, and that any decision made in relation towards these variables is a recognition of the political constraints that affect the recruitment and selection of medical students. In other words, such choices are to do with questions of policy. When policy decisions have been made, however, and the background variables are brought into consideration, I want to argue that, once certain minimal criteria are satisfied, and where selection is made necessary (in other words as a result of whatever policy decisions are made, there are more people than there are places available within given quotas), this selection should be made randomly-and this is to be done on the grounds of fairness. Let me first dwell on the issue of policy: the whole question of the political implications of selection in relation to variables like sex, birth, and school. The important point here is that one must recognize that there are certain differences, consequent on innate or environmental factors over which there is little control, and which do affect the performance and availability of certain kinds of candidates. For example, it is well known that, in general terms, women tend to do rather better than men in academic examinations in medical schools (mainly as a result of the fact that in most medical schools women, to gain a place, have to have higher qualifications than men). At the same time, it is also well known that women seem, if we look to the past for our evidence, to work for a shorter period of time on average than men. Clearly, women’s
H o w should we select?
careers are necessarily interrupted by pregnancies and, to a lesser extent, by other domestic considerations. I do not want to put forward an argument that supports any reduction in the number of places made available to women on the basis of these considerations (nor, indeed, any system which encourages the selection of more women on other grounds), but rather I want to make it clear that these are important considerations which should not be shirked.’ Similarly, place of birth is also an important variable, first because those who were born in certain areas may be educationally disadvantaged, and, as a result, less well prepared for those examinations that have hitherto been regarded as the best indicator of the suitability of students to enter a medical school. There are further considerations, however, and these also need to be borne in mind. Students drawn from certain areas or backgrounds may be more able to gain rapport with and treat patients in that same area, rather than doctors coming from outside - this is, after all, a policy that has been quite explicitly adopted in Russia. Finally, the type of schooling a student receives is also important - Johnson’s work, for example, reminds us that not only are schools differentially successful in preparing their students for certain kmds of examination, but they may also aid them in specific preparation for entry to institutions like a medical school. Any decision that is made in relation to sex or birth or schooling (or other related factors) must be recognized as a political decision. These involve matters of policy, and as such the decisions made on them in relation to such variables should be done so explicitly, and in full recognition of their implications - not in terms of evidence produced about the comparative performance of students in relation to such variables, which I regard as being a dishonest way of avoiding confrontation with the implications of whatever differences might exist. With this rather burdensome topic of political factors on one side, let us now turn to my suggestion that, in the simplest terms possible,
’At present Edinburgh Medical School does not operate any quota system or discrimination against women applicants.
95
when certain minimal criteria have been satisfied, students should be selected randomly. That this should be done randomly, is, I want to argue, because there is no evidence that suggests that selection methods in any way improve on a random system. Indeed, I would also suggest that the immense effort put into the selection of medical students on the basis of a certain limited range of factors has little or no predictive value. As such, such selection serves only to reduce variability, and hence the value, of the population accepted by the medical school. Of course, I do recognize that there may be certain minimal cirteria that must be taken into account - though at present the only one that has clearly been shown to be important is an academic level below which students are rejected if they are not to find the academic work of the course too demanding. Only where students offer evidence which indicates that they are academically unable to study at the level required for medicine should they be rejected, however. This suggestion, as you may know, restores to ‘grading’ the original intention for which it was devised. Grades and the associated idea of measuring IQ. originally came into being when the American Army needed some means of screening out those people who applied to join the army and who were intellectually incapable of the work. IQ and grades were devised as bases for rejection, and not for selection - the present situation of selecting those as best because of their high performance in the examinations or their high IQs is an inversion of the original intention of such tests, which is both unfortunate and misleading. Many of the unsuccessful applicants to medical schools are more than adequately qualified to be trained as doctors, and the ‘problem’ of selection is not to pick the best applicants -but to fairly reject from a large group of suitable people. There is, of course, another problem, which this alternative view of selection neither supersedes nor in any way obviates. This is, quite simply, that, for the majority of medical schools we know about, there is a clear division between the preclinical and the clinical curriculum, not just in the content of the courses, but also in the way in which students are examined
96
Peter Sheldrake
in these years, and the consequences of poor or the medical school. This paper may appear to unsatisfactory performance. Students tend to be have avoided the additional question of how we excluded from continuing to study medicine on should set about selecting for the future -but the basis of their performance in preclinical sub- this is simply because I do not believe you can jects, whereas in the clinical years such failures select for the future. The aspirations and evenare extremely rare. In Edinburgh, for example, tual career choices of students are determined, the failure rate over recent years has wandered to a major extent, by the medical school itself. somewhere between 7 and 12% -but the num- There are, of course, differences between the ber of failures in the clinical years is almost un- interests of applicants to different medical measurable. These differences do not stem from schools - 16% of those wanting to enter Edinvariations in the severity of examinations, but burgh Medical School hope to do research, but appear to reflect changes in the way students none cite this as their eventual career at London are viewed. Irrespective of the selection method Hospital Medical College, for example*, Howthat is adopted, the same kind of care and atten- ever, more influential in the actual distribution tion that is given to students in the clinical of career choice is the support (or discourageyears, in helping them to learn and grapple with ment) given to these aspirations once the stuaspects of their subject, should be provided in dent enters - the research flavour of Edinburgh the preclinical years. To demolish the hurdle of is well known! Questions of future need related selection, erected on inappropriate grounds, is to what goes on inside the medical school and one aim: another, which I would also like to not to selection methods - to think so is to resee realized, is the demolition of the internal inforce one of the many illusions that people hurdles of the medical school, and the return hold about the process of selection. Of course, to the use of examinations as a means of help- this paper is based on a relatively small amount ing the students see how well they have learned of research, and more is needed. But, even with rather than solely as a basis for rejection. reservations in mind, for selection itself, the best method may well require just a pin and a Bloom’s own research is relevant here, too.’ There, the most important factor which affected list of names. the way in which students ‘got on’ inside the Summary medical school was the expectation they brought The implications of recent studies of the selecto it. S U N Y was more than successful in getting tion process for entry to medical school are applicants from ‘highly qualified’ entrants, but discussed. I t is argued that there are two types precisely because of their high qualifications of factors that need to be considered in selecthese people resented a medical school atmos- tion; first, those that are essentially concerned phere whose emphasis was placed on rote with policy decisions in relation to entry, e.g. memory, the acquisition of facts, and frequent sex or nationality; and, second, those that are testing and failure. to do with individual characteristics. In relation to this second group, once minimal criteria are Conclusion satisfied, random choice seems as fair as any The research done in Edinburgh, and in many method of selection, as a means of choosing other places, gives little support to the belief among applicants: none appears to be suffithat there is an ‘ideal’ selection framework. ciently predictive of performance to be suitable It is not just the case that the indicators that as a means for selection in themselves. might be used are insufficient, but, more important, that there is a bewildering variety of dif- Although, of course, the opinions expressed are my own, much of the work carried out on the selection ferent measures of achievement that selection of Edinburgh students was helped enormously by the might be aimed at. An emphasis on selection co-operation of Professor A. S. Duncan, Executive and on achievement is inadequate without also Dean; Dr J. Watt, Associate Dean: and Mr G . P. looking at the social processes going on inside Neill, Faculty Office Edinburgh Medical School. I am ‘In a study oE the Downstate Medical Center, State University of New York (Bloom, 1971).
‘From a questionnaire administered in Edinburgh and London Hospital Medical College by Paul Atkinson, results not yet published.
H o w should we select?
also particularly indebted to Lynn Alexander, who carried out the bulk of the data collection, aided by Gillian Sharp and Elizabeth Nichol.
References Abercrombie, M. L. I., Hunt, S . , and Stringer, P. (1969). Selection and Academic Performance o f Students in a University School of Architecture. Society for Research in Higher Education: London. Bloom, S. (1971). The medical school as a social system. Milbank Memorial Fund Quarterly, 49, No. 2, part 2. Bloom, S. (1972). Report on Selection f o r Medical School. World Health Organization : Copenhagen. Coombs, R. H., and Vincent, C. E. (1971). Psychosocial Aspects of Medical Training. Charles C. Thomas : Springfield, Illinois.
Johnson, M. L. characteristics students and cants. British 260-263.
97 (1971). A comparison of the social and academic achievement of medical unsuccessful medical school appliJournal of Medical Education, 5,
Jones, C . L., MacKintosh, H., and McPherson, A. (1973). Dictating failure: a comment on national differences in pedagogy and achievement. Universities Quarterly. In the press. Perry, W. L. M. (1966). A study of medical school selection and performance in the Edinburgh Medical School. British Journal of Medical Education, 1, 16-24. Savage, R. D. (1972). An explanatory study of individual characteristics associated with attainment in medical school. British Journal o f Medical Education, 6, 68-77.
How should w e select? - a sociologist‘s view FETER SHELDRAKE’ E d i n b u r g h University
C e n t r e for Research in t h e E d u c a t i o n a I S c i e n c e s ,
K e y words *COLLEGE ADMISSION TEST ‘SCHOOLS, MEDICAL EDUCATION, MEDICAL, UNDERGRADUATEINTELLIGENCE TESTS PERSONALITSCURRICULUM STUDENT DROPOUTS ENGLANDSCOTLANDUNITEDSTATES
especially earlier on in the medical school course). A good and recent example of a study of this kind comes from the work of Jones and McPherson, looking at school record and student performance in the University of Edinburgh (Jones and McPherson, 1974). When other factors are added to school grades, correlations can be squeezed up to values of 0.5 or 0.6 (and even higher in the case of the Jones/ Selection and prediction McPherson study). Abercrombie’s; study of Selection is concerned with the process of identiselection and performance of architecture stufying successful applicants - a process that has dents offers comparable data of this kind three main concerns. (Abercrombie, Hunt, and Stringer, 1969). and 1. The nature of the pool of applicants from Gough’s paper in Coombs and Vincent’s inwhich the selection is to be made. valuable collection is an excellent summary of 2. The likely performance of the people con- research on medical students themselves cerned if they enter the medical school. (Coombs and Vincent, 1971). 3. The type of person that the medical school Personality factors appear to be somewhat would like to attract. less useful in prediction, with correlations tendSe!ection for medicine is an area that has ing to be in the region of 0.2 to 0.3. Again, received much interest and research, especially considerable research has been carried out in in relation to the prediction of performance. In this area, and a good example is a recently regeneral, three types of predictors that have been ported study by Savage (1972). He concluded, studied can be identified : a. intellectual factors; on the basis of his study, that an effective selecb. personality factors; and c. ‘sociological’ tion framework needed to consider intelligence (he found a correlation of 0.36 between A levels factors. On the first of these, intellectual factors, and medical school performance - and 0.7 where considerable research has to be carried between AH5: scores and medical school perout, the consistent findings are of correlations formance); study habits (where the correlation in the order of 0-3 to 0.4 between A level per- was 0.37); and various personality variables. formance and subsequent medical school results Savage’s results led him to end with the sugges(with the correlations tending to be a little tion that students with certain characteristics higher when science subjects are considered. should be given careful consideration before any decision was made to admit them : included
This brief paper looks at the recurring issue of selection for entry into the medical school. In part it is based on data collected and analysed by the author; in part it is based on deliberately controversial views. It is intended to stimulate thought and argument, not to act as a definitive statement.
‘Requests for reprints to Dr Peter Sheldrake, Centre in the Educational Sciences, University of Edinburgh, 23 /24 Buccleuch Place, Edinburgh EH8 9JT. for Research
T h e AH5 intelligence test: A. Hein, A H 5 Test. National Foundation for Educational Research, 1956. 91
92
Prier Sheldrake
were those who are unconventional; independent or strongly extrovert; showing a large amount of self-conflict; or being schizoid. This recommendation was based on correlations that included - 0.23 on ‘independence’,- 0.20 on ‘psychoticism’,- 0.19 on ‘easily upset’, - all from the DP1, factors accounting for just over 5 % of the variance at best. Research of my own on entry data and subsequent performance of students at the Edinburgh Medical School has found that A levels correlate at 0.3 with medical school examinations, and various head teachers’ comments in the region of 0.2 to 0.3. Various combinations of variables can actually give multiple correlations as high as 0.6 to 0.9. In one study, together with Lynne Alexander, I looked at the school and medical school records of all those who graduated from Edinburgh in 1971, together with all those who might have graduated had they not withdrawn or been excluded at some earlier point. We also collected background information on these students. Our interest was in examining how far, singly or in combination, information about an applicant’s background and school record could be of use in predicting his eventual performance in medicine, or indeed his likelihood of completing the course. The results of this study will be referred to at various points in this paper, but our major finding was that there was no consistent means by which this could be done. Of course, questions raised about the value of predictors also raise the questions of what the student is being selected for. This is a question that has received far less consideration in the literature, but one suggestion is that what is most important is to select people who will do well in the initial medical school examinations. This appears to be based on the belief that correlations are fairly high between early degree examination results and later ones, and that selection for the earlier examinations will also select for the later ones. In fact, in Edinburgh at least, I found that correlations between earlier and later examinations were not particularly high (of the order of 0.3 to 0.4). Of course, there are differences in the importance ascribed to examinations as the criteria by which students are assessed as competent to move from preclinical to clinical medicine.
The ‘contest’ model I have just been referring to is one favoured by a number of medical schools, but in others ‘sponsored mobility’ is at least as important. When sponsored mobility is more characteristic of the situation, selection for preclinical performance becomes a less important factor. Finally, the search for methods and information about ‘how’ to select leads to consideration of the pool of applicants, as a whole rather than just the entrants. Again this has received scant consideration, though Johnson (1971) tells us that a substantial proportion of the people left in the ‘pool’ and not selected by the medical school he was studying were academically as good as those selected, both in terms of level of achievement in these A and 0 levels. He also tells us that pupils from state schools seem to be at a disadvantage compared to those from direct grant and public schools, as are those from some geographical areas. Most strikingly, he found that lower class children tended to be self-selected out from the pool -very few even apply to read medicine. Finally, he noted various background factors that apparently contribute to having a successful interview - without providing any statistical data on this. Other research recently reported has shown that there are interesting regional variations that affect the pool of applicants. On the one hand, it is clearly an advantage to live in certain parts of the country if you want to get in to a medical school; at present it seems that living in East Anglia is a severe disadvantage for the would-be doctor. On the other hand, it also appears that regions ‘lose’ their students : while for some parts of the country the numbers going to read medicine are proportionately coqparable with other regions, once graduated these students do not want to return. There is a distinct preference for doctors to work in the south-east and the south-west and in or around major urban areas-once again East Anglia, and also the west of Yorkshire, appear to suffer from a relative shortage of qualified practitioners. A further study of my own, on applicants to Edinburgh for entry in 1972, confirms the picture given by Johnson.Many unsuccessful applicants are well qualified, and some clearly fail
How should u e select?
to get into other schools. Equally Edinburgh does take a high proportion of its Scottish applicants (but it is important to note here that these students find it difficult to gain admission into English medical schools, having experienced a different school and examination system). Bloom, in a recent paper as a contribution to a working group on the selection of students for medical education organised by the World Health Organization (1972), has summarized the work done on medical school selection in looking at the selection criteria in the following way :
93
high qualifications were not sufficient to keep them in, despite the lower entry standard of other schools.
Alternatives in selection Savage clearly had little doubt about the value of the kind of study he had done. In his paper he suggests that high correlations between variables can provide the pattern on which a selection programme can be based. However, one only needs to look at the results we obtained in our very precise study of Edinburgh students to wonder whether or .not the selection frameworks that they imply are at all feasible.
‘ The pattern of results . . . has been consistently negative: that is, virtually no satisfactory method of evaluating the effectiveness of current selection procedures has been found. Within this pattern, objective criteria, such as college grades,and the medical college aptitude test (MCAT) scores, have predictive value only for those students who perform poorly in the first year of medical school. To a lesser extent, these variables also predict performance in the second year. In the clinical years, however, the correlation of the performance with these prediction variables has not been notably successful. Moreover, correlations between performance in the first two years of medical school (the basic science or preclinical years) and the last two years (the clinical years) tend to be zero, suggesting that performance criteria are so different in these two parts of medical school as to render such prediction studies highly questionable, particularly with reference to the presumed major objective of medical education, performance as a doctor ’.
In looking at the performance of the 1971 graduates and failures in the examinations they took at various stages of their medical school careers, we found, according to the examination being considered, a different combination of variables (from a group which included a variety of head teacher’s comments, A level performance, and the student’s expressed interest in studying medicine) would combine to give the best prediction of their actual marks. The data gave two sorts of problem: first, that the particular variables concerned were different for each examination considered; and, second, that even when the same variable appeared as a prediction for different examinations, they were sometimes related to performance in quite different and even contradictory ways. Thus, we found that a negative recommendation from a head teacher on an applicant’s suitability for studying medicine would be correlated with performance on another, and so on.
Bloom himself is a pioneer in the recent trend in the study of selection to look at more sociological factors, in his case of how students select medical schools and the implications of their views for their subsequent reactions. Bloom, in a study of SUNY Downstate Medical Center, found that the medical school was regarded as a prestigious institution and that this had drawn students to it. As a result the entry standard was very high as measured by MCAT scores. However, these students then became very disillusioned on entering the school to find they continued to be tested and ‘flunked’, and their
Simply, there is no consistent, or even meaningful, framework that such results indicate. More generally, many studies, such as the one I have carried out in Edinburgh, have shown that only very few single ‘entry’ variables show any significant correlation with subsequent performance, and that these tend to be low but (as Abercrombie points out) moderately consistent correlations; Apart from measures of academic ability, especially A level grades, little else is consistent, however. When we go on to consider combinations of variables, the predictability of performance does go up, but the consistency and the meaning of these combinations becomes
94
Peter Sheldrake
more and more tenuous. Of course, the correlations that A levels show are important - though it must be emphasized they only account for something like 10 to 15% of the variance in most cases. Clearly, the data do argue strongly for making some ‘cut-off‘ point on the basis of academic ability when admitting students to a medical school, but little else. It is very instructive to make a small aside here. Perry, who studied Edinburgh selection 10 years before my own study, produced some fascinating data (Perry, 1966). He showed that the failure rate then (the proportion who were excluded and those who withdrew as well) went down as the number of direct entrants (into second year) increased; and proportionately, women did better than men (fewer failed to complete the course, and did so with less resits, and repeated years). Yet, in recommending the selection policy to be followed in Edinburgh, Perry, taking necessary account of the constraints of reality, actually recommended keeping down both the number of direct entrants, and the number of women. He did so because Scottish students find it difficult to gain places in English medical schools, as they do not usually take A levels, and so Scottish medical schools have a special responsibility. Women, he pointed out, tend on average, to work half the working life of men, and are a poorer investment in economic terms. Perry’s views point to the first of two important areas that we now need to consider: first, the question of policy, especially in relation to quotas made up on the basis of ‘political’ factors; and second, the notion of fairness in selection. We first of all need to distinguish between two types of variables: those that I will label ‘political’- that is variables concerned with things like sex. place of birth, type of school attended (and even, possibly, father’s occupation); and, those I will label ‘background’ variables -including the more common variables looked at, including such things as interests, academic record at school, health, personality, etc. I hope that the basis of the distinction between the two types of variables is quite clear. Political variables are those that refer to characteristics of people which are both outside their control, and which refer to attributes not consequent on their own development
or innate potentialities. Further, as I hope is clear from the labelling, any selection method based on such variables will be one that is taking account of political considerations. Similarly, those variables I have labelled as ‘background’ variables include attributes of people that are, to a certain extent, under their own control, or at least consequent to their own individual abilities and development. A selection system based on these kinds of variables would be of a rather different kmd, though it too has very important implications. I would like to suggest that selection should operate with a clear recognition of this distinction - that, in the first instance, there should be explicit recognition of the political implications of differences between candidates in terms of features such as sex, birth, and schooling, and that any decision made in relation towards these variables is a recognition of the political constraints that affect the recruitment and selection of medical students. In other words, such choices are to do with questions of policy. When policy decisions have been made, however, and the background variables are brought into consideration, I want to argue that, once certain minimal criteria are satisfied, and where selection is made necessary (in other words as a result of whatever policy decisions are made, there are more people than there are places available within given quotas), this selection should be made randomly-and this is to be done on the grounds of fairness. Let me first dwell on the issue of policy: the whole question of the political implications of selection in relation to variables like sex, birth, and school. The important point here is that one must recognize that there are certain differences, consequent on innate or environmental factors over which there is little control, and which do affect the performance and availability of certain kinds of candidates. For example, it is well known that, in general terms, women tend to do rather better than men in academic examinations in medical schools (mainly as a result of the fact that in most medical schools women, to gain a place, have to have higher qualifications than men). At the same time, it is also well known that women seem, if we look to the past for our evidence, to work for a shorter period of time on average than men. Clearly, women’s
H o w should we select?
careers are necessarily interrupted by pregnancies and, to a lesser extent, by other domestic considerations. I do not want to put forward an argument that supports any reduction in the number of places made available to women on the basis of these considerations (nor, indeed, any system which encourages the selection of more women on other grounds), but rather I want to make it clear that these are important considerations which should not be shirked.’ Similarly, place of birth is also an important variable, first because those who were born in certain areas may be educationally disadvantaged, and, as a result, less well prepared for those examinations that have hitherto been regarded as the best indicator of the suitability of students to enter a medical school. There are further considerations, however, and these also need to be borne in mind. Students drawn from certain areas or backgrounds may be more able to gain rapport with and treat patients in that same area, rather than doctors coming from outside - this is, after all, a policy that has been quite explicitly adopted in Russia. Finally, the type of schooling a student receives is also important - Johnson’s work, for example, reminds us that not only are schools differentially successful in preparing their students for certain kmds of examination, but they may also aid them in specific preparation for entry to institutions like a medical school. Any decision that is made in relation to sex or birth or schooling (or other related factors) must be recognized as a political decision. These involve matters of policy, and as such the decisions made on them in relation to such variables should be done so explicitly, and in full recognition of their implications - not in terms of evidence produced about the comparative performance of students in relation to such variables, which I regard as being a dishonest way of avoiding confrontation with the implications of whatever differences might exist. With this rather burdensome topic of political factors on one side, let us now turn to my suggestion that, in the simplest terms possible,
’At present Edinburgh Medical School does not operate any quota system or discrimination against women applicants.
95
when certain minimal criteria have been satisfied, students should be selected randomly. That this should be done randomly, is, I want to argue, because there is no evidence that suggests that selection methods in any way improve on a random system. Indeed, I would also suggest that the immense effort put into the selection of medical students on the basis of a certain limited range of factors has little or no predictive value. As such, such selection serves only to reduce variability, and hence the value, of the population accepted by the medical school. Of course, I do recognize that there may be certain minimal cirteria that must be taken into account - though at present the only one that has clearly been shown to be important is an academic level below which students are rejected if they are not to find the academic work of the course too demanding. Only where students offer evidence which indicates that they are academically unable to study at the level required for medicine should they be rejected, however. This suggestion, as you may know, restores to ‘grading’ the original intention for which it was devised. Grades and the associated idea of measuring IQ. originally came into being when the American Army needed some means of screening out those people who applied to join the army and who were intellectually incapable of the work. IQ and grades were devised as bases for rejection, and not for selection - the present situation of selecting those as best because of their high performance in the examinations or their high IQs is an inversion of the original intention of such tests, which is both unfortunate and misleading. Many of the unsuccessful applicants to medical schools are more than adequately qualified to be trained as doctors, and the ‘problem’ of selection is not to pick the best applicants -but to fairly reject from a large group of suitable people. There is, of course, another problem, which this alternative view of selection neither supersedes nor in any way obviates. This is, quite simply, that, for the majority of medical schools we know about, there is a clear division between the preclinical and the clinical curriculum, not just in the content of the courses, but also in the way in which students are examined
96
Peter Sheldrake
in these years, and the consequences of poor or the medical school. This paper may appear to unsatisfactory performance. Students tend to be have avoided the additional question of how we excluded from continuing to study medicine on should set about selecting for the future -but the basis of their performance in preclinical sub- this is simply because I do not believe you can jects, whereas in the clinical years such failures select for the future. The aspirations and evenare extremely rare. In Edinburgh, for example, tual career choices of students are determined, the failure rate over recent years has wandered to a major extent, by the medical school itself. somewhere between 7 and 12% -but the num- There are, of course, differences between the ber of failures in the clinical years is almost un- interests of applicants to different medical measurable. These differences do not stem from schools - 16% of those wanting to enter Edinvariations in the severity of examinations, but burgh Medical School hope to do research, but appear to reflect changes in the way students none cite this as their eventual career at London are viewed. Irrespective of the selection method Hospital Medical College, for example*, Howthat is adopted, the same kind of care and atten- ever, more influential in the actual distribution tion that is given to students in the clinical of career choice is the support (or discourageyears, in helping them to learn and grapple with ment) given to these aspirations once the stuaspects of their subject, should be provided in dent enters - the research flavour of Edinburgh the preclinical years. To demolish the hurdle of is well known! Questions of future need related selection, erected on inappropriate grounds, is to what goes on inside the medical school and one aim: another, which I would also like to not to selection methods - to think so is to resee realized, is the demolition of the internal inforce one of the many illusions that people hurdles of the medical school, and the return hold about the process of selection. Of course, to the use of examinations as a means of help- this paper is based on a relatively small amount ing the students see how well they have learned of research, and more is needed. But, even with rather than solely as a basis for rejection. reservations in mind, for selection itself, the best method may well require just a pin and a Bloom’s own research is relevant here, too.’ There, the most important factor which affected list of names. the way in which students ‘got on’ inside the Summary medical school was the expectation they brought The implications of recent studies of the selecto it. S U N Y was more than successful in getting tion process for entry to medical school are applicants from ‘highly qualified’ entrants, but discussed. I t is argued that there are two types precisely because of their high qualifications of factors that need to be considered in selecthese people resented a medical school atmos- tion; first, those that are essentially concerned phere whose emphasis was placed on rote with policy decisions in relation to entry, e.g. memory, the acquisition of facts, and frequent sex or nationality; and, second, those that are testing and failure. to do with individual characteristics. In relation to this second group, once minimal criteria are Conclusion satisfied, random choice seems as fair as any The research done in Edinburgh, and in many method of selection, as a means of choosing other places, gives little support to the belief among applicants: none appears to be suffithat there is an ‘ideal’ selection framework. ciently predictive of performance to be suitable It is not just the case that the indicators that as a means for selection in themselves. might be used are insufficient, but, more important, that there is a bewildering variety of dif- Although, of course, the opinions expressed are my own, much of the work carried out on the selection ferent measures of achievement that selection of Edinburgh students was helped enormously by the might be aimed at. An emphasis on selection co-operation of Professor A. S. Duncan, Executive and on achievement is inadequate without also Dean; Dr J. Watt, Associate Dean: and Mr G . P. looking at the social processes going on inside Neill, Faculty Office Edinburgh Medical School. I am ‘In a study oE the Downstate Medical Center, State University of New York (Bloom, 1971).
‘From a questionnaire administered in Edinburgh and London Hospital Medical College by Paul Atkinson, results not yet published.
H o w should we select?
also particularly indebted to Lynn Alexander, who carried out the bulk of the data collection, aided by Gillian Sharp and Elizabeth Nichol.
References Abercrombie, M. L. I., Hunt, S . , and Stringer, P. (1969). Selection and Academic Performance o f Students in a University School of Architecture. Society for Research in Higher Education: London. Bloom, S. (1971). The medical school as a social system. Milbank Memorial Fund Quarterly, 49, No. 2, part 2. Bloom, S. (1972). Report on Selection f o r Medical School. World Health Organization : Copenhagen. Coombs, R. H., and Vincent, C. E. (1971). Psychosocial Aspects of Medical Training. Charles C. Thomas : Springfield, Illinois.
Johnson, M. L. characteristics students and cants. British 260-263.
97 (1971). A comparison of the social and academic achievement of medical unsuccessful medical school appliJournal of Medical Education, 5,
Jones, C . L., MacKintosh, H., and McPherson, A. (1973). Dictating failure: a comment on national differences in pedagogy and achievement. Universities Quarterly. In the press. Perry, W. L. M. (1966). A study of medical school selection and performance in the Edinburgh Medical School. British Journal of Medical Education, 1, 16-24. Savage, R. D. (1972). An explanatory study of individual characteristics associated with attainment in medical school. British Journal o f Medical Education, 6, 68-77.
