Manual Therapy 19 (2014) 158e164
Contents lists available at ScienceDirect
Manual Therapy journal homepage: www.elsevier.com/math
Original article
Anatomical landmark position e Can we trust what we see? Results from an online reliability and validity study of osteopaths Elise Pattyn, Dévan Rajendran* European School of Osteopathy, Boxley House, The Street, Boxley, Kent ME14 3DZ, UK
a r t i c l e i n f o
a b s t r a c t
Article history: Received 3 April 2013 Received in revised form 7 October 2013 Accepted 16 October 2013
Background: Practitioners traditionally use observation to classify the position of patients’ anatomical landmarks. This information may contribute to diagnosis and patient management. Objectives: To calculate a) Inter-rater reliability of categorising the sagittal plane position of four anatomical landmarks (lateral femoral epicondyle, greater trochanter, mastoid process and acromion) on side-view photographs (with landmarks highlighted and not-highlighted) of anonymised subjects; b) Intra-rater reliability; c) Individual landmark inter-rater reliability; d) Validity against a ‘gold standard’ photograph. Design: Online inter- and intra-rater reliability study. Subjects: Photographed subjects: convenience sample of asymptomatic students; raters: randomly selected UK registered osteopaths. Methods: 40 photographs of 30 subjects were used, a priori clinically acceptable reliability was 0.4. Inter-rater arm: 20 photographs without landmark highlights plus 10 with highlights; Intra-rater arm: 10 duplicate photographs (non-highlighted landmarks). Validity arm: highlighted landmark scores versus ‘gold standard’ photographs with vertical line. Research ethics approval obtained. Raters: Osteopaths (n ¼ 48) categorised landmark position relative to imagined vertical-line; Gwet’s Agreement Coefficient 1 (AC1) calculated and chance-corrected coefficient benchmarked against Landis and Koch’s scale; Validity calculation used Kendall’s tau-B. Results: Inter-rater reliability was ’fair’ (AC1 ¼ 0.342; 95% confidence interval (CI) ¼ 0.279e0.404) for non-highlighted landmarks and ’moderate’ (AC1 ¼ 0.700; 95% CI ¼ 0.596e0.805) for highlighted landmarks. Intra-rater reliability was ’fair’ (AC1 ¼ 0.522); range was ‘poor’ (AC1 ¼ 0.160) to ‘substantial’ (AC1 ¼ 0.896). No differences were found between individual landmarks. Validity was ‘low’ (TB ¼ 0.327; p ¼ 0.104). Conclusion: Both inter- and intra-rater reliability was ‘fair’ but below clinically acceptable levels, validity was ‘low’. Together these results challenge the clinical practice of using observation to categorise anterioposterior landmark position. Ó 2014 Elsevier Ltd. All rights reserved.
Keywords: Reliability and validity Observation Osteopathy Clinical skills
1. Introduction Observation of the standing patient is a traditional procedure used by manual therapists to formulate diagnoses and plan treatment (Greenman, 1996; Johnson, 2012). Typically patients stand in an upright relaxed position whilst the clinician observes them from anterior, posterior and lateral viewpoints, judging patients’ position against an imagined vertical line (Houglum et al., 2012). The lateral view is the most challenging to assess as the body is bisected visually in an asymmetric manner. An imagined vertical-line passes through certain external anatomical landmarks: ear lobe (mastoid
* Corresponding author. Tel.: þ44 1622671558. E-mail address: [email protected] (D. Rajendran). 1356-689X/$ e see front matter Ó 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.math.2013.10.002
process); acromio-clavicular joint; hip; knee and foot; and assists clinicians to assess the lateral view (Magee, 2006). Recognising deviation of landmarks’ position from the vertical may help clinicians identify abnormal tissue loading that could be contributing to patients’ symptoms: forward head posture correlates with neck pain and headaches; knee flexion correlates with patella-femoral and low back pain (Szeto et al., 2002; Tsuji et al., 2002). Some traditional osteopathic models use standing observation to classify patients into several postural typologies that dictate a particular treatment approach (Hall, 1955; Wernham, 1955). Despite the prevalence of static observation within manual therapy, there are limited reliability and validity data on this process. This conflicts with current expectations that practitioners base their ‘hypotheses on the best level of evidence possible’ and strive to use diagnostic tests ‘that have been shown to be most reliable
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
and valid’ (Murphy, 2002). The few published studies that have assessed reliability of clinical observation have all used differing methodologies (video footage, still photographic images, live subjects) that constrain raters’ views to specific regions of the patient (e.g. head/neck or lumbar/pelvic regions), rather than show the subject’s whole body (Bryan et al., 1990; Fedorak et al., 2003; Moran and Ljubotenski, 2006; Silva et al., 2010b). These studies reported inter- and intra-rater reliability scores for spinal curve assessment that ranged from ‘fair’ to ‘poor’, and scores appearing to be independent of practitioners’ experience and professional background (Fedorak et al., 2003). The use of aids (vertical lines added to photographs, video that panned around a subject) failed to improve reliability of observer ratings (Bryan et al., 1990; Moran and Ljubotenski, 2006). Including symptomatic patients within an inter- and intra-rater reliability of head/neck posture observation similarly failed to increase reliability scores and led the authors to question the value of observation in patients presenting with neck pain (Silva et al., 2009b). Other issues relate to the statistical analyses used in reliability studies; concerns have been raised about reporting percentage agreement (Kaufman and Rosenthal, 2009). Subsequently, many authors adopted Cohen’s kappa statistic when dealing with discrete data. However, the existence of the ‘kappa paradox’ may create inaccurate results (Feinstein and Cicchetti, 1990). Finally there exists an additional potential error associated with interpreting the calculated k value, which is not corrected for chance agreement, against a benchmark scale (Gwet, 2010). As far as we know, no study has assessed inter- and intra-rater reliability of full body posture using photographs to assess anterior-posterior (A/P) positioning of external anatomical landmarks relative to the vertical, despite this being a widely accepted component of a comprehensive musculoskeletal assessment (Magee, 2006; Silva et al., 2009a; Koes et al., 2010). This study aimed to estimate the inter- and intra-rater reliability of osteopaths’ visual assessment of four landmarks on lateral-view photographs of a subject’s full body relative to an imagined vertical line, and to see if artificially highlighting these landmarks would alter reliability. A secondary aim was to estimate the validity of these ratings relative to a ‘gold standard’ photograph that showed the landmarks relative to a marked vertical line. Since we were calculating agreement and not hypothesis testing, power and sample size calculation were not carried out.
159
addresses. This method had the potential benefit of allowing registered osteopaths who were practising outside the UK to be selected. 2.2.2. Photographed subjects A convenience sample of adult students was recruited from the European School of Osteopathy (UK). Subjects had no visually identifiable external marks (visible tattoos, birthmarks, scars or piercings). Subjects’ gender, age, height and weight were recorded; however raters were blinded to this information. 3. Materials 3.1. Photographic image processing Two sets of identical photographs were produced as follows. All photographs were taken in a well-lit, clinical room using a camera (Prosumer G9 with f2.8e4.8, 35e210 mm lens, Canon Inc., Japan) set at a height of 150 cm using a spirit levelled tripod located 175 cm from the subject. Adhesive floor markers were used to designate tripod location and a taped reference line for positioning subjects (Pownall et al., 2008). Subjects removed outer clothing and on their left side EP placed five white skin markers (19 mm in diameter) over the lateral malleolus, lateral femoral condyle, greater femoral trochanter, acromion process and mastoid process (Dunk et al., 2005). All photographs were imported into image processing software (Adobe Photoshop Elements v6.0, USA); those with inadequate subject positioning were deleted. Remaining photographs were anonymised (a grey circle obscured subjects’ faces) and duplicated, producing two sets of identical photographs: sets A and B (Fig. 1). Using image processing software, set A photographs had all skin markers removed except the one over the lateral malleolus, which was highlighted; set B photographs had all of the skin markers highlighted. The set B photographs were then duplicated to produce a third set of photographs, set C, which had a vertical line, bisecting the lateral malleolus, added (Fig. 1). The set C photographs stood as the ‘gold standard’ and were subjected to a validation process to identify photographs with any ambiguous landmark positioning. At two occasions, 1 week apart, EP rated every landmark relative to the vertical line in the set C photographs. Throughout this process EP remained blinded to all results; only photographs with 100% landmark agreement on both occasions were retained.
2. Methods 3.2. Online questionnaire design 2.1. Study design and ethics A blinded online inter- and intra-rater reliability questionnaire design was used to measure agreement of visual assessment using lateral-view full-body photographs of standing subjects. Raters assessed position of four anatomical landmarks with respect to an imagined vertical line in two sets of photographs: 1) with the lateral malleolus highlighted 2) all landmarks highlighted. Categories for landmark position were: ‘anterior’, ‘middle’ or ‘posterior’. Rater validity was assessed against a gold standard photograph that had an actual vertical line drawn on it to show correct landmark position. Ethical approval was obtained from the European School of Osteopathy Research Ethics Committee. 2.2. Participants and procedures 2.2.1. Raters Rater recruitment was designed to randomly select and invite, by email, approximately one third of all osteopaths listed in the 2008 General Osteopathic Council’s (GOsC) register who had email
All set A and B photographs were used to pilot the online questionnaire; feedback indicated that rating 80 photographs was excessive. The final questionnaire contained 40 randomly selected photographs comprising: 20 from set A plus 10 duplicates (intrarater arm), and 10 photographs from set B. The online questionnaire utilised a survey web-site (www. kwiksurveys.com) comprising two introductory pages: page 1 contained an information/consent sheet, instructions on how to complete the questionnaire plus a guided example of what was expected; the consent section explained that should the osteopath complete the questionnaire, this would be deemed as giving consent to use their anonymised data. Page 2 captured the number of years since qualification, training college, country of practice and how frequently the respondent used visual observation in a clinical setting. The subsequent 40 pages each contained one photograph and one scale to trichotomously rate the position of all four landmarks (i.e. anterior, middle, posterior). The survey stopped raters going backwards to prevent them comparing photographs. Only data from fully completed questionnaires were used; raters with
160
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
Fig. 1. Examples of each of the 3 sets of photographs. Set A: Non-duplicate photographs used for inter-rater reliability arm; duplicate photographs used for intra-rater reliability arm. Set B: used for inter-rater reliability arm (all landmarks highlighted). Set C: used as ‘gold standard’ for validity arm.
incomplete data were treated as if they had dropped out. Raters would make 160 ratings in total. 4. Outcome data and statistical analysis All data were exported into a spreadsheet (Excel, Microsoft Corp. USA) and split into three main data sets.
(Gwet, 2010). To increase benchmarking precision, a ‘critical value’ (AC1c) was calculated and subtracted from AC1 value and the resultant ‘significant value’ (AC1s) interpreted against the Landis and Koch benchmark scale (Table 1) (Landis and Koch, 1977; Gwet, 2010). An a priori level of ‘clinically acceptable reliability’ was set at AC1c S 0.40 (Sim and Wright, 2005). 4.3. Rater validity
4.1. Inter-rater reliability Rating data obtained from 20 non-duplicate set A photographs were used to calculate inter-rater reliability of landmark position without any skin markers visible, and data from 10 set B photographs were used to calculate inter-rater reliability of anatomical landmark position when skin markers were highlighted to assist the rater. 4.2. Intra-rater reliability Rating data from 10 pairs of duplicated photographs within set A were used to calculate intra-rater reliability. The Agreement Coefficient 1 macro (AgreeStat, Advanced Analytics, USA) was used to calculate an Agreement Coefficient 1 (AC1) value for inter- and intra-rater reliability of all categorical data Table 1 Landis and Koch’s (1977) benchmark scale applied to AC1s (significant value). Significant value of AC1
Strength of agreement
0.5 more than 5% of the time (Gwet, 2010); equivalent to an intra-rater
Fig. 3. Comparison of AC1c inter-rater scores between set A (markers (M): landmarks highlighted) photographs and set B photographs (no markers (No M): landmarks without highlights); scores and 95% confidence intervals are shown for all landmarks (All) and for each individual landmark.
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
arm where one blinded rater scores 10 subjects twice. To control for this error, Gwet proposed that any calculated agreement coefficient statistic (Gwet’s AC1, Cohen’s kappa, Brennan and Prediger coefficient and Scott’s P) should have its ‘critical value’ (the 95th percentile of the agreement coefficient when the rating is performed randomly) subtracted to find the ‘significant part or value’ of the agreement statistic, which is compared to the benchmark scale. This significant value is always less than the calculated value, but reflects a more precise benchmark interpretation, especially for scores derived from the intra-rater arm. 6.2. Study limitations, strengths and proposals for future research The use of digitised skin markers to highlight anatomical landmarks can be prone to bias (Dunk et al., 2005); however our blinded double-rating procedure and resultant removal of photographs without 100% agreement from the study would have minimised this. Since raters were categorising the highlighted landmark’s position, if there was an error in skin-marker placement (when compared with an X-ray), this would not have impacted on the reliability scores. Interestingly, using X-ray as a ‘gold standard’ for positional assessment is not without critics (Marks et al., 2003; Zaina et al., 2012). Similarly, although use of full-body photographs seems more reflective of practice compared with regional views, Weatherburn et al. (2003) reports that receiver operator curve and kappa analysis of different radiographic image modes is not significantly different. Since we were assessing the reliability of observation, using photographs rather than live models does not threaten the external validity of our results. Simplifying/standardising the observation process and limiting raters to a trichotomous decision would have had a positive bias on reliability scores. Our photographic subjects were young and asymptomatic; not representative of a typical patient population. However, symptom status appears unrelated to posture (Grimmer-Somers et al., 2008) and using young subjects controlled for age-related postural changes (Yip et al., 2008). We did not control for gender, but as far as we know this is not a confounding factor. Limiting raters to GOsC practitioners resulted in a predominantly UK-trained and based sample. Although professional background and experience do not appear to influence reliability of landmark observation (Fedorak et al., 2003; Sutton et al., 2013), our raters did display heterogeneity typical of a convenience based sample and this may impact on the generalisability of our finding. A review on reliability research reported 97% of retrieved articles used a mean of two raters (Van Genderen et al., 2003). Using 48 raters lends weight to the precision of our findings: variation of the calculated agreement coefficient in studies with more than 40 raters is 5%, with less than seven raters is 30% (Gwet, 2010). The use of a paradox free statistic (AC1) and chance-correcting the resultant coefficient prior to benchmarking strengthens our results, potentially presenting a better reflection of reliability: future authors researching reliability may wish to consider using this statistical method. To increase power of reliability studies, Sim and Wright (2005) suggest that: 1) inter-rater studies increase the number of subjects rated; 2) intra-rater studies increase the number of observations each rater makes per subject. As patient symptoms often correlate poorly with observer-rated postural disturbances (Harrison et al., 1996; Fernández-de-lasPeñas et al., 2007), any significant measurable differences in postural parameters are very small and difficult for practitioners to accurately observe (Yip et al., 2008; Silva et al., 2010a). Combining the subjective nature of observation with the wide variability of patient morphology (gender, height, weight, size, age and posture),
163
undermines the notion that clinicians can reliably observe and rate landmark position in a clinically meaningful manner. Future studies could assess: a) the role of confirmation bias within manual therapy practitioners’ clinical examination skills; b) the importance manual practitioners attach to observational findings when formulating a diagnosis and treatment plan; c) whether practitioner awareness of the reliability and accuracy of observation influences their decision to use this within in a clinical setting; d) which postural features osteopaths’ eyes are drawn to using eyetracking technology. 7. Conclusion In this study, inter-rater reliability of categorising landmarks with respect to an imagined vertical reference did not reach clinically acceptable levels; although intra-rater reliability did reach levels of clinical acceptability, only 8% of raters could maintain this over repeated measures. The validity of observed landmark positions, when compared with a ‘gold standard’ was categorised as ‘low’. Our findings challenge both the use of observation to categorise anatomical landmark position reliably and the validity of using these findings in a clinical reasoning process. Manual practitioners and students should be cognisant of the subjectivity, inaccuracy and unreliability of such observational techniques; educational institutes may wish to review the importance attached to this traditional skill within their curricula and question the merit of using an ‘experienced’ practitioner to guide a ‘novice’ student’s landmark observation in a teaching-clinic scenario. Acknowledgements The authors would like to thank Brenda Mullinger for her invaluable critique of this manuscript. References Bono CM, Schoenfeld AJ, Anderson PA, Harrop JS, France J, Vaccaro AR, et al. Observer variability of radiographic measurements of C2 (axis) fractures. Spine 2010;35(12):1206e10. http://dx.doi.org/10.1097/BRS.0b013e3181d9bb58. Bryan JM, Mosner E, Shippee R, Stull MA. Investigation of the validity of postural evaluation skills in assessing lumbar lordosis using photographs of clothed subjects. J Orthop Sports Phys Ther 1990;12(1):24e9. Cook C, Turney L, Ramirez L, Miles A, Haas S, Karakostas T. Predictive factors in poor inter-rater reliability among physical therapists. J Man Manipulative Ther 2002;10(4):200e5. Donovan T, Manning DJ. The radiology task: Bayesian theory and perception. Br J Radiol 2007;80(954):389e91. Dror IE. Perception is far from perfection: the role of the brain and mind in constructing realities. Behav Brain Sci 2005;28(06):763. Dunk NM, Lalonde J, Callaghan JP. Implications for the use of postural analysis as a clinical diagnostic tool: reliability of quantifying upright standing spinal postures from photographic images. J Manipulative Physiol Ther 2005;28(6): 386e92. Farmer ER, Gonin R, Hanna MP. Discordance in the histopathologic diagnosis of melanoma and melanocytic nevi between expert pathologists. Hum Pathol 1996;27(6):528e31. Fedorak C, Ashworth N, Marshall J, Paull H. Reliability of the visual assessment of cervical and lumbar lordosis: how good are we? Spine 2003;28(16):1857e9. http://dx.doi.org/10.1097/01.BRS.0000083281.48923.BD. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. the problems of two paradoxes. J Clin Epidemiol 1990;43(6):543e9. Fernández-de-las-Peñas C, Alonso-Blanco C, Cuadrado ML, Pareja JA. Neck mobility and forward head posture are not related to headache parameters in chronic tension-type headache. Cephalalgia 2007;27(2):158e64. Field A. Correlation, discovering statistics using SPSS. London, UK: Sage Publications; 2005. p. 1e779. Fryer G, McPherson HC, O’Keefe P. The effect of training on the inter-examiner and intra-examiner reliability of the seated flexion test and assessment of pelvic anatomical landmarks with palpation. Int J Osteopath Med 2005;8(4):131e8. Gibbons P, Tehan P. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):21e3.
164
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
Goossen S. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):23e5. Greenman P. Structural diagnosis, principles of manual medicine. 2nd ed. Maryland: Williams and Wilkins; 1996. p. 13e38. Grimmer-Somers K, Milanese S, Louw Q. Measurement of cervical posture in the sagittal plane. J Manipulative Physiol Ther 2008;31(7):509e17. Gwet KL. Handbook of inter-rater reliability. The definitive guide to measuring the extent of agreement among multiple raters. 2nd ed. Gaithersburg, USA: Advanced Analytics; 2010. p. 1e197. Hall TE. Posterior type, normal, anterior type. In: Wernham SGJ, editor. The 1956 year book. Maidstone, UK: The Osteopathic Institute of Applied Technique; 1955. p. 26e39. Haneline MT, Young M. A review of intraexaminer and interexaminer reliability of static spinal palpation: a literature synthesis. J Manipulative Physiol Ther 2009;32(5):379. Harrison AL, Barry-Greb T, Wojtowicz G. Clinical measurement of head and shoulder posture variables. J Orthop Sports Phys Ther 1996;23(6):353. Hayhoe M, Ballard D. Eye movements in natural behavior. Trends Cogn Sci 2005;9(4):188e94. Houglum P, Delores P. Brunnstrom’s clinical kinesiology. Philadelphia: FA Davis; 2012. p. 1e704. Johnson J. Postural assessment. Leeds: Human Kinetics; 2012. p. 1e184. Kaufman AB, Rosenthal R. Can you believe my eyes? The importance of interobserver reliability statistics in observations of animal behaviour. Anim Behav 2009;78(6):1487e91. Kmita A, Lucas NP. Reliability of physical examination to assess asymmetry of anatomical landmarks indicative of pelvic somatic dysfunction in subjects with and without low back pain. Int J Osteopath Med 2008;11(1):16e25. Koes BW, van Tulder M, Lin CW, Macedo LG, McAuley J, Maher C. An updated overview of clinical guidelines for the management of non-specific low back pain in primary care. Eur Spine J 2010;19(12):2075e94. Koning CP, Heuvel S, Staal JB, Smits-Engelsman BM, Hendriks EM. Clinimetric evaluation of active range of motion measures in patients with non-specific neck pain: a systematic review. Eur Spine J 2008;17(7):905e21. Krebs DE, Edelstein JE, Fishman S. Reliability of observational kinematic gait analysis. Phys Ther 1985;65(7):1027e33. Krupinski EA, Berger WG, Dallas WJ, Roehrig H. Searching for nodules: what features attract attention and influence detection? Acad Radiol 2003;10(8):861e8. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977:159e74. Lee D. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):26e7. Magee D. Assessment of posture, orthopaedic physical assessment e enhanced edition. Missouri: Saunders Elsevier; 2006. p. 873e903. Marks MC, Stanford CF, Mahar AT, Newton PO. Standing lateral radiographic positioning does not represent customary standing balance. Spine (Phila Pa 1976) 2003;28(11):1176e82. Moran R, Ljubotenski E. Reliability of visual assessment for lumbar curves in standing: does clinical experience matter? Int J Osteopath Med 2006;9(1):42. Murphy DR. The palpation reliability debate: the experts respond. J Bodyw Move Therapies 2002;6(1):30e4. Nitschke JE, Nattrass CL, Disler PB, Chou MJ, Ooi KT. Reliability of the American Medical Association guides’ model for measuring spinal range of motion. Its
implication for whole-person impairment rating. Spine (Phila Pa 1976) 1999;24(3):262e8. Paatelma M, Karvonen E, Heinonen A. Inter- and intra-tester reliability of selected clinical tests in examining patients with early phase lumbar spine and sacroiliac joint pain and dysfunction. Adv Physiotherapy 2010;12(2):74e80. Pett MA. Nonparametric statistics for health care research: statistics for small samples and unusual distributions. California, USA: Sage Publications; 1997. Pownall PJ, Moran RW, Stewart AM. Consistency of standing and seated posture of asymptomatic male adults over a one-week interval: a digital camera analysis of multiple landmarks. Int J Osteopath Med 2008;11(2):43e51. Rajendran D, Gallagher D. The assessment of pelvic landmarks using palpation: a reliability study of undergraduate students. Int J Osteopath Med 2011;14(2): 57e60. Silva A, Punt D, Johnson M. A postal survey gathering information about physiotherapists’ assessment of forward head posture for patients with chronic idiopathic neck pain. Eur J Pain 2009a;13(S1). S223a-S. Silva A, Sharples P, Johnson M. Studies comparing surrogate measures for head posture in individuals with and without neck pain. Phys Ther Rev 2010a;15(1): 12e22. Silva AG, Punt TD, Johnson MI. Reliability and validity of head posture assessment by observation and a four-category scale. Man Ther 2010b;15(5):490e5. Silva AG, Punt TD, Sharples P, Vilas-Boas JP, Johnson MI. Head posture assessment for patients with neck pain: Is it useful? Int J Ther Rehabil 2009b;16(1):43e53. Simons DG. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):34e6. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005;85(3):257e68. Sutton C, Nono L, Johnston RG, Thomson OP. The effects of experience on the interreliability of osteopaths to detect changes in posterior superior iliac spine levels using a hidden heel wedge. J Bodyw Move Therapies 2013;17(2):143e50. Szeto GPY, Straker L, Raine S. A field comparison of neck and shoulder postures in symptomatic and asymptomatic office workers. Appl Ergon 2002;33(1):75e84. Taylor M, Hipp JA, Gertzbein SD, Gopinath S, Reitman CA. Observer agreement in assessing flexion-extension X-rays of the cervical spine, with and without the use of quantitative measurements of intervertebral motion. Spine J 2007;7(6): 654e8. Tsuji T, Matsuyama Y, Goto M, Yimin Y, Sato K, Hasegawa Y, et al. Kneeespine syndrome: correlation between sacral inclination and patellofemoral joint pain. J Orthop Sci 2002;7(5):519e23. Van Genderen F, De Bie R, Helders P, Van Meeteren N. Reliability research: towards a more clinically relevant approach. Phys Ther Rev 2003;8(4):169e76. Weatherburn GC, Ridout D, Strickland NH, Robins P, Glastonbury CM, Curati W, et al. A comparison of conventional film, CR hard copy and PACS soft copy images of the chest: analyses of ROC curves and inter-observer agreement. Eur J Radiol 2003;47(3):206e14. Wernham SGJ. Mechanics of the spine. In: Wernham SGJ, editor. The 1956 year book. Maidstone, UK: The Osteopathic Institute of Applied Technique; 1955. p. 26e39. Yip CHT, Chiu TTW, Poon ATK. The relationship between head posture and severity and disability of patients with neck pain. Man Ther 2008;13(2):148e54. Zaina F, Pizzetti P, Donzelli S, Negrini F, Negrini S. Why X-rays are not reliable to assess sagittal profile: a cross sectional study. Stud Health Technol Inform 2012;176:268e72.
Contents lists available at ScienceDirect
Manual Therapy journal homepage: www.elsevier.com/math
Original article
Anatomical landmark position e Can we trust what we see? Results from an online reliability and validity study of osteopaths Elise Pattyn, Dévan Rajendran* European School of Osteopathy, Boxley House, The Street, Boxley, Kent ME14 3DZ, UK
a r t i c l e i n f o
a b s t r a c t
Article history: Received 3 April 2013 Received in revised form 7 October 2013 Accepted 16 October 2013
Background: Practitioners traditionally use observation to classify the position of patients’ anatomical landmarks. This information may contribute to diagnosis and patient management. Objectives: To calculate a) Inter-rater reliability of categorising the sagittal plane position of four anatomical landmarks (lateral femoral epicondyle, greater trochanter, mastoid process and acromion) on side-view photographs (with landmarks highlighted and not-highlighted) of anonymised subjects; b) Intra-rater reliability; c) Individual landmark inter-rater reliability; d) Validity against a ‘gold standard’ photograph. Design: Online inter- and intra-rater reliability study. Subjects: Photographed subjects: convenience sample of asymptomatic students; raters: randomly selected UK registered osteopaths. Methods: 40 photographs of 30 subjects were used, a priori clinically acceptable reliability was 0.4. Inter-rater arm: 20 photographs without landmark highlights plus 10 with highlights; Intra-rater arm: 10 duplicate photographs (non-highlighted landmarks). Validity arm: highlighted landmark scores versus ‘gold standard’ photographs with vertical line. Research ethics approval obtained. Raters: Osteopaths (n ¼ 48) categorised landmark position relative to imagined vertical-line; Gwet’s Agreement Coefficient 1 (AC1) calculated and chance-corrected coefficient benchmarked against Landis and Koch’s scale; Validity calculation used Kendall’s tau-B. Results: Inter-rater reliability was ’fair’ (AC1 ¼ 0.342; 95% confidence interval (CI) ¼ 0.279e0.404) for non-highlighted landmarks and ’moderate’ (AC1 ¼ 0.700; 95% CI ¼ 0.596e0.805) for highlighted landmarks. Intra-rater reliability was ’fair’ (AC1 ¼ 0.522); range was ‘poor’ (AC1 ¼ 0.160) to ‘substantial’ (AC1 ¼ 0.896). No differences were found between individual landmarks. Validity was ‘low’ (TB ¼ 0.327; p ¼ 0.104). Conclusion: Both inter- and intra-rater reliability was ‘fair’ but below clinically acceptable levels, validity was ‘low’. Together these results challenge the clinical practice of using observation to categorise anterioposterior landmark position. Ó 2014 Elsevier Ltd. All rights reserved.
Keywords: Reliability and validity Observation Osteopathy Clinical skills
1. Introduction Observation of the standing patient is a traditional procedure used by manual therapists to formulate diagnoses and plan treatment (Greenman, 1996; Johnson, 2012). Typically patients stand in an upright relaxed position whilst the clinician observes them from anterior, posterior and lateral viewpoints, judging patients’ position against an imagined vertical line (Houglum et al., 2012). The lateral view is the most challenging to assess as the body is bisected visually in an asymmetric manner. An imagined vertical-line passes through certain external anatomical landmarks: ear lobe (mastoid
* Corresponding author. Tel.: þ44 1622671558. E-mail address: [email protected] (D. Rajendran). 1356-689X/$ e see front matter Ó 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.math.2013.10.002
process); acromio-clavicular joint; hip; knee and foot; and assists clinicians to assess the lateral view (Magee, 2006). Recognising deviation of landmarks’ position from the vertical may help clinicians identify abnormal tissue loading that could be contributing to patients’ symptoms: forward head posture correlates with neck pain and headaches; knee flexion correlates with patella-femoral and low back pain (Szeto et al., 2002; Tsuji et al., 2002). Some traditional osteopathic models use standing observation to classify patients into several postural typologies that dictate a particular treatment approach (Hall, 1955; Wernham, 1955). Despite the prevalence of static observation within manual therapy, there are limited reliability and validity data on this process. This conflicts with current expectations that practitioners base their ‘hypotheses on the best level of evidence possible’ and strive to use diagnostic tests ‘that have been shown to be most reliable
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
and valid’ (Murphy, 2002). The few published studies that have assessed reliability of clinical observation have all used differing methodologies (video footage, still photographic images, live subjects) that constrain raters’ views to specific regions of the patient (e.g. head/neck or lumbar/pelvic regions), rather than show the subject’s whole body (Bryan et al., 1990; Fedorak et al., 2003; Moran and Ljubotenski, 2006; Silva et al., 2010b). These studies reported inter- and intra-rater reliability scores for spinal curve assessment that ranged from ‘fair’ to ‘poor’, and scores appearing to be independent of practitioners’ experience and professional background (Fedorak et al., 2003). The use of aids (vertical lines added to photographs, video that panned around a subject) failed to improve reliability of observer ratings (Bryan et al., 1990; Moran and Ljubotenski, 2006). Including symptomatic patients within an inter- and intra-rater reliability of head/neck posture observation similarly failed to increase reliability scores and led the authors to question the value of observation in patients presenting with neck pain (Silva et al., 2009b). Other issues relate to the statistical analyses used in reliability studies; concerns have been raised about reporting percentage agreement (Kaufman and Rosenthal, 2009). Subsequently, many authors adopted Cohen’s kappa statistic when dealing with discrete data. However, the existence of the ‘kappa paradox’ may create inaccurate results (Feinstein and Cicchetti, 1990). Finally there exists an additional potential error associated with interpreting the calculated k value, which is not corrected for chance agreement, against a benchmark scale (Gwet, 2010). As far as we know, no study has assessed inter- and intra-rater reliability of full body posture using photographs to assess anterior-posterior (A/P) positioning of external anatomical landmarks relative to the vertical, despite this being a widely accepted component of a comprehensive musculoskeletal assessment (Magee, 2006; Silva et al., 2009a; Koes et al., 2010). This study aimed to estimate the inter- and intra-rater reliability of osteopaths’ visual assessment of four landmarks on lateral-view photographs of a subject’s full body relative to an imagined vertical line, and to see if artificially highlighting these landmarks would alter reliability. A secondary aim was to estimate the validity of these ratings relative to a ‘gold standard’ photograph that showed the landmarks relative to a marked vertical line. Since we were calculating agreement and not hypothesis testing, power and sample size calculation were not carried out.
159
addresses. This method had the potential benefit of allowing registered osteopaths who were practising outside the UK to be selected. 2.2.2. Photographed subjects A convenience sample of adult students was recruited from the European School of Osteopathy (UK). Subjects had no visually identifiable external marks (visible tattoos, birthmarks, scars or piercings). Subjects’ gender, age, height and weight were recorded; however raters were blinded to this information. 3. Materials 3.1. Photographic image processing Two sets of identical photographs were produced as follows. All photographs were taken in a well-lit, clinical room using a camera (Prosumer G9 with f2.8e4.8, 35e210 mm lens, Canon Inc., Japan) set at a height of 150 cm using a spirit levelled tripod located 175 cm from the subject. Adhesive floor markers were used to designate tripod location and a taped reference line for positioning subjects (Pownall et al., 2008). Subjects removed outer clothing and on their left side EP placed five white skin markers (19 mm in diameter) over the lateral malleolus, lateral femoral condyle, greater femoral trochanter, acromion process and mastoid process (Dunk et al., 2005). All photographs were imported into image processing software (Adobe Photoshop Elements v6.0, USA); those with inadequate subject positioning were deleted. Remaining photographs were anonymised (a grey circle obscured subjects’ faces) and duplicated, producing two sets of identical photographs: sets A and B (Fig. 1). Using image processing software, set A photographs had all skin markers removed except the one over the lateral malleolus, which was highlighted; set B photographs had all of the skin markers highlighted. The set B photographs were then duplicated to produce a third set of photographs, set C, which had a vertical line, bisecting the lateral malleolus, added (Fig. 1). The set C photographs stood as the ‘gold standard’ and were subjected to a validation process to identify photographs with any ambiguous landmark positioning. At two occasions, 1 week apart, EP rated every landmark relative to the vertical line in the set C photographs. Throughout this process EP remained blinded to all results; only photographs with 100% landmark agreement on both occasions were retained.
2. Methods 3.2. Online questionnaire design 2.1. Study design and ethics A blinded online inter- and intra-rater reliability questionnaire design was used to measure agreement of visual assessment using lateral-view full-body photographs of standing subjects. Raters assessed position of four anatomical landmarks with respect to an imagined vertical line in two sets of photographs: 1) with the lateral malleolus highlighted 2) all landmarks highlighted. Categories for landmark position were: ‘anterior’, ‘middle’ or ‘posterior’. Rater validity was assessed against a gold standard photograph that had an actual vertical line drawn on it to show correct landmark position. Ethical approval was obtained from the European School of Osteopathy Research Ethics Committee. 2.2. Participants and procedures 2.2.1. Raters Rater recruitment was designed to randomly select and invite, by email, approximately one third of all osteopaths listed in the 2008 General Osteopathic Council’s (GOsC) register who had email
All set A and B photographs were used to pilot the online questionnaire; feedback indicated that rating 80 photographs was excessive. The final questionnaire contained 40 randomly selected photographs comprising: 20 from set A plus 10 duplicates (intrarater arm), and 10 photographs from set B. The online questionnaire utilised a survey web-site (www. kwiksurveys.com) comprising two introductory pages: page 1 contained an information/consent sheet, instructions on how to complete the questionnaire plus a guided example of what was expected; the consent section explained that should the osteopath complete the questionnaire, this would be deemed as giving consent to use their anonymised data. Page 2 captured the number of years since qualification, training college, country of practice and how frequently the respondent used visual observation in a clinical setting. The subsequent 40 pages each contained one photograph and one scale to trichotomously rate the position of all four landmarks (i.e. anterior, middle, posterior). The survey stopped raters going backwards to prevent them comparing photographs. Only data from fully completed questionnaires were used; raters with
160
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
Fig. 1. Examples of each of the 3 sets of photographs. Set A: Non-duplicate photographs used for inter-rater reliability arm; duplicate photographs used for intra-rater reliability arm. Set B: used for inter-rater reliability arm (all landmarks highlighted). Set C: used as ‘gold standard’ for validity arm.
incomplete data were treated as if they had dropped out. Raters would make 160 ratings in total. 4. Outcome data and statistical analysis All data were exported into a spreadsheet (Excel, Microsoft Corp. USA) and split into three main data sets.
(Gwet, 2010). To increase benchmarking precision, a ‘critical value’ (AC1c) was calculated and subtracted from AC1 value and the resultant ‘significant value’ (AC1s) interpreted against the Landis and Koch benchmark scale (Table 1) (Landis and Koch, 1977; Gwet, 2010). An a priori level of ‘clinically acceptable reliability’ was set at AC1c S 0.40 (Sim and Wright, 2005). 4.3. Rater validity
4.1. Inter-rater reliability Rating data obtained from 20 non-duplicate set A photographs were used to calculate inter-rater reliability of landmark position without any skin markers visible, and data from 10 set B photographs were used to calculate inter-rater reliability of anatomical landmark position when skin markers were highlighted to assist the rater. 4.2. Intra-rater reliability Rating data from 10 pairs of duplicated photographs within set A were used to calculate intra-rater reliability. The Agreement Coefficient 1 macro (AgreeStat, Advanced Analytics, USA) was used to calculate an Agreement Coefficient 1 (AC1) value for inter- and intra-rater reliability of all categorical data Table 1 Landis and Koch’s (1977) benchmark scale applied to AC1s (significant value). Significant value of AC1
Strength of agreement
0.5 more than 5% of the time (Gwet, 2010); equivalent to an intra-rater
Fig. 3. Comparison of AC1c inter-rater scores between set A (markers (M): landmarks highlighted) photographs and set B photographs (no markers (No M): landmarks without highlights); scores and 95% confidence intervals are shown for all landmarks (All) and for each individual landmark.
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
arm where one blinded rater scores 10 subjects twice. To control for this error, Gwet proposed that any calculated agreement coefficient statistic (Gwet’s AC1, Cohen’s kappa, Brennan and Prediger coefficient and Scott’s P) should have its ‘critical value’ (the 95th percentile of the agreement coefficient when the rating is performed randomly) subtracted to find the ‘significant part or value’ of the agreement statistic, which is compared to the benchmark scale. This significant value is always less than the calculated value, but reflects a more precise benchmark interpretation, especially for scores derived from the intra-rater arm. 6.2. Study limitations, strengths and proposals for future research The use of digitised skin markers to highlight anatomical landmarks can be prone to bias (Dunk et al., 2005); however our blinded double-rating procedure and resultant removal of photographs without 100% agreement from the study would have minimised this. Since raters were categorising the highlighted landmark’s position, if there was an error in skin-marker placement (when compared with an X-ray), this would not have impacted on the reliability scores. Interestingly, using X-ray as a ‘gold standard’ for positional assessment is not without critics (Marks et al., 2003; Zaina et al., 2012). Similarly, although use of full-body photographs seems more reflective of practice compared with regional views, Weatherburn et al. (2003) reports that receiver operator curve and kappa analysis of different radiographic image modes is not significantly different. Since we were assessing the reliability of observation, using photographs rather than live models does not threaten the external validity of our results. Simplifying/standardising the observation process and limiting raters to a trichotomous decision would have had a positive bias on reliability scores. Our photographic subjects were young and asymptomatic; not representative of a typical patient population. However, symptom status appears unrelated to posture (Grimmer-Somers et al., 2008) and using young subjects controlled for age-related postural changes (Yip et al., 2008). We did not control for gender, but as far as we know this is not a confounding factor. Limiting raters to GOsC practitioners resulted in a predominantly UK-trained and based sample. Although professional background and experience do not appear to influence reliability of landmark observation (Fedorak et al., 2003; Sutton et al., 2013), our raters did display heterogeneity typical of a convenience based sample and this may impact on the generalisability of our finding. A review on reliability research reported 97% of retrieved articles used a mean of two raters (Van Genderen et al., 2003). Using 48 raters lends weight to the precision of our findings: variation of the calculated agreement coefficient in studies with more than 40 raters is 5%, with less than seven raters is 30% (Gwet, 2010). The use of a paradox free statistic (AC1) and chance-correcting the resultant coefficient prior to benchmarking strengthens our results, potentially presenting a better reflection of reliability: future authors researching reliability may wish to consider using this statistical method. To increase power of reliability studies, Sim and Wright (2005) suggest that: 1) inter-rater studies increase the number of subjects rated; 2) intra-rater studies increase the number of observations each rater makes per subject. As patient symptoms often correlate poorly with observer-rated postural disturbances (Harrison et al., 1996; Fernández-de-lasPeñas et al., 2007), any significant measurable differences in postural parameters are very small and difficult for practitioners to accurately observe (Yip et al., 2008; Silva et al., 2010a). Combining the subjective nature of observation with the wide variability of patient morphology (gender, height, weight, size, age and posture),
163
undermines the notion that clinicians can reliably observe and rate landmark position in a clinically meaningful manner. Future studies could assess: a) the role of confirmation bias within manual therapy practitioners’ clinical examination skills; b) the importance manual practitioners attach to observational findings when formulating a diagnosis and treatment plan; c) whether practitioner awareness of the reliability and accuracy of observation influences their decision to use this within in a clinical setting; d) which postural features osteopaths’ eyes are drawn to using eyetracking technology. 7. Conclusion In this study, inter-rater reliability of categorising landmarks with respect to an imagined vertical reference did not reach clinically acceptable levels; although intra-rater reliability did reach levels of clinical acceptability, only 8% of raters could maintain this over repeated measures. The validity of observed landmark positions, when compared with a ‘gold standard’ was categorised as ‘low’. Our findings challenge both the use of observation to categorise anatomical landmark position reliably and the validity of using these findings in a clinical reasoning process. Manual practitioners and students should be cognisant of the subjectivity, inaccuracy and unreliability of such observational techniques; educational institutes may wish to review the importance attached to this traditional skill within their curricula and question the merit of using an ‘experienced’ practitioner to guide a ‘novice’ student’s landmark observation in a teaching-clinic scenario. Acknowledgements The authors would like to thank Brenda Mullinger for her invaluable critique of this manuscript. References Bono CM, Schoenfeld AJ, Anderson PA, Harrop JS, France J, Vaccaro AR, et al. Observer variability of radiographic measurements of C2 (axis) fractures. Spine 2010;35(12):1206e10. http://dx.doi.org/10.1097/BRS.0b013e3181d9bb58. Bryan JM, Mosner E, Shippee R, Stull MA. Investigation of the validity of postural evaluation skills in assessing lumbar lordosis using photographs of clothed subjects. J Orthop Sports Phys Ther 1990;12(1):24e9. Cook C, Turney L, Ramirez L, Miles A, Haas S, Karakostas T. Predictive factors in poor inter-rater reliability among physical therapists. J Man Manipulative Ther 2002;10(4):200e5. Donovan T, Manning DJ. The radiology task: Bayesian theory and perception. Br J Radiol 2007;80(954):389e91. Dror IE. Perception is far from perfection: the role of the brain and mind in constructing realities. Behav Brain Sci 2005;28(06):763. Dunk NM, Lalonde J, Callaghan JP. Implications for the use of postural analysis as a clinical diagnostic tool: reliability of quantifying upright standing spinal postures from photographic images. J Manipulative Physiol Ther 2005;28(6): 386e92. Farmer ER, Gonin R, Hanna MP. Discordance in the histopathologic diagnosis of melanoma and melanocytic nevi between expert pathologists. Hum Pathol 1996;27(6):528e31. Fedorak C, Ashworth N, Marshall J, Paull H. Reliability of the visual assessment of cervical and lumbar lordosis: how good are we? Spine 2003;28(16):1857e9. http://dx.doi.org/10.1097/01.BRS.0000083281.48923.BD. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. the problems of two paradoxes. J Clin Epidemiol 1990;43(6):543e9. Fernández-de-las-Peñas C, Alonso-Blanco C, Cuadrado ML, Pareja JA. Neck mobility and forward head posture are not related to headache parameters in chronic tension-type headache. Cephalalgia 2007;27(2):158e64. Field A. Correlation, discovering statistics using SPSS. London, UK: Sage Publications; 2005. p. 1e779. Fryer G, McPherson HC, O’Keefe P. The effect of training on the inter-examiner and intra-examiner reliability of the seated flexion test and assessment of pelvic anatomical landmarks with palpation. Int J Osteopath Med 2005;8(4):131e8. Gibbons P, Tehan P. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):21e3.
164
E. Pattyn, D. Rajendran / Manual Therapy 19 (2014) 158e164
Goossen S. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):23e5. Greenman P. Structural diagnosis, principles of manual medicine. 2nd ed. Maryland: Williams and Wilkins; 1996. p. 13e38. Grimmer-Somers K, Milanese S, Louw Q. Measurement of cervical posture in the sagittal plane. J Manipulative Physiol Ther 2008;31(7):509e17. Gwet KL. Handbook of inter-rater reliability. The definitive guide to measuring the extent of agreement among multiple raters. 2nd ed. Gaithersburg, USA: Advanced Analytics; 2010. p. 1e197. Hall TE. Posterior type, normal, anterior type. In: Wernham SGJ, editor. The 1956 year book. Maidstone, UK: The Osteopathic Institute of Applied Technique; 1955. p. 26e39. Haneline MT, Young M. A review of intraexaminer and interexaminer reliability of static spinal palpation: a literature synthesis. J Manipulative Physiol Ther 2009;32(5):379. Harrison AL, Barry-Greb T, Wojtowicz G. Clinical measurement of head and shoulder posture variables. J Orthop Sports Phys Ther 1996;23(6):353. Hayhoe M, Ballard D. Eye movements in natural behavior. Trends Cogn Sci 2005;9(4):188e94. Houglum P, Delores P. Brunnstrom’s clinical kinesiology. Philadelphia: FA Davis; 2012. p. 1e704. Johnson J. Postural assessment. Leeds: Human Kinetics; 2012. p. 1e184. Kaufman AB, Rosenthal R. Can you believe my eyes? The importance of interobserver reliability statistics in observations of animal behaviour. Anim Behav 2009;78(6):1487e91. Kmita A, Lucas NP. Reliability of physical examination to assess asymmetry of anatomical landmarks indicative of pelvic somatic dysfunction in subjects with and without low back pain. Int J Osteopath Med 2008;11(1):16e25. Koes BW, van Tulder M, Lin CW, Macedo LG, McAuley J, Maher C. An updated overview of clinical guidelines for the management of non-specific low back pain in primary care. Eur Spine J 2010;19(12):2075e94. Koning CP, Heuvel S, Staal JB, Smits-Engelsman BM, Hendriks EM. Clinimetric evaluation of active range of motion measures in patients with non-specific neck pain: a systematic review. Eur Spine J 2008;17(7):905e21. Krebs DE, Edelstein JE, Fishman S. Reliability of observational kinematic gait analysis. Phys Ther 1985;65(7):1027e33. Krupinski EA, Berger WG, Dallas WJ, Roehrig H. Searching for nodules: what features attract attention and influence detection? Acad Radiol 2003;10(8):861e8. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977:159e74. Lee D. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):26e7. Magee D. Assessment of posture, orthopaedic physical assessment e enhanced edition. Missouri: Saunders Elsevier; 2006. p. 873e903. Marks MC, Stanford CF, Mahar AT, Newton PO. Standing lateral radiographic positioning does not represent customary standing balance. Spine (Phila Pa 1976) 2003;28(11):1176e82. Moran R, Ljubotenski E. Reliability of visual assessment for lumbar curves in standing: does clinical experience matter? Int J Osteopath Med 2006;9(1):42. Murphy DR. The palpation reliability debate: the experts respond. J Bodyw Move Therapies 2002;6(1):30e4. Nitschke JE, Nattrass CL, Disler PB, Chou MJ, Ooi KT. Reliability of the American Medical Association guides’ model for measuring spinal range of motion. Its
implication for whole-person impairment rating. Spine (Phila Pa 1976) 1999;24(3):262e8. Paatelma M, Karvonen E, Heinonen A. Inter- and intra-tester reliability of selected clinical tests in examining patients with early phase lumbar spine and sacroiliac joint pain and dysfunction. Adv Physiotherapy 2010;12(2):74e80. Pett MA. Nonparametric statistics for health care research: statistics for small samples and unusual distributions. California, USA: Sage Publications; 1997. Pownall PJ, Moran RW, Stewart AM. Consistency of standing and seated posture of asymptomatic male adults over a one-week interval: a digital camera analysis of multiple landmarks. Int J Osteopath Med 2008;11(2):43e51. Rajendran D, Gallagher D. The assessment of pelvic landmarks using palpation: a reliability study of undergraduate students. Int J Osteopath Med 2011;14(2): 57e60. Silva A, Punt D, Johnson M. A postal survey gathering information about physiotherapists’ assessment of forward head posture for patients with chronic idiopathic neck pain. Eur J Pain 2009a;13(S1). S223a-S. Silva A, Sharples P, Johnson M. Studies comparing surrogate measures for head posture in individuals with and without neck pain. Phys Ther Rev 2010a;15(1): 12e22. Silva AG, Punt TD, Johnson MI. Reliability and validity of head posture assessment by observation and a four-category scale. Man Ther 2010b;15(5):490e5. Silva AG, Punt TD, Sharples P, Vilas-Boas JP, Johnson MI. Head posture assessment for patients with neck pain: Is it useful? Int J Ther Rehabil 2009b;16(1):43e53. Simons DG. The palpation reliability debate: the experts respond. J Bodywork Move Therapies 2002;6(1):34e6. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005;85(3):257e68. Sutton C, Nono L, Johnston RG, Thomson OP. The effects of experience on the interreliability of osteopaths to detect changes in posterior superior iliac spine levels using a hidden heel wedge. J Bodyw Move Therapies 2013;17(2):143e50. Szeto GPY, Straker L, Raine S. A field comparison of neck and shoulder postures in symptomatic and asymptomatic office workers. Appl Ergon 2002;33(1):75e84. Taylor M, Hipp JA, Gertzbein SD, Gopinath S, Reitman CA. Observer agreement in assessing flexion-extension X-rays of the cervical spine, with and without the use of quantitative measurements of intervertebral motion. Spine J 2007;7(6): 654e8. Tsuji T, Matsuyama Y, Goto M, Yimin Y, Sato K, Hasegawa Y, et al. Kneeespine syndrome: correlation between sacral inclination and patellofemoral joint pain. J Orthop Sci 2002;7(5):519e23. Van Genderen F, De Bie R, Helders P, Van Meeteren N. Reliability research: towards a more clinically relevant approach. Phys Ther Rev 2003;8(4):169e76. Weatherburn GC, Ridout D, Strickland NH, Robins P, Glastonbury CM, Curati W, et al. A comparison of conventional film, CR hard copy and PACS soft copy images of the chest: analyses of ROC curves and inter-observer agreement. Eur J Radiol 2003;47(3):206e14. Wernham SGJ. Mechanics of the spine. In: Wernham SGJ, editor. The 1956 year book. Maidstone, UK: The Osteopathic Institute of Applied Technique; 1955. p. 26e39. Yip CHT, Chiu TTW, Poon ATK. The relationship between head posture and severity and disability of patients with neck pain. Man Ther 2008;13(2):148e54. Zaina F, Pizzetti P, Donzelli S, Negrini F, Negrini S. Why X-rays are not reliable to assess sagittal profile: a cross sectional study. Stud Health Technol Inform 2012;176:268e72.
