Annotation of Figures from the Biomedical Imaging Literature: A Comparative Analysis of RadLex and Other Standardized Vocabularies Charles E. Kahn, Jr., MD, MS Rationale and Objectives: RadLex is a standardized vocabulary developed for clinical practice, research, and education in radiology. This report sought to analyze the use of RadLex to annotate and index the captions of images from the peer-reviewed biomedical literature and to compare the number of annotations per term for RadLex and five other biomedical ontologies in a large corpus of figure captions from biomedical imaging publications. Materials and Methods: RadLex and five other biomedical vocabularies were evaluated. A fully automated web service was used to discover the vocabularies’ terms in a collection of 385,018 figure captions from 613 peer-reviewed biomedical journals. Annotations (i.e., figure-term pairs) were analyzed by vocabulary. RadLex annotations were analyzed by journal and RadLex term class. Results: RadLex had the greatest number of annotations per term of the six vocabularies. On average, there were 10.1 RadLex annotations per figure; 380,338 figures (98.8%) were annotated with at least one RadLex term and 288,163 figures (74.8%) were annotated with six or more RadLex terms. Of 39,218 RadLex terms, 8504 (21.7%) were mapped to images in the collection, which was the highest percentage of any of the vocabularies. Conclusions: Although comprising four to 10 times fewer terms than other vocabularies, RadLex showed excellent performance in indexing radiology-centric content. Almost all of the images in a large collection of figures from peer-reviewed biomedical journals were annotated with at least one RadLex term, and almost 75% of the images were annotated with six or more terms. Key Words: RadLex; lexicon; ontology; vocabulary; indexing; figure captions; annotation; ARRS GoldMiner. ªAUR, 2014

T

he RadLex radiology lexicon (www.radlex.org) is a collection of terms designed to provide a uniform vocabulary of radiology (1). It has been developed under the aegis of the Radiological Society of North America and plays an increasingly important role in radiology practice, research, and education. RadLex is being used to categorize journal articles and reviewers (2), encode radiology result information (3,4), search the content of radiology reports (5), analyze queries to web-based search engines (6), and standardize names of imaging procedures for the American College of Radiology’s Dose Index Registry (7). RadLex provides an ontology (knowledge model) that incorporates relationships between terms (8). The main relationship is the subsumption (‘‘is-a’’) relation, which defines a hierarchy of subclasses. For example, the term ‘‘left lung’’ RadLex term identifier (RID1326) is a subclass of ‘‘lung’’ (RID1301), and ‘‘bronchitis’’ (RID34637) is a subclass of ‘‘respiratory disorder’’ (RID5316). In RadLex, each entity is related

Acad Radiol 2014; 21:384–392 From the Department of Radiology, Medical College of Wisconsin, 9200 W. Wisconsin Ave., Milwaukee, WI 53226. Received September 28, 2013; accepted November 3, 2013. Address correspondence to: C.E.K. e-mail: [email protected] ªAUR, 2014 http://dx.doi.org/10.1016/j.acra.2013.11.007

384

to a single, higher level entity, or ‘‘parent.’’ This hierarchical representation of radiology terms allows retrieval of information more effectively. One of the primary intended purposes of RadLex is to index resources for research and education in radiology. This study explored the extent to which RadLex could be used to annotate and index a large database of biomedical images. The study’s goal was to better understand the scope and coverage of RadLex in comparison to other, more widely used biomedical ontologies.

MATERIALS AND METHODS With permission of the American Roentgen Ray Society (ARRS), the database of the ARRS GoldMiner image search engine (goldminer.arrs.org) provided the materials for this investigation. This search engine focuses on clinically relevant images, particularly those that use medical imaging technologies (9). ARRS GoldMiner indexes images from a core set of radiology journals (AJR American Journal of Roentgenology, American Journal of Neuroradiology, British Journal of Radiology, Journal of Nuclear Medicine, Radiology, and RadioGraphics) and the European Society of Radiology’s Eurorad case database. ARRS GoldMiner also includes selected images from other journals, including articles submitted to the US National

Academic Radiology, Vol 21, No 3, March 2014

RADLEX ANNOTATION OF IMAGING JOURNAL FIGURES

TABLE 1. Comparison of Six Ontologies for Annotation of the ARRS GoldMiner Corpus of Figure Captions Using the Most Recent Version of Each Ontology at the NCBO BioPortal Site Annotated Terms Ontology FMA ICD-10-CM LOINC MeSH RadLex SNOMED CT Total

Annotated Figures

Annotations

Version

Release Date

No. of Terms

Number

Percent

Number

Percent

Number

Per Term

Per Figure

3.1 2011_01 236 2012 3.8 2011_07_31

March 3, 2010 January 1, 2011 June 1, 2011 September 9, 2011 February 19, 2013 July 31, 2011

83,281 91,590 171,399 229,698 39,218 395,036 1,010,222

5398 1635 7683 15,792 8504 41,371 80,383

6.5 1.8 4.5 6.9 21.7 10.5 8.0

324,376 84,987 380,834 381,978 380,338 384,492 385,018

84.2 22.1 98.9 99.2 98.8 99.9

1,288,568 104,095 5,008,536 3,097,452 3,871,573 11,588,578 24,958,802

15.5 1.1 29.2 13.5 98.7 29.3 24.7

3.3 0.3 13.0 8.0 10.1 30.1 64.8

FMA, Foundational Model of Anatomy; ICD-10-CM, International Classification of Diseases, Version 10, Clinical Modification; LOINC, Logical Observation Identifier Names and Codes; MeSH, Medical Subject Headings; SNOMED CT, Systematized Nomenclature of Medicine–Clinical Terms. The Annotated Terms column shows the number of terms from each ontology that appeared in the annotations. The Annotated Figures column shows the number of figures captions from the collection that were annotated.

Library of Medicine’s open-access PubMed Central collection. All of the journals are indexed by PubMed and provide the images as part of articles that are made freely available through the web, usually after a subscriber-only period of 6 to 24 months. As of May 2013, ARRS GoldMiner contained information about 385,018 figures from 613 peer-reviewed biomedical journals. Available information included each figure’s source, caption, and the title of article in which the figure appeared. For the purposes of the current study, the textual information for each figure consisted of the figure’s caption text concatenated to the title of the article in which the figure appeared. To provide a point of comparison for the performance of RadLex, five other widely used biomedical ontologies were explored. The Foundational Model of Anatomy (FMA) is an anatomical reference vocabulary (10), part of which has been incorporated into RadLex. The International Classification of Diseases, 10th version, Clinical Modification, includes standardized terms and codes for symptoms, signs, abnormal findings, and diseases; it is based on the medical classification list developed under the auspices of the World Health Organization, modified for clinical use in the United States (11). The Logical Observation Identifiers Names and Codes (LOINC) lexicon provides a standard for identifying medical laboratory procedures and observations (12,13). The Medical Subject Headings (MeSH) vocabulary is used by the US National Library of Medicine to index the biomedical literature (14). The Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) offers a standard to encode the meanings of health information and supports effective recording of clinical data (15). The National Center for Biomedical Ontology (NCBO) BioPortal (16,17) provided current versions of all six ontologies (Table 1). Except for RadLex, all of the ontologies are part of the U.S. National Library of Medicine’s Unified Medical Language System (18).

The NCBO Annotator (19) was used to discover terms from these six ontologies in the article title and figure caption text for each entry in the ARRS GoldMiner database. This process, called ‘‘annotation,’’ produced a list of terms associated with each figure. This fully automated web service identified the appearance of ontology terms in a block of text, which was passed to the server through an application programming interface using Simple Object Access Protocol and Representational State Transfer architecture. The NCBO Annotator web service presents the input text to a concept recognition tool along with a dictionary that consists of all concept names from the specified ontologies and other string forms, such as synonyms or labels that syntactically identify the concepts. The Annotator recognizes concepts using string matching on the dictionary to produce a set of direct annotations. For terms that have synonyms or abbreviations, the preferred term is used in the output. The current work did not use the Annotator’s semantic expansion components, which allow the system to use ‘‘is-a’’ relation transitivity, semantic distance, and/or cross-ontology mapping to expand the set of annotations to include parent concepts or related concepts from other ontologies. The appearance of a term was counted only once for each figure in the database, even if that term or a synonym appeared more than once in the figure’s caption text or article title. The number of annotations per figure was tallied to ascertain the extent to which each ontology covered the content of figure captions. Conversely, the number of annotations per term was assessed as well. For RadLex terms, the possibility of a power-law relationship between the number of annotations per term and the frequency of such terms was explored. To better understand the use of RadLex terms, they were divided into major classes. A major RadLex class was defined as a term whose parent was the top-level term ‘‘RadLex entity’’ (RID1). For example, ‘‘anatomical entity’’ (RID3) is-a ‘‘RadLex entity’’ (RID1) and hence was defined here as 385

KAHN

Academic Radiology, Vol 21, No 3, March 2014

Figure 1. Distribution of the number of RadLex annotations per figure. The distribution has a mode of seven annotations per figure and a median of nine annotations per figure; the mean value is 10.1 annotations per figure. Only 1.2% of figures lack RadLex annotation.

a major class. For each major RadLex class, its descendants were identified by following the subsumption hierarchy. We computed the number of descendants, the percentage that had annotations, and the mean number of annotations per term. Radiology ‘‘core sources’’ were defined as the six primary radiology journals plus the Eurorad case database. We tested the hypothesis that the frequency of RadLex annotations among figures in this core set of radiology content would be greater than that in the entire collection. To test whether the number of annotations might reflect longer titles and/or figure captions in the set of core journals, we computed the number of words in each figure’s caption and article title. RESULTS Of the 1,010,980 terms from the six ontologies, the annotation process identified 80,383 terms in the corpus of figure captions (Table 1). There were a total of 24,958,802 annotations (mean, 24.7 annotations per term), of which RadLex had the greatest number of annotations per term (98.7 annotations per term). Four of the ontologies—LOINC, MeSH, RadLex, and SNOMED CT—annotated more than 98% of the figures with at least one term. RadLex had the fewest terms of the six ontologies investigated and had the greatest percentage of terms (8504 of 39,218 terms; 21.7%) to appear in annotations. The automated annotation process required 46.8 hours of real time. For RadLex, 3,871,573 annotations were discovered. The number of RadLex terms per figure ranged from 0 to 72 (mean, 10.1 annotations per figure). Of the 385,018 figures, 380,338 (98.8%) were annotated with one or more RadLex terms, 288,163 figures (74.8%) had six or more annotations, and 30,236 figures (7.9%) had 20 or more annotations. The distribution of RadLex annotations per figure is shown in 386

Figure 1. The number of annotations per RadLex term ranged from 0 to 71,190 (mean, 98.7 annotations per term). Linear regression (applied to the logarithm of both terms) interpolated a power-law relationship between the number of annotations per RadLex term (a) and the number of terms (t), t ¼ 499:69,a0:889 : This relationship’s correlation coefficient (r) of 0.8968 indicates strong agreement with the underlying distribution of annotations per RadLex term (Fig 2). There were notable differences among the major classes of RadLex terms in the frequency of annotations (Table 2). Imaging modalities (e.g., computed tomography, magnetic resonance imaging), clinical findings, processes, and RadLex descriptors were the classes of terms that most frequently were used to annotate the GoldMiner images. Only 55 terms belonged to the ‘‘imaging modality’’ class, but 46 (84%) of them had associated annotations, with a mean of 1717 annotations per term. In contradistinction, the 32,728 ‘‘anatomical entity’’ terms constituted 83% of all RadLex terms, but only 4458 (14%) of them had associated annotations. Appendix A details for each major class the 10 most frequently annotated RadLex terms lists a sample of 10 terms that did not appear in annotations. The seven core radiology sources accounted for 249,192 (64.7%) of the images indexed by ARRS GoldMiner (Table 3). These sources had a mean of 10.8 annotations per figure, which was significantly greater than the mean of 8.7 annotations per figure among non-core journals; a chi-squared test showed a significant difference at P < 10100. The ‘‘core’’ sources had a mean of 54.5 words per figure, which was significantly less than 76.1 words per figure for the noncore journals. The core journals accounted for 13,592,119 of 23,921,767 (56.8%) of the words of title and caption text and 2,695,277

Academic Radiology, Vol 21, No 3, March 2014

RADLEX ANNOTATION OF IMAGING JOURNAL FIGURES

Figure 2. Scatterplot of the frequency of figures annotated per RadLex term and the number of RadLex terms with that property, plotted as a logarithmic (‘‘log-log’’) graph. Among RadLex terms, 30,729 (78.4%) had no associated annotations; 1071 terms had one annotation; and 556 terms had two annotations. A trendline, based on a power-law relationship, is plotted for reference; the equation is specified in the text. TABLE 2. Frequency of Annotations for Major RadLex Classes Terms with Annotations Major RadLex Class Anatomical entity Clinical finding Imaging modality Imaging observation Imaging procedure attribute Medical device Nonanatomical substance Object Procedure Procedure step Process Property RadLex descriptor RadLex nonanatomical set Report component Total

Number of Terms

Number

Percent

Mean Number of Annotations per Term

32,728 2146 55 932 878

4458 1587 46 400 322

14 74 84 43 37

30 247 1717 277 81

325 366 45 423 97 6 234 886 4

168 205 28 278 57 6 153 742 0

52 56 62 66 59 100 65 84 0

126 75 43 397 451 1086 801 1598 —

18

11

61

1157

39,143

8461

22

98

of 3,871,573 (69.6%) of the annotations. Table 4 presents the 20 RadLex terms that appeared most frequently in the concatenated article title and figure captions. DISCUSSION RadLex was developed to address a significant gap in vocabulary coverage of radiology terms (1). Numerous vocabularies for biomedicine have been developed over the past 25 years. The Unified Medical Language System is a large, wellrecognized terminology resource that includes SNOMED CT, MeSH, and other biomedical coding schemes (18). The Unified Medical Language System has been shown to contain

up to 80% of clinical terms used in outpatient medical records, but no more than 50% of terms from a sample of radiology text corpora (20). RadLex provided near-complete coverage in annotating the source material. RadLex was one of four ontologies that appeared at least once in more than 98% of all figure captions. Almost all of the entries in a representative sample of figures from peer-reviewed biomedical journals could be annotated with at least one RadLex term, and almost three-quarters of the images were annotated with six or more terms. This result suggests that RadLex provides useful coverage for indexing text associated with clinical images in biomedical journals. The significantly greater number of RadLex annotations for 387

KAHN

Academic Radiology, Vol 21, No 3, March 2014

TABLE 3. RadLex Figure Annotations by Source

Source American Journal of Neuroradiology American Journal of Roentgenology British Journal of Radiology Eurorad Journal of Nuclear Medicine RadioGraphics Radiology ‘‘Core’’ sources Other sources Total

Number of Figures

Number of Words

Mean Words per Figure

Number of RadLex Annotations

Mean RadLex Annotations per Figure

16,210

1,284,355

79.2

215,125

13.3

77,998

3,954,639

50.7

902,100

11.6

8831

500,368

56.7

90,346

10.2

22,220 13,300

648,241 887,357

29.2 66.7

191,269 99,432

8.6 7.5

51,841 58,792 249,192 135,826 385,018

3,143,696 3,173,463 13,592,119 10,329,648 23,921,767

60.6 54.0 54.5 76.1 62.1

540,659 656,346 2,695,277 1,176,296 3,871,573

10.4 11.2 10.8 8.7 10.1

The seven ‘‘core’’ radiology sources accounted for almost two-thirds of all figures. Although they had fewer words per figure, they had a greater number of RadLex annotations per figure. TABLE 4. The 20 Most Frequently Annotated RadLex Terms RadLex Identifier

Term Name (Synonyms)

Number of Figures Annotated

RID5824 RID39330 RID5825 RID10312

Left After Right Magnetic resonance imaging (MRI) Breast mass (mass; nodule) Artery Mass Liver mass (mass) Lung mass (mass) Mass in or on skin (mass) Normal Coronal Neoplasm Treatment Lesion Anterior Malignant neoplastic disease (cancer) Cell Carcinoma Sagittal

71,190 68,807 68,309 39,034

RID39055 RID478 RID3874 RID39466 RID39056 RID34373 RID13173 RID5861 RID3957 RID8 RID38780 RID5818 RID34616 RID39348 RID4247 RID5860

35,150 33,224 30,349 30,349 30,349 30,349 28,438 28,190 26,234 26,014 24,638 23,676 21,393 21,034 21,034 20,943

Note that all of these terms consist of a single word or have a singleword synonym, which likely accounts for their high frequency in the figure captions. Three terms that have the synonym ‘‘mass’’ (RID39466, RID39056, and RID34373) have the same number of annotations as term RID3874 (‘‘mass’’).

images from core radiology sources, despite the presence of relatively fewer words in their figure captions, is concordant with RadLex’s focus on the domain of radiology. 388

Compared to other biomedical ontologies, RadLex showed excellent performance in annotating the text of figure captions from biomedical journals. RadLex had the greatest number of annotations per term (98.7 annotations per RadLex term vs. 24.7 annotations per term overall). In evaluating the application of ontologies, the goal is not to determine which ontology is best, because that concept has little meaning. SNOMED CT contains the names of many diseases, drugs, symptoms, physical findings, laboratory tests, and other terms that are not included in RadLex, along with their semantic relationships; thus, there is no need to copy those terms into RadLex. However, many terms used in medical imaging—such as imaging signs (e.g., tree-inbud opacity) or anatomical references (e.g., upper lung fields) were not part of any ontology before RadLex. Although SNOMED CT accounted for almost half of all observed annotations and has 10 times as many terms as RadLex, in annotating the set of figures, RadLex had three times more annotations per term. The major classes of RadLex terms exhibited variation in their numbers of annotations. Although anatomical entities constitute the majority of terms in RadLex (84%), they had relatively few annotations. One may surmise that many of these highly specific terms, such as ‘‘medial part of left medial mammillary nucleus,’’ did not appear in the text corpus. Most of the RadLex anatomic terms are derived from the FMA (10). However, because of their formal naming convention, FMA terms (e.g., upper lobe of right lung) may not be detected verbatim in a figure caption or in clinical text. Although RadLex uses the FMA term, it also includes the synonym ‘‘right upper lobe’’ and abbreviation ‘‘RUL’’ to allow automated systems, such as the NCBO Annotator, to recognize the term more frequently in clinical text. The current study did not evaluate the performance of the annotation service using standard information-retrieval metrics such as precision and recall, which are analogues of

Academic Radiology, Vol 21, No 3, March 2014

RADLEX ANNOTATION OF IMAGING JOURNAL FIGURES

positive predictive value and sensitivity, respectively. However, investigators have used the MetaMap Transfer concept recognizer (21) to annotate a subset of the present database with SNOMED CT and MeSH terms and have demonstrated precision of 0.897 and recall of 0.930 (22). The NCBO Annotator, used in the present investigation and based on the Mgrep concept recognizer (23), has been found superior to MetaMap in terms of precision, recall, speed of execution, scalability, and customizability (24). The annotation process may identify too many or too few ontology terms from the specified text. If an ontology lacks synonyms, concepts that should match will not be found. Conversely, because annotation uses string matching, it may match too many concepts. For example, from the text string ‘‘left lung,’’ the NCBO Annotator identified four SNOMED CT terms (left lung structure, left, entire lung, and lung structure). The annotation process does not disambiguate words or phrases with more than one meaning; thus the string ‘‘. was left for several days.’’ would match to the concept ‘‘left.’’ The rapid growth of biomedical information has posed new challenges for physicians and researchers. By helping to bring structure to that information, ontologies such as RadLex can aid clinical practice, medical education, and biomedical research. Ontologies expand the range of functions for processing information: they allow search and query of heterogeneous biomedical datasets, promote integration and exchange of data among applications, support representation of encyclopedic knowledge, enable sophisticated natural language processing techniques, and facilitate automated reasoning (25,26). NCBO BioPortal provides access to 352 ontologies—including SNOMED CT, MeSH, FMA, the Gene Ontology, and RadLex—comprising more than 5.7 million terms (17,27). The Open Biomedical Ontology Foundry seeks to coordinate ontology design so that individual ontologies are logically well-formed efforts and can be interoperable (28). Guidelines have been proposed to harmonize the approaches taken by ontology developers and promote knowledge sharing through the Semantic Web (29). These efforts to coordinate ontologies and to annotate diverse datasets, including microarray experiments and PubMed abstracts, underpin advances in translational research (30). Annotation of radiology-centric figures from biomedical journals contributes to the resources available for integration into research efforts.

4. Channin DS, Mongkolwat P, Kleper V, et al. The annotation and image mark-up project. Radiology 2009; 253:590–592. 5. Lacson R, Andriole KP, Prevedello LM, et al. Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT). J Digit Imaging 2012; 25:512–519. 6. Rubin DL, Flanders A, Kim W, et al. Ontology-assisted analysis of web queries to determine the knowledge radiologists seek. J Digit Imaging 2011; 24:160–164. 7. Morin RL, Coombs LP, Chatfield MB. ACR Dose Index Registry. J Am Coll Radiol 2011; 8:288–291. 8. Rubin DL. Creating and curating a terminology for radiology: ontology modeling and analysis. J Digit Imaging 2008; 21:355–362. 9. Kahn CE Jr, Thao C. GoldMiner: a radiology image search engine. AJR Am J Roentgenol 2007; 188:1475–1478. 10. Rosse C, Mejino JL Jr. A reference ontology for biomedical informatics: the foundational model of anatomy. J Biomed Inform 2003; 36:478–500. 11. Barta A, McNeill G, Meli P, et al. ICD-10-CM primer. J AHIMA 2008; 79: 64–66. quiz 67–68. 12. Huff SM, Rocha RA, McDonald CJ, et al. Development of the Logical Observation Identifier Names and Codes (LOINC) vocabulary. J Am Med Inform Assoc 1998; 5:276–292. 13. McDonald CJ, Huff SM, Suico JG, et al. LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clin Chem 2003; 49:624–633. 14. Medical Subject Headings. National Library of Medicine; 2007. Available at: http://www.nlm.nih.gov/mesh/. Accessed October 3, 2007. 15. SNOMED CT User Guide. Copenhagen: International Health Terminology Standards Development Organisation, 2012. 16. Musen MA, Noy NF, Shah NH, et al. The National Center for Biomedical Ontology. J Am Med Inform Assoc 2012; 19:190–195. 17. Whetzel PL, Noy NF, Shah NH, et al. BioPortal: enhanced functionality via new web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res 2011; 39(Web Server issue):W541–W545. 18. Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004; 32(Database issue): D267–D270. 19. Jonquet C, Shah NH, Musen MA. The open biomedical annotator. Summit Translat Bioinform 2009;56–60. 20. Langlotz CP, Caldwell SA. The completeness of existing lexicons for representing radiology report information. J Digit Imaging 2002; 15(Suppl 1):201–205. 21. Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp 2001;17–21. 22. Kahn CE Jr, Rubin DL. Improving radiology image retrieval through automated semantic indexing of figure captions. J Am Med Informatics Assoc 2009; 16:370–376. 23. Dai M, Shah NH, Xuan W, et al. An efficient solution for mapping free text to ontology terms. AMIA Summit on Translational Bioinformatics. San Francisco, CA; 2008. 24. Shah NH, Bhatia N, Jonquet C, et al. Comparison of concept recognizers for building the Open Biomedical Annotator. BMC Bioinform 2009; 10(Suppl 9):S14. 25. Rubin DL, Shah NH, Noy NF. Biomedical ontologies: a functional perspective. Brief Bioinform 2008; 9:75–90. 26. Bodenreider O. Biomedical ontologies in action: role in knowledge management, data integration and decision support. Yearbk Med Inform 2008;67–79. 27. Musen MA, Shah NH, Noy NF, et al. BioPortal: ontologies and data resources with the click of a mouse. AMIA Annu Symp Proc 2008;1223–1224. 28. Smith B, Ashburner M, Rosse C, et al. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 2007; 25:1251–1255. 29. Tao C, Pathak J, Solbrig HR, et al. Terminology representation guidelines for biomedical ontologies in the semantic web notations. J Biomed Inform 2013; 46:128–138. 30. Shah NH, Jonquet C, Chiang AP, et al. Ontology-driven indexing of public datasets for translational bioinformatics. BMC Bioinform 2009; 10 (Suppl 2):S1.

REFERENCES 1. Langlotz CP. RadLex: a new method for indexing online educational materials. RadioGraphics 2006; 26:1595–1597. 2. Klein JS. A look back at 2012 and plans for 2013. RadioGraphics 2013; 33: 1–2. 3. Kahn CE Jr, Langlotz CP, Burnside ES, et al. Toward best practices in radiology reporting. Radiology 2009; 252:852–856.

389

KAHN

Academic Radiology, Vol 21, No 3, March 2014

APPENDIX

Terms with the Most Annotations Major Class anatomical entity

clinical finding

No. of Figures Annotated 33,224 21,034 18,497 18,337

artery cell brain liver

17,752 16,500 16,233 16,175 15,534

wall human portion of tissue lung thorax

14,111

breast

26,234 21,393 21,034 14,311 10,743 9,468

neoplasm malignant neoplastic disease carcinoma pain stenosis obstruction

7,999 7,581

imaging modality

6,508 6,389 39,034 14,295 11,215 5,713 5,585 2,723 2,477 2,234 1,923 1,518

imaging observation

imaging procedure attribute

Term

35,150 30,349 30,349 30,349 30,349 24,638 19,871 13,683 10,816 5,504 8,321 5,518 4,181 3,978

infarction thickening aneurysm hemorrhage magnetic resonance imaging ultrasound tomography computed tomography projection radiography mammography spectroscopy positron emission tomography scintigraphy functional magnetic resonance imaging breast mass liver mass lung mass mass mass in or on skin lesion enhancement assessment cirrhosis-associated nodules enhancing multi-detector radiographic projection regression arterial phase

Terms without Annotations (randomly selected) Term Brodmann area 13 of left temporal lobe hyaloid canal long ciliary nerve to left ciliary body medial part of left medial mammillary nucleus muscle body of right pronator teres region of prostatic sinus subaortic common iliac lymph node surface of left occipital lobe tendon of left extensor digitorum to left little finger thoracic subsegment of dorsal gray column of spinal cord aspiration of digestive secretions complicated cysts congenital agammaglobulinemia congenital anomaly of spine disorder caused by drugs or toxins focal cortical dysplasia without balloon cells fracture healing disorder injury due to thermal or electrical trauma melanocytic medulloblastoma type 2 endplate marrow change conventional tomography high intensity pulse high-energy scan low intensity pulse low-energy scan panographic radiograph phase-cancellation harmonic ultrasound planar nuclear medicine imaging single photon imaging

cockade image deflected calyx sign hilum convergence sign lace-like pattern of joint paradoxical halo sign pelvocalyceal wall opacification sign popcorn calcification sign spatial enhancement pattern waterfall hilum yo-yo-on-a-string sign arrhythmia artifact closure device access closure technique dual interpretation duration of ablation endpoint (continued)

390

Academic Radiology, Vol 21, No 3, March 2014

RADLEX ANNOTATION OF IMAGING JOURNAL FIGURES

(continued)

Terms with the Most Annotations Major Class

medical device

non-anatomical substance

object

procedure

procedure step

No. of Figures Annotated

Term

3,535 3,008 2,836 2,655 1,973 1,919 5,203 5,153 4,957 4,623 3,741 2,835 2,125 1,202 1,106 1,091 3,032 2,571 1,689

fast-spin echo flip angle artifact fluid-attenuated inversion recovery portal venous phase t1 weighted continuously rotating tube ct valve stent catheter needle tube balloon electrode prosthesis filter gadolinium contrast agent iron

1,633 457 377 225 216 134 134 46 45 41 33 26,014 17,989 11,468 8,909 8,630 5,969 5,751 5,669 5,342 4,724 18,647 5,720 4,391 3,155 1,932 1,669 1,143 967 853 749

Terms without Annotations (randomly selected) Term

gaynor hart view immediate complication microscopic motion artifact response to non-vascular treatment settegast view wrong demographics ACL graft device biopsy coil cryoplasty balloon hybrid fixator low-pressure balloon monopolar applicator non-linear shim coil nuclear imaging device temporary pacemaker patch xenon ionization chamber americium I-123 carcinoembryonic antigen ionic low-osmolality dimeric iodinated contrast agent calcium Meitnerium foreign body bevel needle tip needle tip diamond needle tip adhesive flexible diffuser laser tip knife Hemostatic sheath bullet medical object intrauterine device personal item coin platinum embolization coil bezoar serrated needle tip embolic material steerable wire tip shrapnel straight wire tip treatment three phase imaging angiography nuchal translucency evaluation follow-up procedure obtain access surgical procedure ultrasound-assisted venipuncture biopsy platelet-rich plasma injection ablation extraction of biliary calculi diffusion imaging with iv contrast echocardiography nephrotomography placement imaging without iv contrast screening small bowel transplantation mean value calculation counts per pixel calculation average value calculation diffusion tensor reconstruction maximum intensity projection hardware image fusion with reregistration apparent diffusion coefficient map image-based parallel imaging reconstruction intensity projection iterative partial fourier reconstruction multiplanar reformat k-space filtering volume rendering primary reconstruction step calculation reformat procedure step cerebral blood flow map trace of diffusion tensor reconstruction tractography zero-filling of 3d volume k-space data (continued)

391

KAHN

Academic Radiology, Vol 21, No 3, March 2014

(continued)

Terms with the Most Annotations Major Class process

property

RadLex descriptor

392

No. of Figures Annotated

Term

3,900 1,238 1,163 146 67 4 16,785 15,024 13,832 13,374 11,143

motion metabolism gestation swallowing peristalsis biological process diagnosis flow volume phase function

7,235 7,156 6,453 6,277 6,136 71,190 68,807 68,309 28,438 28,190 23,676

measurement color maximum location typical left after right normal coronal anterior

20,943 20,098 19,974 19,888

sagittal transverse small both

Terms without Annotations (randomly selected) Term

10 o’clock position 2 o’clock position 3 o’clock position arm pedaling exercise diffusely increased vascularity in surrounding tissue intermediate perceptual difficulty mimics another entity rolled medial t2 tumor stage vascular flow measurement mode aortic dissection modifier body region covered disc containment displaces portal vein extra-amniotic hormonal pathophysiologic process modifier intrameningeal polyp morphology characteristic quality descriptor turbidity modifier

Annotation of figures from the biomedical imaging literature: a comparative analysis of RadLex and other standardized vocabularies.

RadLex is a standardized vocabulary developed for clinical practice, research, and education in radiology. This report sought to analyze the use of Ra...
395KB Sizes 0 Downloads 0 Views