Computer Methods and Programs in Biomedicine, 37 (1992) 259-263
259
© 1992 Elsevier Science Publishers B.V. All rights reserved 0169-2607/92/$05.00 COMMET 01278
The use of multimedia in patient care T. Kitanosono
1 y.
Kurashita 1, M. H o n d a 1, T. Hishida 1, H. Konishi 1, M. Mizuno and M. Anzai 3
1 Department of Radiology, School of Medicine, Showa University, Tokyo, Japan 2 Nice Partners cooperation and 3 Apple Computer Japan Inc., Japan
A personal computer based system was constructed to assess the use of various forms of information (multimedia) in patient record keeping. A patient's file with his records kept in a multimedia fashion was made by using the system. We describe the hardware and software construction of the system together with the results and the memory requirements of each type of media. Potential usage of the system in the future is discussed especially in connection with the Picture Archiving and Communications Systems (PACS). PACS; Multimedia
1. Introduction Development of PACS has been significant this past decade. The emphasis has been on management of radiological images. Recently, there have been many reports on adding extra functions to the system. They include coupling with Hospital Information Systems [1], image processing [2], coupling with radiotherapy software packages [3] etc. In such an approach we have began a trial of applying multimedia to patient care. The procedures of a physician involve the management of much information and images are just a part of them. During a physician's work he collects informations by making use of all his senses, by performing examinations and by obtaining various data from the laboratories and from the radiological department. Many of the data could
Correspondence: T. Kitanosono, Department of Radiology, School of Medicine, Showa University, 1-5-8- Hatanodai, Shinagawa-ku, Tokyo, Japan. Tel.: 03-3784-8573; Fax: 03-37848360.
by kept in an objective way (Table 1), but currently, in most institutions, they are kept in form of charts and film jackets. Loss of objectivity in the process of making the charts is inevitable. A third person could never know what the physician exactly heard, saw or felt during his encounter with the patient. With the progress in computers and its applications it has become possible to manage many forms of information such as sound, graphics images, video recordings and text on a computer. We have created a prototype system using a personal computer to assess the possible use of multimedia in patient record keeping and its possible use in the future.
2. Experimental set-up A Macintosh Ilci (Apple Computer) was connected with a Truvel TZ-3X film scanner, an Epson GT-6000 scanner and a Sony Magnetooptical (MO) disk unit NWP-559 via SCSI interface. Two monitors, one Apple 13-inch full colour
260 TABLE l Information produced and possible ways of record keeping from a physician's procedures Procedure Interview Inspection Auscultation Physical examinations
Type of information Vision, sound Vision Sound Vision, sound Sense of touch Alphanumerical Alphanumerical graphics
Laboratory examinations Physiological examinations Radiological images
PACS
monitor is connected through a full colour 32-bit video board (Raster Ops) and a 19-inch colour monitor with resolution of 1024 x 768 (Raster Ops) is connected through an 8-bit colour board. Macrecorder (Apple computers) was used to digitize the sound recordings. The Truvel scanner can digitize X-ray images at 4, 16 or 256 gray levels with a variable resolution ranging up to 700 dots per inch (dpi). The films are digitized by a 3600-element CCD placed over mercury neon light. Epson GT-6000 is a colour image scanner with a maximum resolution of 300 dpi and capable of scanning 16 700 000 colours or 256 levels of gray. T H e MO unit has 5-inch disks with 330 Megabytes each side. A diagram of the hardware H
I
[
Possible recording method Video, sound recording Video, photograph Sound recording Video, sound recording Not possible HIS
I
F
I
l
o
A
o
E
Fig. 1. Schema of the Hardware A: Macintosh llci, B: 13" Monitor, C: 21" Monitor, D: Truvel Scanner, E: Epson Scanner, F: MO Disk Unit, G: MacRecorder, H: Video Interface.
is shown in Fig. 1. A 64-year-old patient who was hospitalized for esophageal cancer was selected and his charts were scanned with the image scanner. His laboratory data (blood count, biochemistry, urinalysis and serological examinations) were input manually. All other examination results such as microbiology cultures and radiological reports, were taken in through the image scanner. Colour photographs of the patient were also scanned through the image scanner. The radiological images were taken in through the film scanner. The plain radiographs were scanned at a resolution of 80-90 dpi or 160-200 dpi, digital images such as CT and M R I were scanned at 60-70 dpi or 120-140 dpi. The higher resolution was used when magnification was thought necessary. The patient's heart and breath sounds were recorded into a cassette tape through a small microphone connected to a stethoscope. The recordings were subsequently digitized with the Macrecorder and taken in. Hypercard system (Apple Computer) was used to create the patient file. The software is essentially a database software consisting of 'cards'. Each card can contain a bitmap or text information and programmable 'buttons' that enable transfer from one card to another, starting up a separate software and realizing designated effects (in this case, playing of the recorded sounds). The scanned charts were copied on to the cards of the hypercard with a page of the chart corresponding to one card. The date of the chart was used as a mark for each card. The laboratory data were typed into a table created on a card. Four. days'
261
worth of data was kept on one card. By 'pressing' the appropriate button on a card, the card of desired laboratory data nearest to the data of the chart is searched and shown. The sound recordings are programmed to be replayed by pressing the appropriate button. The recordings are each linked to the card by date of the recording. We did not have the means to display grayscale or colour images on the hypercard so the im'hges were kept in PICT format and were displayed by opening an image processing software Photoshop (Adobe). The images are kept in an image folder which is accessible from Photoshop. From the hypercard, by pressing the button for the images, Photoshop is started up automatically and shows the list of images that are kept in the image folder. The images are each named by the modality and the date of the examination. Hypercard is automatically called back when Photoshop is closed. All of the patient's charts and plain radiographs, selected digital images (4 images per examination) and colour photographs were put into the system. The sound recordings were done twice weekly for 15 s a day. The cards and the image folder were stored in the MO disk while the hypercard and Photoshop are in the computer itself. In addition, a video recording of the patient was done and taken into the computer through the videoboard for experimental purposes but was not connected to the patient file created on the computer.
TABLE 2 Data requirements for each type of information Radiological images Plain radiographs Digital images (CT&MRI) Sound Colour photographs Text Total
63.8 MB 45.2 MB 18.6 MB 4.0 MB 1.1 MB 1.2 MB 70.1 MB
(91%)
(5.6%) (1.6%) (1.7%)
1 MRI examination were performed. Total number of films scanned were three 1 4 " x 18", 26 14"-4", 15 8 " x 6" films for plain radiographs and upper GI enema, 20 CT, MRI images. The total amount of data for the radiological images was 50.8 MB. Sound recording requires 11 kB of memory per second with 11 kHz of sampling frequency. We recorded the patients chest auscultation for 15 s a day twice weekly, and the total was 4 MB. Four colour photographs taken to record the field of radiotherapy were also taken in with a data volume of 1.1 MB. Images, especially the plain films, take up 89.1% of data volume. Video images could be taken in through the video board by sampling static images at a frequency of 15 frames/s and played on the monitor by showing the sampled frames one after another. Memory requirement for each frame was around 280 kB. The memory capacity of our computer enabled 19 frames or just over a second of recording. Replay was not smooth and it was considered impractical at our current state.
3. Results
The amount of data that were needed for taking in our patient's information were as follows (see Table 2). Each card of the hypercard will contain up to 32 kilobytes (kB). The patient of our presentation has been in our hospital for 90 days, with 121 cards of text information. On average, 10 kB per card was used amounting to approximately 1.2 MB. As for the images, 17 chest anteroposterior and 10 lateral radiographs, 2 abdominal radiographs, 2 upper gastrointestinal tract enema study, 4 CT (3 chest, 1 abdomen) and
4. Discussion
The types of data used in medicine are numerous. Currently, they are kept in forms of charts and film jackets, or in other words, text and hard copy images. Electronical management of the data has been attempted mainly in Hospital Information Systems (HIS) and PACS. They focus on the management of data that are conventionally utilized. HIS is for management of text data and
262
PACS for images. Recent progress in computer technology has enabled handling of sound, pictures and motion pictures on a computer in addition to text and images and such applications are referred to as multimedia or hypermedia. Multimedia has not, as yet, had a major impact on patient care as in other fields such as education, presentation and publishing, arts, etc., but as the technology is becoming more and more sophisticated, we feel that studies of its possible use in medicine and patient care are required. From our experiment, the amount of data for sound, text and colour photographs were approximately 12.8% of that of radiological images. The video recording was not practical in our experimental setting, but currently many hardware a n d / o r software based data compression techniques are being released from many manufacturers enabling significant decrease in the size of the memory requirement, and we speculate that the video recordings may not have to be of outstanding quality as is the case with radiological images allowing greater compression. Therefore, we expect that management of video recordings will be feasible by using such techniques. Management o f patient data in multimedia apparatus was thought to be possible at least on stand-alone personal computers. Our model may represent a terminal in a hospital-based information management system. The whole system could be just like HIS or PACS but with capability to acquire, archive and communicate various forms of data. As the amount of data generated by the extra information are relatively small compared with radiological images, the burden of the extra data will probably not be significant when technology for PACS is mature. However, we expect that the problem would be related to software and interface bases once the technology for PACS is mature for management of images only. Therefore we fell that concern should be paid to multimedia when standardization of PACS is being discussed. Furthermore, with the development of large capacity portable memory medium such as IC cards, studies are currently under way for Personal Health Data recording systems (PHDRS) [4]. In this system, a patient's health data is recorded into a portable
medium as is carried on the patient on his own responsibility. The patient will present the medium whenever he visits his physician and the physician will make use of the contained data, at the same time updating it. Personal computers with multimedia capabilities are becoming widespread among the general public. By applying multimedia to P H D R S and making it compatible with the commercial products, instructions on a patient's daily activities like nutrition, exercises, medical procedures like insulin injections and expected complications could be put into the P H D R S medium in forms of picture, sound or video recording or combination of them. In return, patients could record their status (for instance blood pressure) in daily life and present the results at the next meeting with their physician. The impact of multimedia on physicians' work will be significant. It will enable a given physician to follow a patient's status more objectively, and the possible discrepancies among physicians on a patient's physical findings could be diminished. Opinions of the product of our work by the physicians in our hospital have been quite encouraging. However, the current methods of acquiring sound and video recordings for the system are complex, and acquired data need to be processed so that it can be replayed in the required fashion. It cannot be expected that physicians will be willing to go through such complexities in their daily routine, and data acquisition will have to be simplified as much as possible for the system to be accepted in clinical use. Cost effectiveness of applying multimedia to patient care is difficult to estimate. However, the current trend of deploying PACS is to make use of commercial products whenever possible. As the computer industry is moving towards multimedia, much of the hardware being used for PACS will have capabilities for handling multimedia. By paying attention to multimedia while developing PACS, especially in standardization, the cost of applying multimedia could be minimized. Also, as in our model, studies for multimedia could be done on a personal computer basis and could be carried out without too much expenditure.
263
5. Conclusion
References
A p e r s o n a l c o m p u t e r - b a s e d m u l t i m e d i a system for p a t i e n t c a r e was c o n s t r u c t e d a n d its initial trial was d o n e by r e c o r d i n g a p a t i e n t ' s d a t a . A l t h o u g h m a n y issues n e e d f u r t h e r e v a l u a t i o n , possibilities for a new way o f p a t i e n t c a r e w e r e shown. A m o n g m u c h m e d i c a l i n f o r m a t i o n , images p r e s e n t t h e most f o r m i d a b l e obstacle. Currently, o n e of the m a i n p r o b l e m s b e i n g discussed c o n c e r n i n g P A C S is s t a n d a r d i z a t i o n . W e w o u l d like to see o t h e r forms of i n f o r m a t i o n t a k e n into c o n s i d e r a t i o n , not just t h e images, w h e n stand a r d i z a t i o n is b e i n g discussed.
[1] H. Lodder, B.M. Van Poppel, H. Wilmink et al., HISPACS coupling: BAZIS/ZIS and Philips/MARCOM on speaking terms. Med. Inf. 13 (4) (1988) 361-367. [2] T. Arimoto, Innervision 5.8 (1990) 39-41. [3] K. Kicuchi, M. Kowada, H, Ogayama et al., Automated image processing by PACS as a simple tool for designing craniotomy; synthesis of sagittal reconstructed CT images and cerebral angiograms. No Shinkei geka 18 (4) (1990) 355-360. [4] M. Kaneko, S. Ikeda, N. Ohyama, Research on personal health data recording system (PHDRS). Proceedings of IMAG '89, IEEE Computer Society Press, Los Alamitos, pp. 68-71 (1989).
259
© 1992 Elsevier Science Publishers B.V. All rights reserved 0169-2607/92/$05.00 COMMET 01278
The use of multimedia in patient care T. Kitanosono
1 y.
Kurashita 1, M. H o n d a 1, T. Hishida 1, H. Konishi 1, M. Mizuno and M. Anzai 3
1 Department of Radiology, School of Medicine, Showa University, Tokyo, Japan 2 Nice Partners cooperation and 3 Apple Computer Japan Inc., Japan
A personal computer based system was constructed to assess the use of various forms of information (multimedia) in patient record keeping. A patient's file with his records kept in a multimedia fashion was made by using the system. We describe the hardware and software construction of the system together with the results and the memory requirements of each type of media. Potential usage of the system in the future is discussed especially in connection with the Picture Archiving and Communications Systems (PACS). PACS; Multimedia
1. Introduction Development of PACS has been significant this past decade. The emphasis has been on management of radiological images. Recently, there have been many reports on adding extra functions to the system. They include coupling with Hospital Information Systems [1], image processing [2], coupling with radiotherapy software packages [3] etc. In such an approach we have began a trial of applying multimedia to patient care. The procedures of a physician involve the management of much information and images are just a part of them. During a physician's work he collects informations by making use of all his senses, by performing examinations and by obtaining various data from the laboratories and from the radiological department. Many of the data could
Correspondence: T. Kitanosono, Department of Radiology, School of Medicine, Showa University, 1-5-8- Hatanodai, Shinagawa-ku, Tokyo, Japan. Tel.: 03-3784-8573; Fax: 03-37848360.
by kept in an objective way (Table 1), but currently, in most institutions, they are kept in form of charts and film jackets. Loss of objectivity in the process of making the charts is inevitable. A third person could never know what the physician exactly heard, saw or felt during his encounter with the patient. With the progress in computers and its applications it has become possible to manage many forms of information such as sound, graphics images, video recordings and text on a computer. We have created a prototype system using a personal computer to assess the possible use of multimedia in patient record keeping and its possible use in the future.
2. Experimental set-up A Macintosh Ilci (Apple Computer) was connected with a Truvel TZ-3X film scanner, an Epson GT-6000 scanner and a Sony Magnetooptical (MO) disk unit NWP-559 via SCSI interface. Two monitors, one Apple 13-inch full colour
260 TABLE l Information produced and possible ways of record keeping from a physician's procedures Procedure Interview Inspection Auscultation Physical examinations
Type of information Vision, sound Vision Sound Vision, sound Sense of touch Alphanumerical Alphanumerical graphics
Laboratory examinations Physiological examinations Radiological images
PACS
monitor is connected through a full colour 32-bit video board (Raster Ops) and a 19-inch colour monitor with resolution of 1024 x 768 (Raster Ops) is connected through an 8-bit colour board. Macrecorder (Apple computers) was used to digitize the sound recordings. The Truvel scanner can digitize X-ray images at 4, 16 or 256 gray levels with a variable resolution ranging up to 700 dots per inch (dpi). The films are digitized by a 3600-element CCD placed over mercury neon light. Epson GT-6000 is a colour image scanner with a maximum resolution of 300 dpi and capable of scanning 16 700 000 colours or 256 levels of gray. T H e MO unit has 5-inch disks with 330 Megabytes each side. A diagram of the hardware H
I
[
Possible recording method Video, sound recording Video, photograph Sound recording Video, sound recording Not possible HIS
I
F
I
l
o
A
o
E
Fig. 1. Schema of the Hardware A: Macintosh llci, B: 13" Monitor, C: 21" Monitor, D: Truvel Scanner, E: Epson Scanner, F: MO Disk Unit, G: MacRecorder, H: Video Interface.
is shown in Fig. 1. A 64-year-old patient who was hospitalized for esophageal cancer was selected and his charts were scanned with the image scanner. His laboratory data (blood count, biochemistry, urinalysis and serological examinations) were input manually. All other examination results such as microbiology cultures and radiological reports, were taken in through the image scanner. Colour photographs of the patient were also scanned through the image scanner. The radiological images were taken in through the film scanner. The plain radiographs were scanned at a resolution of 80-90 dpi or 160-200 dpi, digital images such as CT and M R I were scanned at 60-70 dpi or 120-140 dpi. The higher resolution was used when magnification was thought necessary. The patient's heart and breath sounds were recorded into a cassette tape through a small microphone connected to a stethoscope. The recordings were subsequently digitized with the Macrecorder and taken in. Hypercard system (Apple Computer) was used to create the patient file. The software is essentially a database software consisting of 'cards'. Each card can contain a bitmap or text information and programmable 'buttons' that enable transfer from one card to another, starting up a separate software and realizing designated effects (in this case, playing of the recorded sounds). The scanned charts were copied on to the cards of the hypercard with a page of the chart corresponding to one card. The date of the chart was used as a mark for each card. The laboratory data were typed into a table created on a card. Four. days'
261
worth of data was kept on one card. By 'pressing' the appropriate button on a card, the card of desired laboratory data nearest to the data of the chart is searched and shown. The sound recordings are programmed to be replayed by pressing the appropriate button. The recordings are each linked to the card by date of the recording. We did not have the means to display grayscale or colour images on the hypercard so the im'hges were kept in PICT format and were displayed by opening an image processing software Photoshop (Adobe). The images are kept in an image folder which is accessible from Photoshop. From the hypercard, by pressing the button for the images, Photoshop is started up automatically and shows the list of images that are kept in the image folder. The images are each named by the modality and the date of the examination. Hypercard is automatically called back when Photoshop is closed. All of the patient's charts and plain radiographs, selected digital images (4 images per examination) and colour photographs were put into the system. The sound recordings were done twice weekly for 15 s a day. The cards and the image folder were stored in the MO disk while the hypercard and Photoshop are in the computer itself. In addition, a video recording of the patient was done and taken into the computer through the videoboard for experimental purposes but was not connected to the patient file created on the computer.
TABLE 2 Data requirements for each type of information Radiological images Plain radiographs Digital images (CT&MRI) Sound Colour photographs Text Total
63.8 MB 45.2 MB 18.6 MB 4.0 MB 1.1 MB 1.2 MB 70.1 MB
(91%)
(5.6%) (1.6%) (1.7%)
1 MRI examination were performed. Total number of films scanned were three 1 4 " x 18", 26 14"-4", 15 8 " x 6" films for plain radiographs and upper GI enema, 20 CT, MRI images. The total amount of data for the radiological images was 50.8 MB. Sound recording requires 11 kB of memory per second with 11 kHz of sampling frequency. We recorded the patients chest auscultation for 15 s a day twice weekly, and the total was 4 MB. Four colour photographs taken to record the field of radiotherapy were also taken in with a data volume of 1.1 MB. Images, especially the plain films, take up 89.1% of data volume. Video images could be taken in through the video board by sampling static images at a frequency of 15 frames/s and played on the monitor by showing the sampled frames one after another. Memory requirement for each frame was around 280 kB. The memory capacity of our computer enabled 19 frames or just over a second of recording. Replay was not smooth and it was considered impractical at our current state.
3. Results
The amount of data that were needed for taking in our patient's information were as follows (see Table 2). Each card of the hypercard will contain up to 32 kilobytes (kB). The patient of our presentation has been in our hospital for 90 days, with 121 cards of text information. On average, 10 kB per card was used amounting to approximately 1.2 MB. As for the images, 17 chest anteroposterior and 10 lateral radiographs, 2 abdominal radiographs, 2 upper gastrointestinal tract enema study, 4 CT (3 chest, 1 abdomen) and
4. Discussion
The types of data used in medicine are numerous. Currently, they are kept in forms of charts and film jackets, or in other words, text and hard copy images. Electronical management of the data has been attempted mainly in Hospital Information Systems (HIS) and PACS. They focus on the management of data that are conventionally utilized. HIS is for management of text data and
262
PACS for images. Recent progress in computer technology has enabled handling of sound, pictures and motion pictures on a computer in addition to text and images and such applications are referred to as multimedia or hypermedia. Multimedia has not, as yet, had a major impact on patient care as in other fields such as education, presentation and publishing, arts, etc., but as the technology is becoming more and more sophisticated, we feel that studies of its possible use in medicine and patient care are required. From our experiment, the amount of data for sound, text and colour photographs were approximately 12.8% of that of radiological images. The video recording was not practical in our experimental setting, but currently many hardware a n d / o r software based data compression techniques are being released from many manufacturers enabling significant decrease in the size of the memory requirement, and we speculate that the video recordings may not have to be of outstanding quality as is the case with radiological images allowing greater compression. Therefore, we expect that management of video recordings will be feasible by using such techniques. Management o f patient data in multimedia apparatus was thought to be possible at least on stand-alone personal computers. Our model may represent a terminal in a hospital-based information management system. The whole system could be just like HIS or PACS but with capability to acquire, archive and communicate various forms of data. As the amount of data generated by the extra information are relatively small compared with radiological images, the burden of the extra data will probably not be significant when technology for PACS is mature. However, we expect that the problem would be related to software and interface bases once the technology for PACS is mature for management of images only. Therefore we fell that concern should be paid to multimedia when standardization of PACS is being discussed. Furthermore, with the development of large capacity portable memory medium such as IC cards, studies are currently under way for Personal Health Data recording systems (PHDRS) [4]. In this system, a patient's health data is recorded into a portable
medium as is carried on the patient on his own responsibility. The patient will present the medium whenever he visits his physician and the physician will make use of the contained data, at the same time updating it. Personal computers with multimedia capabilities are becoming widespread among the general public. By applying multimedia to P H D R S and making it compatible with the commercial products, instructions on a patient's daily activities like nutrition, exercises, medical procedures like insulin injections and expected complications could be put into the P H D R S medium in forms of picture, sound or video recording or combination of them. In return, patients could record their status (for instance blood pressure) in daily life and present the results at the next meeting with their physician. The impact of multimedia on physicians' work will be significant. It will enable a given physician to follow a patient's status more objectively, and the possible discrepancies among physicians on a patient's physical findings could be diminished. Opinions of the product of our work by the physicians in our hospital have been quite encouraging. However, the current methods of acquiring sound and video recordings for the system are complex, and acquired data need to be processed so that it can be replayed in the required fashion. It cannot be expected that physicians will be willing to go through such complexities in their daily routine, and data acquisition will have to be simplified as much as possible for the system to be accepted in clinical use. Cost effectiveness of applying multimedia to patient care is difficult to estimate. However, the current trend of deploying PACS is to make use of commercial products whenever possible. As the computer industry is moving towards multimedia, much of the hardware being used for PACS will have capabilities for handling multimedia. By paying attention to multimedia while developing PACS, especially in standardization, the cost of applying multimedia could be minimized. Also, as in our model, studies for multimedia could be done on a personal computer basis and could be carried out without too much expenditure.
263
5. Conclusion
References
A p e r s o n a l c o m p u t e r - b a s e d m u l t i m e d i a system for p a t i e n t c a r e was c o n s t r u c t e d a n d its initial trial was d o n e by r e c o r d i n g a p a t i e n t ' s d a t a . A l t h o u g h m a n y issues n e e d f u r t h e r e v a l u a t i o n , possibilities for a new way o f p a t i e n t c a r e w e r e shown. A m o n g m u c h m e d i c a l i n f o r m a t i o n , images p r e s e n t t h e most f o r m i d a b l e obstacle. Currently, o n e of the m a i n p r o b l e m s b e i n g discussed c o n c e r n i n g P A C S is s t a n d a r d i z a t i o n . W e w o u l d like to see o t h e r forms of i n f o r m a t i o n t a k e n into c o n s i d e r a t i o n , not just t h e images, w h e n stand a r d i z a t i o n is b e i n g discussed.
[1] H. Lodder, B.M. Van Poppel, H. Wilmink et al., HISPACS coupling: BAZIS/ZIS and Philips/MARCOM on speaking terms. Med. Inf. 13 (4) (1988) 361-367. [2] T. Arimoto, Innervision 5.8 (1990) 39-41. [3] K. Kicuchi, M. Kowada, H, Ogayama et al., Automated image processing by PACS as a simple tool for designing craniotomy; synthesis of sagittal reconstructed CT images and cerebral angiograms. No Shinkei geka 18 (4) (1990) 355-360. [4] M. Kaneko, S. Ikeda, N. Ohyama, Research on personal health data recording system (PHDRS). Proceedings of IMAG '89, IEEE Computer Society Press, Los Alamitos, pp. 68-71 (1989).
