1979, British Journal of Radiology, 52, 478-481
A computerized three-dimensional treatment planning system utilizing interactive colour graphics By D. L McShan, Ph.D., A. Silverman,* Dorina M. Lanza, M.S.,f L. E. Reinstein, Ph.D. and A. S. Glicksman, M.D. Department of Radiation Oncology, Rhode Island Hospital, and The Section on Radiation Medicine, Division of Biology and Medicine, Brown University, Providence, Rhode Island 02902, USA {Received August, 1978 and in revised form January, 1979) ABSTRACT
A new computerized radiation treatment planning system has been developed to aid in three-dimensional treatment planning. Using interactive colour graphics in conjunction with a PDP 11 /45 computer, the system can take multiple transverse contours and construct a perspective display of the treatment region showing organ surfaces as well as crosssectional contours. With interactively selected orientations, the display allows easy perception of the relative positioning of the treatment volume and neighbouring anatomy. For external beam treatment planning, interactive computer simulation is used to select diaphragm sizes which best conform to the target area while avoiding sensitive structures. Dose calculations for the selected beams are carried out on multiple transverse planes. The calculational planes and surfaces are displayed in perspective with radiation dosage displayed in an interactively manipulated colour display. Altogether the system provides an easy assessment of the volume to be irradiated, interactive selection of optimal arrangements of treatment fields and a means for visualizing and evaluating the resulting dose distributions.
Following earlier work in the Department of Radiation Oncology at Rhode Island Hospital in development of a three-dimensional radiation treatment planning system, a new interactive computerized planning system has been developed. The new system uses colour graphics to provide better insight into the planning of radiation treatments which will best conform the radiation distribution to the target volume while minimizing radiation elsewhere, especially to sensitive organs. The new system is designed to help the planner visualize relative anatomy within the proposed treatment volume, simulate proposed treatment fields relative to this anatomy, and make qualitive and quantitative assessment of dose throughout the treatment volume. The earlier work at the Rhode Island Hospital approached the problem of three-dimensional planning of external beam treatments by using reconstructed multi-plane images taken from transverse axial tomographs (Reinstein et al., 1978). In that approach, contours at multiple levels could be viewed in three-dimensional perspective and rotated. The main advantage of this method was that contours could be placed in orientations which would simulate *Present address: Stanford University, Stanford, Ca. f Present address: Draper Laboratories, Cambridge, Ma.
their position relative to an incoming beam. Using this method, gantry angles, collimator angles, beam width, and height and blocking could be determined. A disadvantage was that the system was not entirely interactive and required a number of time-consuming trial and error tests. Further, the resulting dose distribution could only be represented by using conventional two-dimensional isodose displays. The system reported here removes many of these disadvantages. Using colour video image hardware and computer generated graphics, transverse contours and reconstructed surfaces are displayed in three-dimensional perspective with colour used to differentiate anatomical features. Interactive graphics have been developed to enable the treatment planner to simulate beams for external treatment planning. Further, the radiation dosage can be calculated for multiple planes and surfaces and can be displayed using a colour representation of the dose distribution. METHODS AND EQUIPMENT
The new display software is implemented on a Digital Equipment PDP 11 /45 mini-computer with floating point hardware and 80k words of core memory. The computer operates with a multipartitioning time-sharing operating system that supports up to twelve terminals. The colour image display hardware is made by DeAnza Systems, Inc. and was purchased with interfacing to the PDP 11 system. The graphics software uses an interactive console which contains four push buttons, four potentiometers and a "joy-stick" with two additional potentiometers. The graphics software is written to use the interactive console exclusively with no terminal input. Cues for the colour-coded buttons and potentiometers are displayed in annotation fields so that the buttons and knobs are easily programmed for different functions. A schematic for the computer system is shown in Fig. 1. The program is written primarily in Fortran and currently consists of nine overlays each running in
478
JUNE 1979
A computerized three-dimensional treatment planning system utilizing interactive colour graphics
III
/cjps,/
PDP 1 1 / 4 5 80K WORDS FLOATING 116
MB
POINT
u
DISC
N I r
COLOR MONITOR
PT
B U S
TERMINAL IMAGE REFRESH MEMORY
I
MOMENTARY PUSH-BUTTONS
r
• • • • + "JOY" STICK
£3
o
o
0
0 POTENTIOMETERS
i
,
,
k
,
ADC
DIGITAL
—
I/O INTERACTIVE CONSOLE F I G . 1. Schematic of computer and graphics system.
for the surface reconstruction. The contours in Fig. 2 are distinguished by colour with the external contour in blue, the lungs in white, the spinal cord in green, and the target (in this case the oesophagus) in yellow. The front surfaces of the spinal cord and target are shown as well as the back surface of the right lung. This figure graphically illustrates the variation of the relative anatomy throughout the volume to be irradiated. For the perspective displays the orientations and magnifications are selected interactively through the use of the console potentiometers. The image can be expanded along any of the original axes (used in Fig. 2 to increase separation between planes). Contour line elements are removed when hidden by any Anatomical display Figure 2 shows a perspective reconstruction of of the planes (rectangular limits used). The surfaces are constructed using the outline three of five transverse planes taken in the region between the thoracic inlet and the mid-line thorax data on all of the transverse levels taken. The reconregion. The planes were taken at a relative spacing struction begins by defining a set of mesh points of + 7 , + 3 , 0, - 4 , and - 7 cm. The planes at + 3 which are uniformly spaced around each outline. and —4 cm are not shown in Fig. 2 but were used Spacing for the points is determined by calculating
less than 16k of core. The software is interfaced with the conventional treatment planning programs and can be entered from the external beam planning program. As described in the earlier paper, a separate program in the conventional planning system is used to enter contour data with a sonic-digitizer from transverse axial tomography films. Individual contours are labelled by organ type and the relative positioning of the transverse level. For a typical case, about five levels are required throughout the proposed treatment area. Once entered, these contours are available for three-dimensional reconstruction.
479
VOL.
52, No. 618 D. L. Me Shan, A. Silverman, Dorina M. Lanza, L. E. Reinstein and A. S. Glicksman
the total chord or path length and dividing this length by the number of desired segments (30 for this example). The points are aligned on each level by first calculating a mean or average centre. A line is then drawn through the centre parallel to one of the co-ordinate axes. The most positive intercept of this line with the outline is taken as a starting point for generating the mesh. For interpolation between transverse levels, the generated mesh points are connected to their aligned counterparts with a smooth curve, constructed by using spline curve-drawing techniques (Ahlberg, 1974). The technique takes the initial set of co-ordinate points and derives a larger set of mesh point values separately for each of the three dimensions. The interpolated values are constrained to a curve which passes through the initial set of values and which exhibits no discontinuities in its slope. For Fig. 2, a set of 20 mesh points has been generated for each set of points around the outlines. The result of these two interpolation operations is a
grid of three-dimensional co-ordinates which map out the surface. All of the contour surface maps are generated and stored prior to entry into the three-dimensional display system. Within the display system, individual surfaces may be recalled and displayed showing either their front or back (away from viewer) surfaces. Radiation beam selection
After a look at the relative anatomy within the proposed treatment region, treatment planning for external beam therapy may proceed by selecting one of two beam simulation modes. The first mode is similar to conventional planning systems which allow interactive selection of beam parameters. It differs in that beam paths along the transverse planes can be displayed simultaneously and manipulated on the multi-plane perspective images. The second mode of simulation is particularly suited for aligning
Figure 2: Multiple cross-sectional contours and organ surfaces in threedimensional perspective.
Figure 3: Simulation of external beams. The red rectangle shows the beam area enclosing the target but avoiding the spinal cord.
Figure 4: Simulation of beam with unnacceptable overlap of target and spinal cord. At this orientation part of the spinal cord is inevitably included in the field.
Figure 5:
480
Coloured dose distribution on perspective multiplanar display.
JUNE 1979
A computerized three-dimensional treatment planning system utilizing interactive colour graphics integration of differential scatter-air ratios which are generated using a table of scatter-air ratios. The dose is calculated at a matrix of points across the plane and the data are stored in a magnetic disc file for later display and evaluation. Instead of displaying conventional isodose lines, the relative dose is evaluated and loaded into each display pixel that projects onto a calculational plane. This process takes only a few seconds. Using intensity transformation hardware on the display system any colour can be assigned to any dose level. For evaluating the dose, the potentiometers on the console are used to set and vary the lower limit and width of one or more bands of uniform colour. These bands reflect in radiated areas which lie between two dose limits. Because the colour-to-dose assignment is easily made, the limit adjustments are highly interactive and simultaneous for the entire display area. Used in combination with the perspective display, the user gains an immediate perception of the dose distribution throughout the irradiated region. Figure 5 shows a perspective display of the planes with dose calculations added. A background of red whose intensity is proportional to the dose shows the radiation paths. Three bands of colour have been selected for this display. Each band shows a uniformity of dose across the target area for each of the three levels. However, the percentage difference Three-dimensional isodose display To evaluate the proposed treatment plan, it is, of between bands shows a problem of uniformity of course, necessary to calculate and display the result- dose along the length of the tumour (corrected in ing dose distribution at points within the target as practice by a wedge compensator). well as at neighbouring locations. For threeWhile the examples shown here relate to the dimensional planning, this implies calculations thorax region, the same techniques of threethroughout the irradiated volume. While the new dimensional display can be applied to other sites in system is capable of displaying dose distributions on the body. Particularly in the pelvis, the uterus, nonparallel planes and surfaces, at present the dose bladder and rectum can be displayed relative to the calculations for external beam therapy are limited to external contour and bony pelvis. Isodose calculaplanes which are coplanar or parallel to the central tions for intracavitary sources can be shown in the axes of the beam combination. Normally, calcula- same manner as for an external beam and can be tions are carried out on the planes used for input of combined with external beam calculations to show contour. The calculations are currently the slowest the composite doses in relation to the pelvic anatomy. part of the three-dimensional planning system and The use of this system for the pelvic region will be require about two minutes per plane. The calcula- published shortly. tional program used is a version of J. R. Cunningham's CBEAM program (Gupta and Cunningham, REFERENCES 1966). The calculation divides the dose into two AHLBERG, J. H., 1974. A picture-view of splines. Technical Report Brown University, Providence, Rhode Island. components-primary and scattered radiation. The GUPTA, S. K., and CUNNINGHAM, J. R., 1966. Measurement of tissue—air ratios and scatter functions for large field primary contribution is obtained by using tissue-air sizes for cobalt 60 gamma radiation. British Journal of ratios for zero area and by making appropriate Radiology, 39, 7-11. geometrical corrections to account for inverse- REINSTEIN L. R. MCSHAN D. L. WEBBER B. M., and GLICKSMAN A. S. 1978. A computer-assisted threesquare fall-off, compensating filters, and penumbra dimensional treatment planning system. Radiology, 127, effects. The scatter component is calculated by 259-264. individual beams. A typical display for this mode is shown in Fig. 3. In this figure, the contours for a single transverse plane are shown in the upper left hand corner along with the relative beam positioning. In the centre of the display is shown the simulation of a field area (shown in red) whose central axis is parallel to the transverse planes. The projection of the spinal cord (in green) and the target (in yellow) are shown relative to the incoming beam, the incident direction of which can be seen by the positioning of the green beam shown in the upper left hand corner. This direction may be interactively rotated by adjusting a potentiometer. The treatment field shown in Fig. 3 is one of two proposed lateral exposures which are planned to avoid irradiation of the spinal cord while minimizing exposure of the lung. At a fully lateral angle, the lung would receive too much dose, therefore the beam is moved anteriorly. If the beam is moved too far forward, the target area and the spinal cord overlap. A simulation of an unacceptable overlap can be seen in Fig. 4. Figure 3, on the other hand, shows adequate separation. Besides altering the incident direction of the beam, the width, height, tilt (collimator angle) and centre may be interactively adjusted relative to the anatomy so as best to conform the radiation area to the target area.
481
A computerized three-dimensional treatment planning system utilizing interactive colour graphics By D. L McShan, Ph.D., A. Silverman,* Dorina M. Lanza, M.S.,f L. E. Reinstein, Ph.D. and A. S. Glicksman, M.D. Department of Radiation Oncology, Rhode Island Hospital, and The Section on Radiation Medicine, Division of Biology and Medicine, Brown University, Providence, Rhode Island 02902, USA {Received August, 1978 and in revised form January, 1979) ABSTRACT
A new computerized radiation treatment planning system has been developed to aid in three-dimensional treatment planning. Using interactive colour graphics in conjunction with a PDP 11 /45 computer, the system can take multiple transverse contours and construct a perspective display of the treatment region showing organ surfaces as well as crosssectional contours. With interactively selected orientations, the display allows easy perception of the relative positioning of the treatment volume and neighbouring anatomy. For external beam treatment planning, interactive computer simulation is used to select diaphragm sizes which best conform to the target area while avoiding sensitive structures. Dose calculations for the selected beams are carried out on multiple transverse planes. The calculational planes and surfaces are displayed in perspective with radiation dosage displayed in an interactively manipulated colour display. Altogether the system provides an easy assessment of the volume to be irradiated, interactive selection of optimal arrangements of treatment fields and a means for visualizing and evaluating the resulting dose distributions.
Following earlier work in the Department of Radiation Oncology at Rhode Island Hospital in development of a three-dimensional radiation treatment planning system, a new interactive computerized planning system has been developed. The new system uses colour graphics to provide better insight into the planning of radiation treatments which will best conform the radiation distribution to the target volume while minimizing radiation elsewhere, especially to sensitive organs. The new system is designed to help the planner visualize relative anatomy within the proposed treatment volume, simulate proposed treatment fields relative to this anatomy, and make qualitive and quantitative assessment of dose throughout the treatment volume. The earlier work at the Rhode Island Hospital approached the problem of three-dimensional planning of external beam treatments by using reconstructed multi-plane images taken from transverse axial tomographs (Reinstein et al., 1978). In that approach, contours at multiple levels could be viewed in three-dimensional perspective and rotated. The main advantage of this method was that contours could be placed in orientations which would simulate *Present address: Stanford University, Stanford, Ca. f Present address: Draper Laboratories, Cambridge, Ma.
their position relative to an incoming beam. Using this method, gantry angles, collimator angles, beam width, and height and blocking could be determined. A disadvantage was that the system was not entirely interactive and required a number of time-consuming trial and error tests. Further, the resulting dose distribution could only be represented by using conventional two-dimensional isodose displays. The system reported here removes many of these disadvantages. Using colour video image hardware and computer generated graphics, transverse contours and reconstructed surfaces are displayed in three-dimensional perspective with colour used to differentiate anatomical features. Interactive graphics have been developed to enable the treatment planner to simulate beams for external treatment planning. Further, the radiation dosage can be calculated for multiple planes and surfaces and can be displayed using a colour representation of the dose distribution. METHODS AND EQUIPMENT
The new display software is implemented on a Digital Equipment PDP 11 /45 mini-computer with floating point hardware and 80k words of core memory. The computer operates with a multipartitioning time-sharing operating system that supports up to twelve terminals. The colour image display hardware is made by DeAnza Systems, Inc. and was purchased with interfacing to the PDP 11 system. The graphics software uses an interactive console which contains four push buttons, four potentiometers and a "joy-stick" with two additional potentiometers. The graphics software is written to use the interactive console exclusively with no terminal input. Cues for the colour-coded buttons and potentiometers are displayed in annotation fields so that the buttons and knobs are easily programmed for different functions. A schematic for the computer system is shown in Fig. 1. The program is written primarily in Fortran and currently consists of nine overlays each running in
478
JUNE 1979
A computerized three-dimensional treatment planning system utilizing interactive colour graphics
III
/cjps,/
PDP 1 1 / 4 5 80K WORDS FLOATING 116
MB
POINT
u
DISC
N I r
COLOR MONITOR
PT
B U S
TERMINAL IMAGE REFRESH MEMORY
I
MOMENTARY PUSH-BUTTONS
r
• • • • + "JOY" STICK
£3
o
o
0
0 POTENTIOMETERS
i
,
,
k
,
ADC
DIGITAL
—
I/O INTERACTIVE CONSOLE F I G . 1. Schematic of computer and graphics system.
for the surface reconstruction. The contours in Fig. 2 are distinguished by colour with the external contour in blue, the lungs in white, the spinal cord in green, and the target (in this case the oesophagus) in yellow. The front surfaces of the spinal cord and target are shown as well as the back surface of the right lung. This figure graphically illustrates the variation of the relative anatomy throughout the volume to be irradiated. For the perspective displays the orientations and magnifications are selected interactively through the use of the console potentiometers. The image can be expanded along any of the original axes (used in Fig. 2 to increase separation between planes). Contour line elements are removed when hidden by any Anatomical display Figure 2 shows a perspective reconstruction of of the planes (rectangular limits used). The surfaces are constructed using the outline three of five transverse planes taken in the region between the thoracic inlet and the mid-line thorax data on all of the transverse levels taken. The reconregion. The planes were taken at a relative spacing struction begins by defining a set of mesh points of + 7 , + 3 , 0, - 4 , and - 7 cm. The planes at + 3 which are uniformly spaced around each outline. and —4 cm are not shown in Fig. 2 but were used Spacing for the points is determined by calculating
less than 16k of core. The software is interfaced with the conventional treatment planning programs and can be entered from the external beam planning program. As described in the earlier paper, a separate program in the conventional planning system is used to enter contour data with a sonic-digitizer from transverse axial tomography films. Individual contours are labelled by organ type and the relative positioning of the transverse level. For a typical case, about five levels are required throughout the proposed treatment area. Once entered, these contours are available for three-dimensional reconstruction.
479
VOL.
52, No. 618 D. L. Me Shan, A. Silverman, Dorina M. Lanza, L. E. Reinstein and A. S. Glicksman
the total chord or path length and dividing this length by the number of desired segments (30 for this example). The points are aligned on each level by first calculating a mean or average centre. A line is then drawn through the centre parallel to one of the co-ordinate axes. The most positive intercept of this line with the outline is taken as a starting point for generating the mesh. For interpolation between transverse levels, the generated mesh points are connected to their aligned counterparts with a smooth curve, constructed by using spline curve-drawing techniques (Ahlberg, 1974). The technique takes the initial set of co-ordinate points and derives a larger set of mesh point values separately for each of the three dimensions. The interpolated values are constrained to a curve which passes through the initial set of values and which exhibits no discontinuities in its slope. For Fig. 2, a set of 20 mesh points has been generated for each set of points around the outlines. The result of these two interpolation operations is a
grid of three-dimensional co-ordinates which map out the surface. All of the contour surface maps are generated and stored prior to entry into the three-dimensional display system. Within the display system, individual surfaces may be recalled and displayed showing either their front or back (away from viewer) surfaces. Radiation beam selection
After a look at the relative anatomy within the proposed treatment region, treatment planning for external beam therapy may proceed by selecting one of two beam simulation modes. The first mode is similar to conventional planning systems which allow interactive selection of beam parameters. It differs in that beam paths along the transverse planes can be displayed simultaneously and manipulated on the multi-plane perspective images. The second mode of simulation is particularly suited for aligning
Figure 2: Multiple cross-sectional contours and organ surfaces in threedimensional perspective.
Figure 3: Simulation of external beams. The red rectangle shows the beam area enclosing the target but avoiding the spinal cord.
Figure 4: Simulation of beam with unnacceptable overlap of target and spinal cord. At this orientation part of the spinal cord is inevitably included in the field.
Figure 5:
480
Coloured dose distribution on perspective multiplanar display.
JUNE 1979
A computerized three-dimensional treatment planning system utilizing interactive colour graphics integration of differential scatter-air ratios which are generated using a table of scatter-air ratios. The dose is calculated at a matrix of points across the plane and the data are stored in a magnetic disc file for later display and evaluation. Instead of displaying conventional isodose lines, the relative dose is evaluated and loaded into each display pixel that projects onto a calculational plane. This process takes only a few seconds. Using intensity transformation hardware on the display system any colour can be assigned to any dose level. For evaluating the dose, the potentiometers on the console are used to set and vary the lower limit and width of one or more bands of uniform colour. These bands reflect in radiated areas which lie between two dose limits. Because the colour-to-dose assignment is easily made, the limit adjustments are highly interactive and simultaneous for the entire display area. Used in combination with the perspective display, the user gains an immediate perception of the dose distribution throughout the irradiated region. Figure 5 shows a perspective display of the planes with dose calculations added. A background of red whose intensity is proportional to the dose shows the radiation paths. Three bands of colour have been selected for this display. Each band shows a uniformity of dose across the target area for each of the three levels. However, the percentage difference Three-dimensional isodose display To evaluate the proposed treatment plan, it is, of between bands shows a problem of uniformity of course, necessary to calculate and display the result- dose along the length of the tumour (corrected in ing dose distribution at points within the target as practice by a wedge compensator). well as at neighbouring locations. For threeWhile the examples shown here relate to the dimensional planning, this implies calculations thorax region, the same techniques of threethroughout the irradiated volume. While the new dimensional display can be applied to other sites in system is capable of displaying dose distributions on the body. Particularly in the pelvis, the uterus, nonparallel planes and surfaces, at present the dose bladder and rectum can be displayed relative to the calculations for external beam therapy are limited to external contour and bony pelvis. Isodose calculaplanes which are coplanar or parallel to the central tions for intracavitary sources can be shown in the axes of the beam combination. Normally, calcula- same manner as for an external beam and can be tions are carried out on the planes used for input of combined with external beam calculations to show contour. The calculations are currently the slowest the composite doses in relation to the pelvic anatomy. part of the three-dimensional planning system and The use of this system for the pelvic region will be require about two minutes per plane. The calcula- published shortly. tional program used is a version of J. R. Cunningham's CBEAM program (Gupta and Cunningham, REFERENCES 1966). The calculation divides the dose into two AHLBERG, J. H., 1974. A picture-view of splines. Technical Report Brown University, Providence, Rhode Island. components-primary and scattered radiation. The GUPTA, S. K., and CUNNINGHAM, J. R., 1966. Measurement of tissue—air ratios and scatter functions for large field primary contribution is obtained by using tissue-air sizes for cobalt 60 gamma radiation. British Journal of ratios for zero area and by making appropriate Radiology, 39, 7-11. geometrical corrections to account for inverse- REINSTEIN L. R. MCSHAN D. L. WEBBER B. M., and GLICKSMAN A. S. 1978. A computer-assisted threesquare fall-off, compensating filters, and penumbra dimensional treatment planning system. Radiology, 127, effects. The scatter component is calculated by 259-264. individual beams. A typical display for this mode is shown in Fig. 3. In this figure, the contours for a single transverse plane are shown in the upper left hand corner along with the relative beam positioning. In the centre of the display is shown the simulation of a field area (shown in red) whose central axis is parallel to the transverse planes. The projection of the spinal cord (in green) and the target (in yellow) are shown relative to the incoming beam, the incident direction of which can be seen by the positioning of the green beam shown in the upper left hand corner. This direction may be interactively rotated by adjusting a potentiometer. The treatment field shown in Fig. 3 is one of two proposed lateral exposures which are planned to avoid irradiation of the spinal cord while minimizing exposure of the lung. At a fully lateral angle, the lung would receive too much dose, therefore the beam is moved anteriorly. If the beam is moved too far forward, the target area and the spinal cord overlap. A simulation of an unacceptable overlap can be seen in Fig. 4. Figure 3, on the other hand, shows adequate separation. Besides altering the incident direction of the beam, the width, height, tilt (collimator angle) and centre may be interactively adjusted relative to the anatomy so as best to conform the radiation area to the target area.
481
