Curr Oncol, Vol. 21, pp. 224-233; doi: http://dx.doi.org/10.3747/co.21.1923
O R I G I N A L
A R T I C L E
Implementing a regional oncology information system: approach and lessons learned W.K. Evans m d * F.D. A shbury G.L. H ogue,# A. Smith bsc mba cma and J. Pun MSc mba ^ ABSTRACT
che,**
and ad ju stm en ts are o ften re q u ired to en su re th at the p ro ject m eets its objectives.
Rationale Conclusions P a p e r-b a s e d m e d ic a l re c o rd sy ste m s a re k n o w n to h ave m ajo r p ro b lem s o f inaccuracy, incom plete d ata, p o o r ac cessib ility , an d ch a llen g es to p atien t co nfidentiality. T h ey are also an inefficient m e c h a n ism o f re co rd -sh a rin g for in terd iscip lin ary p atien t asse ssm e n t a n d m an ag em en t, an d re p resen t a m ajor p ro b lem for k eep in g cu rre n t and m o n ito rin g q u ality co n tro l to facilitate im provem ent. To ad d ress those co n c ern s, n atio n al, regional, an d local h ealth care au th o rities h av e in creased th e p re ssu re on oncology p ra ctices to u p g ra d e fro m p ap e r-b a se d system s to electro n ic h ealth records.
Objectives H e re, w e d e s c rib e a n d d isc u ss th e c h a lle n g e s to im p lem en tin g a reg io n -w id e o ncology in fo rm atio n sy stem across four independent health care o rg an iza tio n s, and w e d esc rib e the lessons learn ed from the in itial p h ase s th at are now b ein g applied in su b se q u en t activ ities o f th is com plex project.
Results T h e n eed for change m ust be sh ared across centres to in cre ase buy-in, ad option, and im plem entation. It is e s s e n tia l to e s ta b lis h p h y s ic ia n le a d e rs h ip , co m m itm e n t, an d en g ag em en t in the p rocess. W ork p ro c esses h ad to be rev ised to o p tim ize u se o f the n ew system . C u ltu re change m u st be in clu d ed in the ch an g e m an ag e m en t strategy. F u rth e rm o re , tra in ing an d re so u rce re q u irem en ts m u st be th o ro u g h ly p lan n ed , im p lem en ted , m o n ito red , and m odified as req u ired for effective adoption o f new w ork processes an d technology. In terfaces m u st b e estab lish ed w ith m ultiple ex istin g electronic system s across the region to en su re ap p ro p riate p atien t flow. P eriodic a sse ss m en t o f th e ex istin g pro ject stru c tu re is necessary,
T he im p lem en tatio n o f reg io n -w id e oncolo g y in fo r m atio n system s across d iffe ren t h ealth p ra ctice lo ca tio n s has m any challenges. L ea d ersh ip is essential. A strong, collaborative in fo rm atio n -sh a rin g strateg y across the reg io n an d w ith the su p p lier is esse n tial to identify, d iscu ss, and resolve im p lem en tatio n p ro b lem s. A stru c tu re th at su p p o rts p ro ject m an ag em en t and acco u n tab ility co n trib u tes to success.
KEYWORDS E lec tro n ic m ed ica l reco rd s, on co lo g y in fo rm atio n system s, im p lem en tatio n
1. INTRODUCTION P rac tic e p a tte rn v aria tio n s are co m m o n in ca n cer care. To ensure b est practice an d to facilitate efficient health system a d m in istra tio n , ac cess to d ata ab out p ractice v aria tio n an d trea tm e n t p ro c esses an d o u t com es is critical for h ealth care p ro v id ers, p atien ts, and system a d m in istra to rs1. O n cology in fo rm atio n system s (oiss) are essential tools for m e a su rin g the ra te o f ad o p tio n an d the effec tiv en e ss o f p ra c tic e stan d ard s, for im p ro v in g p atien t safety, and for fa cilita tin g re searc h 2-5. H ow ever, th ere is co n c ern th at oiss can low er p ro d u c tiv ity by re q u irin g p ro v id ers to en ter d ata and clinical orders, and it h as b een su g g ested th at system resp o n se and dow n tim e could je o p a rd iz e p atien t c a re 6. P aper-based system s are co m m o n ly in acc u rate or in com plete an d difficult to access; th ey o ffer p o o r p ro tectio n o f p atien t confidentiality; th ey are inef ficient for re co rd -sh a rin g , in terd iscip lin ary p atien t assessm en t, an d m an agem ent; an d th ey are difficu lt to keep cu rren t. F u rth erm o re , p ap e r-b a se d reco rd s can pu t p atien t safety at risk an d lim it the cap acity
I C urrent O ncology — V olume 21, N umber 5, O ctober 2014 | Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
to m o n ito r an d ev alu ate q u a lity and effectiv en ess o f care at a tim e w hen o u tco m es asse ssm e n t has b eco m e an im p o rta n t m easu re7,8. In oncology, the ever-increasing volum e o f clinical data, the com plexi ties o f trea tm e n t and supportive care options, and the d em an d for n e a r “re a l-tim e ” outcom es underscore th e n ee d for a n ois to access, o rg a n iz e, and m anage clin ical oncology d ata9 and p o ten tially to realize true h ea lth benefits and to red u ce co sts10. T he reg io n alizatio n o f O ntario can cer services in reg io n a l ca n cer cen tres an d local health in teg ratio n netw orks provided an o p p o rtu n ity to standardize safe care p ro cesses and to im prove efficiency and reduce co sts w h ile b rin g in g care closer to hom e. T h e im ple m en tatio n o f a regional oncology inform ation system ( r o i s ) w as felt to be a key asp e ct in th e in teg ra tio n o f the reg io n al ca n cer program .
2. OBJECTIVE T h is p ap er d escrib es the challenges encountered in im p lem en tin g a r o i s across four independent health care org an izatio n s and the lessons learned from the initial p h ases o f th e im plem entation process th at are now b ein g applied in the next p hases o f this com plex project. T he vision for the original initiative w as to im plem ent a com m on, integrated r o i s to support highquality, stan d ard ized , safe, and efficient delivery o f can cer care to patients in a region o f southern O ntario. T h e im p le m e n ta tio n o f an y n ew in fo rm a tio n te c h n o lo g y ( i t ) is o fte n c h a lle n g in g . S om e c h a l len g es are p u rely tech n ical, such as in teg ra tio n o f th e new tech n o lo g y into ex istin g system s; o th ers are b eh av io u ral and re q u ire m ajor sh ifts in how clinical p ra ctice is co n d u cted. T he im p lem en tatio n p rocess is m ad e m o re com plex w hen it is u n d erta k en w ith m ultiple players at a regional level and w ith o rg an iza tions o f d iffe ren t sizes a n d cultures.
2.1 Setting A fte r ex tern al co n su ltatio n and internal leadership w o rk sh o p s in 2009, the regional oncology clinical and a d m in istra tiv e leadership ag reed to im plem ent a co m m o n r o i s at the fo u r h o sp itals th at prov ide am b u la to ry c a n c e r serv ic es in a reg io n se rv in g a p o p u latio n o f ap p ro x im ately 1.4 m illio n residents. S ystem ic ch em o th erap y w as ad m in iste red at all four facilities, b u t at th e tim e w hen the r o i s w as first d is cussed, o nly one site provided radiotherapy. A second ca n cer cen tre for th e region w as p lan n ed to open in early 2013. T h ere w as ag reem en t th at the radiation p ro g ram at th is second site w ould open as p a rt o f an in teg rated p ro g ram w ith the established radiotherapy c e n tre (site A). Site A o p e ra te d a larg e p ro g ram (11,627 new p atien t visits d u rin g fiscal 2011—2012). It also p ro v id ed 31,607 system ic th era p y trea tm e n ts and 76,333 rad iatio n th erap y trea tm e n ts d u rin g the sam e perio d . T he new ca n cer cen tre (site B) handled ___________________________________________________________________ C
1958 new patient visits and a total o f 9686 system ic therapy treatm ents. Two sm aller facilities (sites C and D) provided on-site consultation for 698 and 371 new cases respectively in 2011-2012 and delivered a total o f 3083 sy stem ic therapy treatm en ts.
2.2 The “Burning Platform” for Change Table i o u tlin es details o f the vario u s system s u sed across the region to su p p o rt p atient care in the am b u latory oncology setting. T hese oiss operated in d ep en dently and did not interface w ith th eir respective host hospital inform ation system s ( h i s s ) , thereby creatin g p o ten tially significant patient care and safety issues. T he in ab ility to tra n sfe r p atien t in fo rm atio n elec tro n ica lly betw een sites m ean t th at ca n cer patien ts receiv in g trea tm e n t in m ore than one location h ad to provide dem o g rap h ic in fo rm atio n an d m edical h is to ry on m ultiple o ccasions. P h o to co p y in g and ch a rt m irro rin g b etw een active trea tm e n t sites w as used, w hich created in crem en tal w ork and the po ten tial for incom plete or in co n sisten t p aper-based in fo rm a tion. F u rth erm o re, patients w ere often asked to ca rry th eir ow n m edical in fo rm atio n w hen referred to a provider at an o th er location. T hat p ractice negatively affected the p atien t ex p erien ce, and the tra n sp o rt o f p ap e r reco rd s w as a th rea t to p atien t confidentiality. In addition, d ata entered into one o ncology system had to be tran sferre d m anually into o ther system s for re p o rtin g to the provincial ca n cer program . T hose tran sfers in tu rn created the risk o f d ata en try erro rs and inefficiencies in re so u rce use.
2.3 Implementation Approach For the r o i s to be su ccessful, it w as felt to be critical th at all p artn e rs p artic ip a te fu lly and collaborativ ely and function as a unified team . G uiding principles for d ecisio n -m ak in g w ere estab lish ed by the leadersh ip team . T he decisions w ere to • •
be m ade in the b est in tere st o f patients; su p p o rt a stan d ard iz ed co n fig u ratio n o f the rois and its use across the region; take acco u n t o f the m ost effective an d efficient w orkflow processes; avoid u n n e c e ssa ry duplication o f ex istin g p ro cesses, d atab ases, and system s; be in fo rm ed by evidence, w hen available; be m ade tran sp a ren tly and by co n sen su s o f the p a rtn e r org an izatio n s; and be co m m u n icated op en ly to all stakeholders.
• • • • •
In addition to those g u id in g p rinciples, several crite ria for the developm ent o f the pro ject stru c tu re and asso ciated w ork p ro cesses w ere follow ed: •
urrent
M an ag ers an d su p erv iso rs w ere involved in re v iew in g and developing solutions. O
ncology—
V o lu m e
21, N u m b e r 5, O c t o b e r 2014
Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
225
EVANS etal.
table
i
Regional systems that supported patient care in the ambulatory oncology setting
Site
Systems in use
Key issues
A Electronic system for radiation treatment planning and operation of radiotherapy equipment
System features not used to full capacity
Practice administration Clinic functions
Out of date, no longer supported by supplier Paper charts in clinics
Chemotherapy system Integration with hospital information system ( his)
Computerized prescriber order entry (cpoe) as a separate system Nonexistent
B Practice administration
Electronic medical record (emr) used was several versions out of date; not supported by supplier or hospital information technology department Paper charts supporting operations and many paper-based embedded functions Electronic cpoe (same system as practice administration)
Clinic functions Chemotherapy system Integration with hospital his
None; registration of admission, discharge, and transfer (a / d/ t) in his was a separate function from the oncology information system (ois)
C Practice administration
Used site A’s system; used his scheduling system
Clinic functions
Paper charts in clinics
Chemotherapy system
Used site A’s electronic cpoe as a separate system
Integration with his
None; registration of a / d/ t in his was a separate function from ois; used for scheduling
his
D Practice administration Clinic functions
•
•
Used site A’s system; used his scheduling system Paper charts in clinics
Chemotherapy system
Used site A’s electronic cpoe as a separate system
Integration with his
Registration of a/ d/t in for scheduling
P h y sician s and o th er clin ician s w ere involved in re v ie w in g , a s s e ss in g , a n d a p p ro v in g n ew p ro c esses an d fu n c tio n s before im plem entation. C lear lin es o f a c co u n tab ility w ere established.
T h e P ro ject S teerin g C o m m ittee ( p s c ) th a t w as in itially created included representatives from sites A an d B b ecau se o f the p rio rity re p resen te d by the ex istin g ra d io th e rap y sy stem at site A and the n eed to co n v ert an o u td ated ois at site B. T he p s c co m p rised sen io r ad m in istrativ e leaders at the v ice-p resid en tial lev el, d ire c to rs o f c lin ic a l p ro g ra m s a n d i t , a n d m an ag erial-lev el rep resen tativ es w ho h ad in tim ate k n o w led g e o f the in fo rm atio n system s. A t site A , a m ajo r u p g rad e to the ra d ia tio n ois sy stem w as su cc essfu lly u n d erta k en w ith o u t signifi can t issues in F e b ru a ry 2012. C lin icia n s an d s ta ff at site B h ad ex p e rien ce in the use o f a d iffe ren t ois. Its lead ersh ip p erceiv ed th at site B w ould ex p erien ce few d iffe ren c es in th e ir w ork p ro cesses. Site B w as m o v in g to a n ew fa cility and any significant changes w ere therefo re su sp en d ed for 1 y e a r before the m ove, as d ire c te d by lead ersh ip at th at site. IC
urrent
O
ncology—
his
was a separate function from ois; used
his
Site B lacked the cap acity to develop new p ro cesses and to design p ro cess m aps identified b y cu r rent an d fu tu re state analyses. In fo rm atics leadership at site A to o k on th at responsibility. In add itio n , the r o i s su p p lier pro v id ed extensive group and o n e-o n one edu catio n over a long p erio d o f tim e w ith v a ri ous educators. Site B ask ed the ed u cato rs to focus on fu n c tio n a lity ra th e r th a n on p ro c esses, b ecau se clin ician s and s ta ff at th at site h ad no t b een involved in the p ro cess redesign. D a ta co n v e rsio n fro m th e site B ois to th e new r o i s w as a n o th e r k ey step in th e tra n sitio n . C o n v er sion re q u ire s a c a re fu l p ro c e ss o f d a ta c le a n in g and m atch in g in ad v a n ce o f the co n v ersio n p ro c ess. T he site B ois cam e fro m a d iffe re n t su p p lie r an d w as sev eral v ersio n s ou t o f date. T h e fields in w h ich som e o f th e d a ta elem e n ts w ere lo c a te d w ere n o t w ell d efin ed , an d m a n y k ey fields w e re m a in ta in e d in te x t fo rm a t r a th e r th a n as d is c re te e le m e n ts , m a k in g tra n sla tio n a n d c o n v e rsio n d ifficu lt, i f n o t im p o ssib le . In a d d itio n , th e o rig in a l su p p lie r o f so ftw a re to site B did n o t a ssist w ith d ata ac cess to fa c ilita te th e co n v ersio n .
V o lu m e 21, N u m b e r 5, O c t o b e r 2014
I Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
T he serv er for the new cancer centre at site B w as located at site A. A n algorithm w as created to describe th e escalation p ro cess and steps to address system problem s w hen they occurred. Im plem entation o f the r o i s at site B failed to occur sm oothly: it w as character ized by m ultiple system crashes, physician and sta ff fru stratio n w ith n ew w ork processes and increased tim e occupied w ith “com puter w ork,” loss o f clinical productivity, and resentm ent tow ard a system that w as seen to be im posed from the outside. T h at experience led the leadership for the r o i s initiative to pause and reco n sid er how b est to u n d ertak e the next steps in the im plem entation process and how to resolve the issues ex p erienced at site B. F igure 1 illustrates the project stru ctu re u sed to guide the reboot o f the r o i s . N ow , in th e re v ise d p ro ject stru c tu re , the project sp o n so rs (the regional c h ie f in fo rm atio n officer and th e h ead o f th e reg ional ca n cer p ro g ram ) have u lti m ate acco u n tab ility for the p ro je c t’s success. T hey c o -c h a ir th e p s c , w h ich includes the clin ical vicep resid en ts fro m th e th ree o th er h o sp itals im plem ent ing th e r o i s . T he p s c has provided oversight, strategic plan n in g , and guid an ce to the project. R epresentation b y se n io r le a d e rsh ip h a s also serv ed to fa c ilita te ac cess to re so u rces across the region w hen needed. T h e p s c h as m a in ta in e d co m m u n ic a tio n w ith the su p p lier on th e statu s o f im plem entation issues and has m ade the final decisions w ith regard to “G o Live”
events. T he project m anagem ent office ( p m o ) an d four w o rk in g gro u p s su p p o rt the p s c . T he p s c co n tin u es to m eet m on th ly w ith the p m o team and the w ork in g g roup leads to receive u p d ates ab o u t the statu s o f th e ir resp ectiv e activities. T he p m o o versees all asp ects o f the p ro ject and supports the w orking groups in their decision-m aking and co m m u n icatio n across the fo u r sites. T he p m o is the p rim a ry conduit for re p o rts to the p s c fro m the w orking groups. T he p m o also m ain tain s and upd ates the m aster project plan, ensures sig n -o ff by the “m ost responsible p erso n ” an d ap p ro p riate d o cu m en tatio n o f decisions and p ro cesses, and establishes p roto co ls for issues m an ag em en t, risk m an ag em en t, chan g e m an ag em en t, and co m m unications. T he supplier supports the project tasks, tim elines, and accountabilities th ro u g h discussion w ith the p m o . T he su p p lier’s reso u rces have included e x p e rtise in i t and in fo rm atio n system s app licatio n fu n c tio n a l ity and in clinical p ro cess redesign. T he su p p lier’s exp erien ces w ith in stalla tio n s at o th er sites in th e province, nationally, and in tern atio n ally w ere shared and used, as n eeded, to help su p p o rt system co n fig u ration, w orkflow p ro c esses, an d decisio n -m ak in g . Table n o utlines the m ajor w o rk in g groups and th eir roles in the im p lem en tatio n p rocess. Ju st as the O p e ratio n al and C lin ica l In terfac e w o rk in g groups have to w ork closely together, so
Project Structure
Task G roups
f ig u r e
D epartm ents and Sites
1 The revised project structure used to guide implementation o f the regional oncology information system (ois).
C urrent O ncology — Volume 21, N umber 5, O ctober 2014 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
227
EVANS etal.
table
n The project working groups and their roles in the implementation process Key working groups11
Rolesb
Operations working group
Ensure that the regional oncology information system (rois) is built and configured to specifications before Go Live events. Ensure that process changes reflect clinic and administrative requirements. Recommend nature, type, order, and sequencing of incremental Go Live events. Oversee multidisciplinary task groups to ensure sign-off on agreed decisions.
(owg )
Clinical Interface working group
Represent all areas of clinical activity. Ensure that the rois can be successfully adopted for use at all sites. Engage clinical leaders and communicate issues and decisions across clinical teams at all sites. Provide input to all clinical components of the system build and design, including readiness for Go Live events.
( ciwg )
Change Readiness working group
Ensure that each site and functional area is prepared to adopt and implement the Go Live elements of the rois . Provide guidance to sites and managers to prepare staff for Go Live events. Plan for change events. Develop change-readiness templates and materials. Use surveys, formal and informal interviews, liaison with key clinical and “superusers” to assess site readiness for Go Live events.
(crwg )
Communications working group
Develop and implement a communications plan for all point-of-service staff to ensure their awareness of the changes about to occur and the potential effects. Develop needs assessment tools. Develop a communications toolkit for information-sharing to all levels of each organization. Organize celebratory events.
(cwg )
Task groups (multidisciplinary teams that report to owg and ciwg )
Work on specific aspects of the rois configuration or workflow processes. Provide recommendations to the owg and ciw g for ratification.
a Provide oversight and facilitate decision-making for the system build and configuration. b Working group lead accountable through the Project Management Office to the Project Steering Committee. to o d o th e C h a n g e R e a d in e s s a n d C o m m u n ic a tio n s w o rk in g g ro u p s , b e c a u s e c o m m u n ic a tio n a n d m a n a g e m e n t o f th is m a jo r c h a n g e a re c ritic a l to a s u c c e s s fu l im p le m e n ta tio n . B e c a u s e c a n c e r c a re is d e liv e re d th r o u g h f u n c tio n a l d e p a rtm e n ts a t e a c h lo c a tio n , it is e sse n tia l th a t d ir e c to r s , m a n a g e rs , a n d s u p e rv is o rs a re s u ffic ie n tly p re p a r e d fo r G o L iv e e v e n ts. C h a n g e r e a d in e s s a s s e s s m e n ts a re p re p a re d p e rio d ic a lly , a n d d e p a rtm e n t a n d site m a n a g e rs a re a c c o u n ta b le fo r e n s u rin g th a t th e ir te a m s a re p r e p a re d fo r th e G o L iv e e v e n ts , in c lu d in g p ro c e s s c h a n g e s a n d d e v e lo p m e n t o f p o lic ie s a n d p ro c e d u re s .
3.
ROIS IMPLEMENTATION
3.1 Challenges and Lessons Learned T h e r o i s im p le m e n ta tio n is s till a w o rk in p ro g re ss. It h a s h a d s u c c e s s e s , b u t th e p ro je c t h a s a lso h a d its sh a re o f c h a lle n g e s a n d le s s o n s le a rn e d . A re g io n a l im p le m e n ta tio n p ro je c t in v o lv in g fo u r in s titu tio n s
228
C
urrent
O
ncology—
V o lu m e
a n d th e re p la c e m e n t o f tw o le g a c y s y s te m s , w ith u p g ra d e s to a n e x is tin g s y s te m , is a v e ry s ig n ific a n t u n d e r ta k in g , b u t th e m a g n itu d e w a s n o t fu lly u n d e r s to o d b y m a n y o f th e s ta k e h o ld e rs. A lth o u g h th e r a d ia tio n o n c o lo g y p r o g r a m at s ite A s u c c e s s f u lly c o m p le te d a m a jo r u p g r a d e w ith o u t in c id e n t, th e ir e x p e rie n c e m ig h t h a v e m a d e th e im p le m e n ta tio n te a m o v e rc o n fid e n t a b o u t th e p ro je c t’s s u b s e q u e n t im p le m e n ta tio n a c tiv itie s . T h is in itia l p h a se o f th e p ro je c t s u c c e e d e d b e c a u se a sm all g ro u p o f c lin ic a l a n d a d m in is tra tiv e s ta f f w e re p r e se le c te d as “ s u p e r u s e rs ” to c re a te a d e ta ile d u p g ra d e p la n , w ith th e su p p lie r c o n trib u tin g r e c o m m e n d a tio n s a n d s u b s ta n tia l re s o u rc e s . In a d d itio n , a c e n tra l p ro je c t te a m w o rk e d w ith a sm a ll g ro u p o f c lin ic a l s ta f f to c re a te a c h e c k lis t o f re q u ire d ta s k s . D u rin g th e G o L ive e v e n t, m e m b e rs o f b o th th e c lin ic a l a n d th e a d m in is tra tiv e s ta f f w e re p re s e n t o v e r a w e e k e n d fo r fin a l e d u c a tio n a n d te s tin g o f th e c lin ic a l sy ste m b e fo re th e s y s te m w a s s w itc h e d o n . I n f o r m a tio n te c h n o lo g y r e s o u rc e s w e re a v a ila b le fro m th e s u p p lie r, b o th o n site a n d re m o te ly , to a s s is t th e w e e k e n d
21 , N u m b e r 5, O c t o b e r 2014________________________________________________
Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
tu rn o v er. D u rin g th e first 2 w eeks o f the up g rad e, su p e r u s e rs a n d su p p lier re so u rc e s b o th p ro v id ed ad d itio n al o n -site and rem ote su p p o rt as required. A k ey lesso n learn ed fro m th at ex p erien ce w as th at a co m m itte d , eng ag ed, and acco u n tab le m u ltid isci p lin a ry team , co m b in ed w ith supplier reso u rces, can ach iev e p o sitiv e results. T he rad iatio n oncology upgrade w as a relatively sm all com ponent o f the m uch larger rois im plem enta tion plan. T he need for a com m on rois had been w idely em b raced , including th ese key elem ents for success: • • • • •
E sta b lish in g th e n eed for change E stab lish in g p h y sician lead ersh ip and en g ag e m en t in th e change p ro cess R evising w ork processes to optim ize use o f the ois Id e n tify in g all e d u c a tio n a n d re so u rc e n ee d s re q u ired for im p lem entation E s ta b lis h in g a p p ro p ria te in te rfa c e s to o th e r h o sp ital an d reg io n al system s to b e tte r in teg rate p atien t care
D e sp ite a c k n o w le d g in g th o se c ritic a l su ccess facto rs, n u m ero u s m isste p s occu rred .
3.2 Establishing the Need for Change T h e lite ratu re on change m anagem ent clearly defines th e n e e d for a “b u rn in g p latfo rm ” for change and effectiv e co m m u n icatio n o f th at n eed for change to all stak eh o ld ers11. T he tim e re q u ire d to id en tify and ch oose the m ost ap p ro p riate ois, to assess the effects on w orkflow p ro cesses, to p u rch ase and im plem ent the new sy stem , to learn the system , and to integrate it in to p ra c tic e c a n n o t b e u n d e rsta te d . A lack o f p artic ip a tio n in the d ecisio n -m ak in g p rocess for the p ro p o se d system ch ange can co n trib u te to change resistance. Inad eq u ate com m unication about w hy the o rg a n iz atio n has chosen to ad o p t a new inform ation system and ab o u t the potential benefits can resu lt in a lack o f co m m itm e n t to the change p rocess, w hich can , in tu rn , re su lt in a lack o f co m m itm e n t to new w o rk pro cesses. T h e fallout can be p o o r d ata entry, in creased tim e sp en t pro b lem -so lv in g and co m m u n icatin g solutions, and challenges in accessin g and re p o rtin g d a ta 12. C o m m u n ica tin g the “ b u rn in g p latfo rm ” to em ployees at m ultiple facilities is challenging. A lthough the a d m in istrativ e lead ersh ip w as fu lly co m m itted to th e v isio n o f a larg er in teg ra te d ca n cer system , th e clin ical lead ersh ip had co m p etin g persp ectiv es, in clu d in g a stro n g em o tio n al loyalty to th e ir host in stitu tio n w ith its em b edded p ro cesses and culture. T he poin t-o f-care s ta ff show ed m uch good w ill at the sta rt o f th e p roject, but little know ledge of, or rela tio n sh ip w ith , th eir co u n te rp a rts at the o th er in stitu tions. R elatio n sh ip s are im p o rtan t. M ore w ork could have b een done to en su re b e tte r co m m u n icatio n and o p p o rtu n itie s for successful collaboration, including
g ettin g to know one a n o th e r’s w ork en v iro n m en ts, so that, as problem s em erged, the focus w ould b e on jo in t problem -solving. D ecisions to freeze it in itia tives to su p p o rt the site B m ove created a “b u rn in g p la tfo rm ,” but one w ith in su fficie n t tim e for key initiatives such as d ata clean-up in the legacy system .
3.3 Establishing Physician Leadership and Engagement in the Change Process C hange in physician b eh a v io u r an d sh ifts in cu ltu re have b een extensively studied. L ack o f change re a d i n ess occu rs largely b ecau se ex p e rien ce d physician s and o th er health care w orkers have en tren ch ed w ork ing styles and routines that suit th eir personal practice styles. P ractitioners also develop w o rkarounds based on the real o r p erceived deficiencies o f a new p ro cess o r electronic system , and such p ra ctices can b e d if ficult to eradicate. T he value p ro p o sitio n o f an ois change has to em p h asize efficiencies an d b e tte r p atien t outcom es; how ever, p h y sician s a n d o th e r s ta f f m u st m ak e a m a te ria l an d h u m an re so u rce in v estm en t, a n d th e benefits o f ad o p tin g innovative technologies m u st be seen to outw eigh the w ork and costs o f im p lem en tation. I f eith er or bo th o f the change p ro cess an d tech n o lo g y are difficult to learn, adoption w ill likely be difficult. E nabling facto rs such as in tu itiv e u ser in terfa ces, co m m u n icatio n o f a clear u n d e rsta n d in g o f how w ork p ro c esses w ill change, an d p rovisio n o f education an d tra in in g in the new w o rk p ro cesses are essential. R ein fo rcin g a c u ltu re o f co n tin u o u s le a rn in g c a n also sig n ific a n tly im p ro v e su cc ess. P h y sician s are b u sy p ro fessio n als. L e a rn in g h o w to use new tech n o lo g ies re q u ires a significant tim e investm ent, and i f an u n d e rsta n d in g o f the valu e o f the tech n o lo g y and its p otential positive re tu rn on the in v estm en t o f th at tim e is not co m m u n icated , u p tak e can b e negatively affected. B enefits such as less tim e, efficient ac cess to test resu lts, availab ility o f in fo rm atio n reso u rces, and im proved q u ality o f p atien t care can be strong m otivators. R ecognition by p e e rs and im p ro v em en t in b illin g p ra ctice are also facto rs th at can co n trib u te to ad option in som e settin g s. S im ilar arg u m e n ts can be m ade for o th er clinical p roviders and for clerical and m an ag em en t staff. To facilitate uptake and optim al use, each group has to be co m fo rtab le w ith the new r o is . N egative attitu d e s tow ard the re p la cem e n t o f an ex istin g system , w ith ch an g es in w ork p ro c esses, have been rep o rted to be particularly com m on am ong physicians w ho are u n fa m ilia r w ith it and co m p u ter env iro n m en ts. T hose in dividuals req u ire leadersh ip , cham pions, p ra ctical education, follow -on supp o rt, and ongoing en c o u rag em en t to adopt the ch a n g e13. M an ag er and su p erv iso r c o m m itm e n t to review , develop, and sig n -o ff on new policies and p roced u res fa c ilita te s success. E sta b lish in g c le a r lin es o f a c co u n tab ility for leaders o f task groups and w ork in g
C urrent O ncology — V olume 21, N umber 5, O ctober 20141 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC). |
EVANS etal.
g ro u p s is also im p o rta n t for the success o f an y im p lem en tatio n project.
rois
3.4 Revising Work Processes to Optimize Use of the ROIS T he rois project involved significant evaluation o f cur rent p ro cesses and proposed fu tu re requirem ents. The tim e required to review and assess workflow, to choose ap p ropriate p ro gram s, to im plem ent and learn how to use the softw are, to enter current and past patient data, to scan relevant im ages, and to learn how to access inform ation all disrupted “norm al” workflow. The care p roviders, it staff, and inform ation system s supplier h ad to jo in tly co nsider how w ork w ould be done in the fu tu re to take best advantage o f the softw are for care d eliv ery an d p atient safety. T h at discussion should have been the catalyst for change w ith in the o rganiza tion. U nfortunately, relatively less w ork w as invested in pro cess re-en g in eerin g for the rois, and th at w ork w as not p erform ed collaboratively w ith all sites. Site A took the lead on process redesign because site B had less capacity. Sites C and D p articipated in the process review and red esign o f w ork processes, but as a result o f lim ited tim e and resources, requested to be left out o f the larger im plem entation process until closer to th eir G o Live dates. T he result w as that site B did not feel o w nership o f the new w ork-process m aps. W hen the new softw are w as im plem ented, som e o f the site B p erso n n el co n tinued to w ork as th o u g h th eir w ork pro cesses h ad not changed, o r they w orked around th e n ew so ftw a re ’s p erceiv ed in ad eq u acies, w hich negatively affected overall adoption o f the new system . F u rth e rm o re , c o n v e rtin g fro m a p a p e r-b a se d m ed ical re co rd (w ith som e level o f electro n ic reco rd use) to “p ap e rless” (or near-paperless) d eliv ery can be trem en d o u sly challenging. W ith p ap e r system s, ad ju stm en ts to care plans or p ra ctice no tes can be m ad e easily; by co n tra st, ac cessin g a co m p u ter and e n terin g d ata o n lin e w as ch allen g in g for som e u s ers. In ad d itio n , n o t everyone w ho w as ex p ected to w o rk w ith th e r o i s h ad the sam e level o f co m p u ter or k ey b o a rd in g skills. M oreover, a p ap er-b ased system ca n h ave in h ere n t p ro m p ts for the steps th at are to follow, w h ich m ig h t no longer ex ist in an electronic en v iro n m en t. T h u s, i f the O IS w orkflow does not in clude th e p ro m p ts w ith w h ich p ro v id ers are fam iliar, co n fu sio n m ay result, and the system u sers m ight not k n o w w h ich action or actions are to follow, even i f th e O IS inclu d es o th er prom pts to a le rt actions. In tro d u c tio n o f the r o i s into a p a p e r e n v iro n m e n t, a n d “ m ix e d e n v iro n m e n ts ” th a t c o m b in e b o th e le c tro n ic a n d p a p e r sy ste m s, re q u ire d th at old h ea lth re co rd s (or selected key d a ta elem ents) a n d so m e c u rre n t d o c u m e n ta tio n be s c a n n e d so th at th e c u rre n t p atien t reco rd w ould be com plete. S can n in g p ro c esses are labour- an d tim e-in ten siv e an d m ig h t h ave re su lted in u n n e c e ssa ry m aterials b ein g scan n ed . It is essen tial to create policies and IC
urrent
O
ncology—
p ro c ed u re s th at sp ec ify the d o cu m en ts th at have to b e sca n n ed and to pro v id e sufficient re so u rces for the w ork. T he p ro je c t’s lead ersh ip d e te rm in e d th at it w as not essential th at all h isto rical p atien t reco rd s be scanned, b u t th at selected patients (those cu rren tly on active trea tm e n t, for exam ple) have a core set o f clin ical in fo rm atio n sca n n ed , p a rtic u la rly i f th ere w as a n eed to tra n sfe r in fo rm atio n b etw e en p ro v id er sites. N e v erth eless, in fo rm atio n in scan n ed fo rm at ca n n o t be easily re trie v e d u sin g electro n ic search strateg ie s, a c irc u m sta n c e th at m ak es it v ita l th at stan d ard s be created for w h at is to be scan n ed and w h ere the scan n ed d o cu m en ts are to b e co n sisten tly m ain tain ed w ith in the electro n ic record.
3.5 Identifying Education and Resource Needs to Support Implementation T he early stage o f a new in fo rm atio n system im p le m e n ta tio n p ro c e ss g e n e ra lly re q u ire s su b s ta n tia l re so u rc e s for ed u c atio n in a c tiv ities such as how to log into the system and how to ty p e, scan, enter, an d tra n sfe r data; o th erw ise , accep tan ce ca n be in jeopardy. Table in su m m arize s the relevant education an d o th er re so u rce needs. A k ey learn in g w as th at so ftw are ap p licatio n s can be cu sto m ized to m eet n eed s, but th at sta n d a rd ized, co n sisten t edu catio n o f end u sers is essen tial. T he su p p lier’s ed u catio n s ta ff o ften p re sen ted too m an y d iffe re n t te c h n iq u e s for a c c o m p lish in g th e sam e end, w h ich u ltim ate ly c o n fu sed clin ician s and s ta ff and u n d erm in e d confidence in the q u ality o f the p ro c esses an d d ata th at w ould be captured . A n extensive evaluation process, including a s ta ff survey, w as u n d e rta k e n w ith s ta ff at sites A , C, and D to im prove upon th e u ser ex p erien ce en c o u n te red at site B. T he su rv e y w as p ilo t-tested w ith a g ro u p o f key in te rd isc ip lin a ry people. It w as then d istrib u te d to all staff. A n overall re sp o n se ra te o f 45% w as achieved. To en su re th at s ta ff w ere m ore com fortable w ith the changes bein g im p lem en ted , k ey learn in g s (h ig h lig h ted in Table iv) w ere used to focus the tra in ing and p ro c ess change tasks.
3.6 Establishing Interfaces with Other Information Systems T he goal o f in teg ra tio n o f the r o i s w ith o th er h i s s re q u ire d c o n sid e ra tio n o f how b e st to h a rm o n iz e p ro c e sse s, d a ta , a n d re p o rtin g . T h e k ey ste p s to ac h iev in g th at goal included • • •
d ev elo p in g a v isio n fo r th e in te g ra tio n o f th e in fo rm atio n system s. fu lly d o c u m e n tin g the sy stem s th a t w o u ld be re q u ire d for p o st-im p lem en tatio n integ ratio n . e n su rin g th e p re se n c e o f i t s ta f f e x p e rie n c e d w ith each system for w h ich an in terface w ould be req u ired .
V o l u m e 21, N u m b e r 5, O c t o b e r 2014
I Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
• • • • • •
harm onizing data transm ission protocols. perform ing system -w ide, integrated testing. ensuring com m unication and collaboration by resources across all sites. perform ing intensive data mapping.
it
creatin g o p eratio n al re p o rts to share across sites. re d esig n in g p ro c esses so th at the in terfaces su p p o rte d clin ical activity.
G iven th e fo u r sep arate his in stalla tio n s, signifi can t com plications arose w ith resp ect to a “com plete” in teg ra tio n ap p ro ach w ith th e rois. T he definition o f th e “ so u rce o f tru th ” for v alid atio n o f p atien ts across th e sep a rate sy stem s proved dau n tin g . F u rth er, the co sts o f p u rc h asin g the gen eric in terfa ces from the his su p p liers acro ss the fo u r sites, and the costs to cu sto m ize a n d m a in ta in th em w ere large. In the end, it w as d e te rm in e d th at the p rim e goal w as to en su re th a t th e a d m issio n , discharge, an d tra n sfe r his w ould send in fo rm atio n u n id ire c tio n a lly to the rois to keep it cu rren t.
table
M any system slow dow ns and “crashes” o cc u rre d b ecau se o f in tera ctio n s w ith o th er so ftw are ap p lica tio n s such as a n tiv iral softw are. T h o se o cc u rre n ces u n d e rs c o re d th e n e e d fo r b e tte r c o m m u n ic a tio n b etw e en it d ep a rtm en ts w ith re g ard to an y system changes th at m ig h t affect th e reg io n -w id e system . A lso, the region-w ide co n fig u ratio n w as h an d led by a c en tral group, w hich (it w as le a rn e d over tim e) h ad not been provided w ith com plete in fo rm atio n about all the system s o p eratin g at the v ario u s sites. F u r th erm o re, no m ech a n ism ex isted to stan d ard iz e the policies and p ro cesses for system co n fig u ratio n and im plem entation acro ss the region. T h e lack o f such a m ech a n ism w as not an rois issue, bu t b ecam e one because o f the in teractio n o f the rois w ith all the hiss. T he com plexity o f an im plem entation increases w hen interfaces are required across m ultiple sites w ith standalone technologies and w ith disparate policies on data ow nership, privacy, and confidentiality (such as w ho is responsible for releasing inform ation to a patient or to a provider). T he need for a higher level o f
m Education and other resources needed during the early stages of implementing a new information system Need
Required support
Key elements
Training sessions tailored to the specific needs of each professional group
Must use standardized methodology and format across all locations Training should be standardized based on workflow processes so that trainer-based fluctuations can be controlled
Process maps and education sessions for each professional group
Each professional has to understand their personal role—and those of others—in relation to orders, data entry, edit, and correction processes
Practice system for individual training and for interprofessional process simulation
A fully functional practice system for use well before “Go Live,” for training, education, and practice purposes
Regular reset of training system to allow for practice with data elements (for example, lab results for management) Realistic simulation of work processes that can manage data in a “learning laboratory”
Assessment of user readiness
Surveys, polls, and direct and indirect feedback with key user representatives and “super-users” to regularly assess the level of acceptance of the new system by the various user groups
Use of “traffic light” indicators to assess readiness of users and to focus training efforts with respect to the nature, type, and effort of training required for key groups
Supplier-based best-practice options
A test system based on initial site reviews and process re-engineering expertise that key users can work with to refine needs Refinements to the initial test system to meet regional requirements
Gain acceptance of key users for standardized protocols and work processes Disseminate best-practice learnings from other sites globally to for local use
Key internal resource development
Cooperation with the vendor to achieve full competency training for members of the key resource team who will be responsible for system maintenance and who will participate in ongoing system development
Create a multidisciplinary group of users from across all locations Determine the key resources—both technical (Information Technology) and developmental (reports, information flow)—to be trained
Login Data entry
Work processes
C urrent O ncology — V olume 21, N umber 5, O ctober 20141 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC). |
EVANS etal.
table
iv Key findings from the staff survey about the new regional oncology information system ( rois) implementation Finding
Comments
Potential increase in workload with new rois
Nurses, followed by physicians, were significantly more likely than other professional staff to believe that the rois implementation would increase workload.
Documentation for care improved
Nurses were significantly more likely than physicians and others to believe that the new rois functionality would improve how care was documented.
Improvements in efficiency with rois
Fewer than half the respondents (44%) believed that the rois would not improve their efficiency, but more than 50% indicated that the rois was an essential part of improving efficiency.
Perceptions
Identified needs Training a key requirement
Training was customized to each user group and presented in modules. Group training was identified as the preferred model, with one-on-one second and selfstudy third.
Cheat sheets and process maps
Resources were developed as part of the training program. A personalized “passport” for training, developed for each user type, highlighted the key learning requirements and provided “test” samples to gauge competency with the modules.
Access to computers
Migration from paper charts necessitated a significant increase in the number of devices available within the clinics and elsewhere to ensure that all appropriate staff could access the rois as required without delay.
Expert users
Cadres of “super-users” were developed to facilitate training and to act as reference staff during implementation. The super-users were drawn from clinical and nonclinical areas at all four sites.
Keyboarding skills
Simple Web-based keyboarding practice tools were provided to all staff who felt that their keyboarding skills were not up to par, a feeling that was both a psychological impedi ment to implementation and a practical one as older staff struggle with computerized technology.
Practice time outside of work hours
All staff were actively encouraged to practice their new skills on the training system outside of their work hours.
Computer response time
The amount of information available within the rois (20+ years of patient history) meant that processing time lags of 5-10 seconds occurred in some transactions.
coordination betw een the it departm ents o f the region’s hospitals w as a catalyst for the creation o f a regional ois g roup to w o rk collaboratively on resolving m any o f the in terface and system standardization issues.
4.
CONCLUSIONS
T he im p lem en tatio n o f a new in stitu tio n al in fo rm a tio n sy stem is ch a lle n g in g at th e b est o f tim es, but im p lem en tatio n on a region-w ide basis b rin g s new challenges. Strong leadership is req u ired to articulate a clear v isio n an d the n eed for m ajor change. A ctive p a rtic ip a tio n by all key leaders w ho w ill be affected b y th e ch an g e en su res ad eq u ate input to, an d o w n ersh ip of, th e w ork p ro cesses n e c e ssa ry to achieve th e d esired p ro jec t results. To fa cilitate co m m itm e n t to an im p lem en tatio n th at w ill u ltim ate ly affec t the fu tu re w o rk th ey w ill do an d how th ey w ill do it, the n eed for ch an g e m u st be clearly an d re p eated ly co m m u n icated to all front-line staff. A plan for the overall
232
im plem entation p ro cess and its subcom pon en ts m ust b e co m m u n icated in an u n d ersta n d ab le form to the relevant constituencies. A ll in s titu tio n s h av e th e ir o w n c u ltu re s an d w ays o f d o in g b u sin e ss, w h ich can be ch a lle n g in g w h e n it is e sse n tia l th a t th e y s ta n d a rd iz e th e ir w o rk p ro c e sse s to a sin g le ap p ro ach . S en io r ex e cu tiv es from each p a rtic ip a tin g in stitu tio n m u st jo in tly steer the p ro c ess and co m m it to a sta n d a rd iz e d ap p ro ach . C lin ic a l a n d p o in t-o f-c a re s t a f f fro m all p a r tn e r o rg a n iz a tio n s m u st a lso p a r tic ip a te in d e v e lo p in g th e n ew sta n d a rd w o rk -p ro ce ss m ap s, b ec au se th e sy stem w ill be co n fig u re d a c c o rd in g to th o se m ap s, and ed u c atio n m u st b e sta n d a rd iz e d to the new w o rk p ro c e sse s. In th is re sp e c t, stro n g su p p o rt fro m the supplier, w o rk in g in a close p a rtn e rs h ip w ith th e clien ts b o th in th e e d u c a tio n o f u se rs and in in fo rm in g the clien t ab o u t b e s t p ra c tic e s fro m p rio r im plem entations, can help to en su re the overall su cc ess o f a m u lti-site ois im p le m e n tatio n .
C urrent O ncology — V olume 21, N umber 5, O ctober 2014______________________________________ Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
5. CONFLICT OF INTEREST DISCLOSURES W K E is th e fo rm er c e o o f the regional ca n cer p ro g ra m and h as no financial conflicts o f interest. A S is th e fo rm er p ro ject d ire c to r o f th e r o i s , h ired by th e reg io n a l ca n cer pro g ram . F D A an d JP are c o n su ltan ts c o n tra cted by the ois supplier and regional ca n cer p ro g ram to ev alu ate im plem entation o f the r o t s . G L H is the fo rm er general m an ag er and senior v ice-p resid en t, E lek ta, su p p lier o f the ois; he also ow n s sto ck in E lekta.
6. REFERENCES 1. Ashbury FD, Madlensky L, Raich P, et al. Antidepressant pre scribing in community cancer care. Supportive Care Cancer 2003;11:278-85. 2. Bates DW. Getting in step: electronic health records and their role in care coordination. J Gen Intern Med 2010;25:174-6. 3. Kralj B, Iverson DC, Hotz K, Ashbury FD. The impact of computerized clinical reminders on physician prescribing behavior. Am J Med Qual 2003;18:197-203. 4. Deutscher D, Hart DL, Dickstein R, Horn SD, Gutvirtz M. Implementing an integrated electronic outcomes and elec tronic health record process to create a foundation for clinical practice improvement. Phys Ther 2008;88:270-85. 5. Kanas G, Morimoto L, Mowat F, O’Malley C, Fryzek J, Nordyke R. Use of electronic medical records in oncology outcomes research. Clinicoecon Outcomes Res 2010;2:1-14. 6. Klumpp TR. Electronic medical records and quality of cancer care. Curr Oncol Rep 2013;15:588-94. 7. Dick RS, Steen EB, Dether DE, eds. The Computer-Based Patient Record: An Essential Technology fo r Health Care. 2nd ed. Washington DC: The National Academies Press; 1997. 8. Fletcher RD, Dayhoff RE, Wu CM, Graves A, Jones RE. Computerized medical records in the Department of Veterans Affairs. Cancer 2001;91(suppl):1603-6.
9. Englebardt SP, Nelson R. Health Care Informatics: An Inter disciplinary Approach. St. Louis, MO: Mosby; 2001. 10. Hillestad R, Bigelow J, Bower A, et al. Can electronic medical record systems transform health care? Potential health benefits, savings and costs. Health A ff (Millwood) 2005;24:1103—17. 11. Hotter JP. Leading change: why transformation efforts fail. Harv Bus Rev 2007;:123-31. 12. Lai JK, Lau F, Shaw N. A study of information technology use and implementation of electronic medical record systems in BC medical practices. B C M edJ 2009;51:114-21. 13. Terry AL, Thorpe CF, Giles G, etal. Implementing electronic health records: key factors in primary care. Can Fam Physician 2008;54:730-6.
C o rresp o n d en ce to: W illiam E vans, O n co sy n th esis C o n su ltin g Inc., 352 B ay S treet S outh, H a m ilto n , O n tario L8P 3J9. E -m a il: billev an s@ co g eco .ca *
D e p artm en t o f O ncology, F acu lty o f H ealth Sci ences, M cM aster U niversity, H a m ilto n , ON. f D alla L ana School o f P ublic H ealth, U niversity o f Toronto, Toronto, ON. t D ivision o f P reventive O ncology, U n iv ersity o f C algary, C algary, A B . § Illaw a rra H ealth and M edical R esearch Institu te, U n iv ersity o f W ollongong, N ew S outh W ales, A ustralia. II In tellig en t Im p ro v em en t C o n su lta n ts, Toronto, ON. # In sig h tfu l S olutions, E nglew ood, CO, U .S.A . ** A D S C o n su ltin g S olutions, A n caster, ON.
C urrent O ncology — V olume 21, N umber 5, O ctober 2014 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
233
Copyright of Current Oncology is the property of Multimed Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
O R I G I N A L
A R T I C L E
Implementing a regional oncology information system: approach and lessons learned W.K. Evans m d * F.D. A shbury G.L. H ogue,# A. Smith bsc mba cma and J. Pun MSc mba ^ ABSTRACT
che,**
and ad ju stm en ts are o ften re q u ired to en su re th at the p ro ject m eets its objectives.
Rationale Conclusions P a p e r-b a s e d m e d ic a l re c o rd sy ste m s a re k n o w n to h ave m ajo r p ro b lem s o f inaccuracy, incom plete d ata, p o o r ac cessib ility , an d ch a llen g es to p atien t co nfidentiality. T h ey are also an inefficient m e c h a n ism o f re co rd -sh a rin g for in terd iscip lin ary p atien t asse ssm e n t a n d m an ag em en t, an d re p resen t a m ajor p ro b lem for k eep in g cu rre n t and m o n ito rin g q u ality co n tro l to facilitate im provem ent. To ad d ress those co n c ern s, n atio n al, regional, an d local h ealth care au th o rities h av e in creased th e p re ssu re on oncology p ra ctices to u p g ra d e fro m p ap e r-b a se d system s to electro n ic h ealth records.
Objectives H e re, w e d e s c rib e a n d d isc u ss th e c h a lle n g e s to im p lem en tin g a reg io n -w id e o ncology in fo rm atio n sy stem across four independent health care o rg an iza tio n s, and w e d esc rib e the lessons learn ed from the in itial p h ase s th at are now b ein g applied in su b se q u en t activ ities o f th is com plex project.
Results T h e n eed for change m ust be sh ared across centres to in cre ase buy-in, ad option, and im plem entation. It is e s s e n tia l to e s ta b lis h p h y s ic ia n le a d e rs h ip , co m m itm e n t, an d en g ag em en t in the p rocess. W ork p ro c esses h ad to be rev ised to o p tim ize u se o f the n ew system . C u ltu re change m u st be in clu d ed in the ch an g e m an ag e m en t strategy. F u rth e rm o re , tra in ing an d re so u rce re q u irem en ts m u st be th o ro u g h ly p lan n ed , im p lem en ted , m o n ito red , and m odified as req u ired for effective adoption o f new w ork processes an d technology. In terfaces m u st b e estab lish ed w ith m ultiple ex istin g electronic system s across the region to en su re ap p ro p riate p atien t flow. P eriodic a sse ss m en t o f th e ex istin g pro ject stru c tu re is necessary,
T he im p lem en tatio n o f reg io n -w id e oncolo g y in fo r m atio n system s across d iffe ren t h ealth p ra ctice lo ca tio n s has m any challenges. L ea d ersh ip is essential. A strong, collaborative in fo rm atio n -sh a rin g strateg y across the reg io n an d w ith the su p p lier is esse n tial to identify, d iscu ss, and resolve im p lem en tatio n p ro b lem s. A stru c tu re th at su p p o rts p ro ject m an ag em en t and acco u n tab ility co n trib u tes to success.
KEYWORDS E lec tro n ic m ed ica l reco rd s, on co lo g y in fo rm atio n system s, im p lem en tatio n
1. INTRODUCTION P rac tic e p a tte rn v aria tio n s are co m m o n in ca n cer care. To ensure b est practice an d to facilitate efficient health system a d m in istra tio n , ac cess to d ata ab out p ractice v aria tio n an d trea tm e n t p ro c esses an d o u t com es is critical for h ealth care p ro v id ers, p atien ts, and system a d m in istra to rs1. O n cology in fo rm atio n system s (oiss) are essential tools for m e a su rin g the ra te o f ad o p tio n an d the effec tiv en e ss o f p ra c tic e stan d ard s, for im p ro v in g p atien t safety, and for fa cilita tin g re searc h 2-5. H ow ever, th ere is co n c ern th at oiss can low er p ro d u c tiv ity by re q u irin g p ro v id ers to en ter d ata and clinical orders, and it h as b een su g g ested th at system resp o n se and dow n tim e could je o p a rd iz e p atien t c a re 6. P aper-based system s are co m m o n ly in acc u rate or in com plete an d difficult to access; th ey o ffer p o o r p ro tectio n o f p atien t confidentiality; th ey are inef ficient for re co rd -sh a rin g , in terd iscip lin ary p atien t assessm en t, an d m an agem ent; an d th ey are difficu lt to keep cu rren t. F u rth erm o re , p ap e r-b a se d reco rd s can pu t p atien t safety at risk an d lim it the cap acity
I C urrent O ncology — V olume 21, N umber 5, O ctober 2014 | Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
to m o n ito r an d ev alu ate q u a lity and effectiv en ess o f care at a tim e w hen o u tco m es asse ssm e n t has b eco m e an im p o rta n t m easu re7,8. In oncology, the ever-increasing volum e o f clinical data, the com plexi ties o f trea tm e n t and supportive care options, and the d em an d for n e a r “re a l-tim e ” outcom es underscore th e n ee d for a n ois to access, o rg a n iz e, and m anage clin ical oncology d ata9 and p o ten tially to realize true h ea lth benefits and to red u ce co sts10. T he reg io n alizatio n o f O ntario can cer services in reg io n a l ca n cer cen tres an d local health in teg ratio n netw orks provided an o p p o rtu n ity to standardize safe care p ro cesses and to im prove efficiency and reduce co sts w h ile b rin g in g care closer to hom e. T h e im ple m en tatio n o f a regional oncology inform ation system ( r o i s ) w as felt to be a key asp e ct in th e in teg ra tio n o f the reg io n al ca n cer program .
2. OBJECTIVE T h is p ap er d escrib es the challenges encountered in im p lem en tin g a r o i s across four independent health care org an izatio n s and the lessons learned from the initial p h ases o f th e im plem entation process th at are now b ein g applied in the next p hases o f this com plex project. T he vision for the original initiative w as to im plem ent a com m on, integrated r o i s to support highquality, stan d ard ized , safe, and efficient delivery o f can cer care to patients in a region o f southern O ntario. T h e im p le m e n ta tio n o f an y n ew in fo rm a tio n te c h n o lo g y ( i t ) is o fte n c h a lle n g in g . S om e c h a l len g es are p u rely tech n ical, such as in teg ra tio n o f th e new tech n o lo g y into ex istin g system s; o th ers are b eh av io u ral and re q u ire m ajor sh ifts in how clinical p ra ctice is co n d u cted. T he im p lem en tatio n p rocess is m ad e m o re com plex w hen it is u n d erta k en w ith m ultiple players at a regional level and w ith o rg an iza tions o f d iffe ren t sizes a n d cultures.
2.1 Setting A fte r ex tern al co n su ltatio n and internal leadership w o rk sh o p s in 2009, the regional oncology clinical and a d m in istra tiv e leadership ag reed to im plem ent a co m m o n r o i s at the fo u r h o sp itals th at prov ide am b u la to ry c a n c e r serv ic es in a reg io n se rv in g a p o p u latio n o f ap p ro x im ately 1.4 m illio n residents. S ystem ic ch em o th erap y w as ad m in iste red at all four facilities, b u t at th e tim e w hen the r o i s w as first d is cussed, o nly one site provided radiotherapy. A second ca n cer cen tre for th e region w as p lan n ed to open in early 2013. T h ere w as ag reem en t th at the radiation p ro g ram at th is second site w ould open as p a rt o f an in teg rated p ro g ram w ith the established radiotherapy c e n tre (site A). Site A o p e ra te d a larg e p ro g ram (11,627 new p atien t visits d u rin g fiscal 2011—2012). It also p ro v id ed 31,607 system ic th era p y trea tm e n ts and 76,333 rad iatio n th erap y trea tm e n ts d u rin g the sam e perio d . T he new ca n cer cen tre (site B) handled ___________________________________________________________________ C
1958 new patient visits and a total o f 9686 system ic therapy treatm ents. Two sm aller facilities (sites C and D) provided on-site consultation for 698 and 371 new cases respectively in 2011-2012 and delivered a total o f 3083 sy stem ic therapy treatm en ts.
2.2 The “Burning Platform” for Change Table i o u tlin es details o f the vario u s system s u sed across the region to su p p o rt p atient care in the am b u latory oncology setting. T hese oiss operated in d ep en dently and did not interface w ith th eir respective host hospital inform ation system s ( h i s s ) , thereby creatin g p o ten tially significant patient care and safety issues. T he in ab ility to tra n sfe r p atien t in fo rm atio n elec tro n ica lly betw een sites m ean t th at ca n cer patien ts receiv in g trea tm e n t in m ore than one location h ad to provide dem o g rap h ic in fo rm atio n an d m edical h is to ry on m ultiple o ccasions. P h o to co p y in g and ch a rt m irro rin g b etw een active trea tm e n t sites w as used, w hich created in crem en tal w ork and the po ten tial for incom plete or in co n sisten t p aper-based in fo rm a tion. F u rth erm o re, patients w ere often asked to ca rry th eir ow n m edical in fo rm atio n w hen referred to a provider at an o th er location. T hat p ractice negatively affected the p atien t ex p erien ce, and the tra n sp o rt o f p ap e r reco rd s w as a th rea t to p atien t confidentiality. In addition, d ata entered into one o ncology system had to be tran sferre d m anually into o ther system s for re p o rtin g to the provincial ca n cer program . T hose tran sfers in tu rn created the risk o f d ata en try erro rs and inefficiencies in re so u rce use.
2.3 Implementation Approach For the r o i s to be su ccessful, it w as felt to be critical th at all p artn e rs p artic ip a te fu lly and collaborativ ely and function as a unified team . G uiding principles for d ecisio n -m ak in g w ere estab lish ed by the leadersh ip team . T he decisions w ere to • •
be m ade in the b est in tere st o f patients; su p p o rt a stan d ard iz ed co n fig u ratio n o f the rois and its use across the region; take acco u n t o f the m ost effective an d efficient w orkflow processes; avoid u n n e c e ssa ry duplication o f ex istin g p ro cesses, d atab ases, and system s; be in fo rm ed by evidence, w hen available; be m ade tran sp a ren tly and by co n sen su s o f the p a rtn e r org an izatio n s; and be co m m u n icated op en ly to all stakeholders.
• • • • •
In addition to those g u id in g p rinciples, several crite ria for the developm ent o f the pro ject stru c tu re and asso ciated w ork p ro cesses w ere follow ed: •
urrent
M an ag ers an d su p erv iso rs w ere involved in re v iew in g and developing solutions. O
ncology—
V o lu m e
21, N u m b e r 5, O c t o b e r 2014
Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
225
EVANS etal.
table
i
Regional systems that supported patient care in the ambulatory oncology setting
Site
Systems in use
Key issues
A Electronic system for radiation treatment planning and operation of radiotherapy equipment
System features not used to full capacity
Practice administration Clinic functions
Out of date, no longer supported by supplier Paper charts in clinics
Chemotherapy system Integration with hospital information system ( his)
Computerized prescriber order entry (cpoe) as a separate system Nonexistent
B Practice administration
Electronic medical record (emr) used was several versions out of date; not supported by supplier or hospital information technology department Paper charts supporting operations and many paper-based embedded functions Electronic cpoe (same system as practice administration)
Clinic functions Chemotherapy system Integration with hospital his
None; registration of admission, discharge, and transfer (a / d/ t) in his was a separate function from the oncology information system (ois)
C Practice administration
Used site A’s system; used his scheduling system
Clinic functions
Paper charts in clinics
Chemotherapy system
Used site A’s electronic cpoe as a separate system
Integration with his
None; registration of a / d/ t in his was a separate function from ois; used for scheduling
his
D Practice administration Clinic functions
•
•
Used site A’s system; used his scheduling system Paper charts in clinics
Chemotherapy system
Used site A’s electronic cpoe as a separate system
Integration with his
Registration of a/ d/t in for scheduling
P h y sician s and o th er clin ician s w ere involved in re v ie w in g , a s s e ss in g , a n d a p p ro v in g n ew p ro c esses an d fu n c tio n s before im plem entation. C lear lin es o f a c co u n tab ility w ere established.
T h e P ro ject S teerin g C o m m ittee ( p s c ) th a t w as in itially created included representatives from sites A an d B b ecau se o f the p rio rity re p resen te d by the ex istin g ra d io th e rap y sy stem at site A and the n eed to co n v ert an o u td ated ois at site B. T he p s c co m p rised sen io r ad m in istrativ e leaders at the v ice-p resid en tial lev el, d ire c to rs o f c lin ic a l p ro g ra m s a n d i t , a n d m an ag erial-lev el rep resen tativ es w ho h ad in tim ate k n o w led g e o f the in fo rm atio n system s. A t site A , a m ajo r u p g rad e to the ra d ia tio n ois sy stem w as su cc essfu lly u n d erta k en w ith o u t signifi can t issues in F e b ru a ry 2012. C lin icia n s an d s ta ff at site B h ad ex p e rien ce in the use o f a d iffe ren t ois. Its lead ersh ip p erceiv ed th at site B w ould ex p erien ce few d iffe ren c es in th e ir w ork p ro cesses. Site B w as m o v in g to a n ew fa cility and any significant changes w ere therefo re su sp en d ed for 1 y e a r before the m ove, as d ire c te d by lead ersh ip at th at site. IC
urrent
O
ncology—
his
was a separate function from ois; used
his
Site B lacked the cap acity to develop new p ro cesses and to design p ro cess m aps identified b y cu r rent an d fu tu re state analyses. In fo rm atics leadership at site A to o k on th at responsibility. In add itio n , the r o i s su p p lier pro v id ed extensive group and o n e-o n one edu catio n over a long p erio d o f tim e w ith v a ri ous educators. Site B ask ed the ed u cato rs to focus on fu n c tio n a lity ra th e r th a n on p ro c esses, b ecau se clin ician s and s ta ff at th at site h ad no t b een involved in the p ro cess redesign. D a ta co n v e rsio n fro m th e site B ois to th e new r o i s w as a n o th e r k ey step in th e tra n sitio n . C o n v er sion re q u ire s a c a re fu l p ro c e ss o f d a ta c le a n in g and m atch in g in ad v a n ce o f the co n v ersio n p ro c ess. T he site B ois cam e fro m a d iffe re n t su p p lie r an d w as sev eral v ersio n s ou t o f date. T h e fields in w h ich som e o f th e d a ta elem e n ts w ere lo c a te d w ere n o t w ell d efin ed , an d m a n y k ey fields w e re m a in ta in e d in te x t fo rm a t r a th e r th a n as d is c re te e le m e n ts , m a k in g tra n sla tio n a n d c o n v e rsio n d ifficu lt, i f n o t im p o ssib le . In a d d itio n , th e o rig in a l su p p lie r o f so ftw a re to site B did n o t a ssist w ith d ata ac cess to fa c ilita te th e co n v ersio n .
V o lu m e 21, N u m b e r 5, O c t o b e r 2014
I Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
T he serv er for the new cancer centre at site B w as located at site A. A n algorithm w as created to describe th e escalation p ro cess and steps to address system problem s w hen they occurred. Im plem entation o f the r o i s at site B failed to occur sm oothly: it w as character ized by m ultiple system crashes, physician and sta ff fru stratio n w ith n ew w ork processes and increased tim e occupied w ith “com puter w ork,” loss o f clinical productivity, and resentm ent tow ard a system that w as seen to be im posed from the outside. T h at experience led the leadership for the r o i s initiative to pause and reco n sid er how b est to u n d ertak e the next steps in the im plem entation process and how to resolve the issues ex p erienced at site B. F igure 1 illustrates the project stru ctu re u sed to guide the reboot o f the r o i s . N ow , in th e re v ise d p ro ject stru c tu re , the project sp o n so rs (the regional c h ie f in fo rm atio n officer and th e h ead o f th e reg ional ca n cer p ro g ram ) have u lti m ate acco u n tab ility for the p ro je c t’s success. T hey c o -c h a ir th e p s c , w h ich includes the clin ical vicep resid en ts fro m th e th ree o th er h o sp itals im plem ent ing th e r o i s . T he p s c has provided oversight, strategic plan n in g , and guid an ce to the project. R epresentation b y se n io r le a d e rsh ip h a s also serv ed to fa c ilita te ac cess to re so u rces across the region w hen needed. T h e p s c h as m a in ta in e d co m m u n ic a tio n w ith the su p p lier on th e statu s o f im plem entation issues and has m ade the final decisions w ith regard to “G o Live”
events. T he project m anagem ent office ( p m o ) an d four w o rk in g gro u p s su p p o rt the p s c . T he p s c co n tin u es to m eet m on th ly w ith the p m o team and the w ork in g g roup leads to receive u p d ates ab o u t the statu s o f th e ir resp ectiv e activities. T he p m o o versees all asp ects o f the p ro ject and supports the w orking groups in their decision-m aking and co m m u n icatio n across the fo u r sites. T he p m o is the p rim a ry conduit for re p o rts to the p s c fro m the w orking groups. T he p m o also m ain tain s and upd ates the m aster project plan, ensures sig n -o ff by the “m ost responsible p erso n ” an d ap p ro p riate d o cu m en tatio n o f decisions and p ro cesses, and establishes p roto co ls for issues m an ag em en t, risk m an ag em en t, chan g e m an ag em en t, and co m m unications. T he supplier supports the project tasks, tim elines, and accountabilities th ro u g h discussion w ith the p m o . T he su p p lier’s reso u rces have included e x p e rtise in i t and in fo rm atio n system s app licatio n fu n c tio n a l ity and in clinical p ro cess redesign. T he su p p lier’s exp erien ces w ith in stalla tio n s at o th er sites in th e province, nationally, and in tern atio n ally w ere shared and used, as n eeded, to help su p p o rt system co n fig u ration, w orkflow p ro c esses, an d decisio n -m ak in g . Table n o utlines the m ajor w o rk in g groups and th eir roles in the im p lem en tatio n p rocess. Ju st as the O p e ratio n al and C lin ica l In terfac e w o rk in g groups have to w ork closely together, so
Project Structure
Task G roups
f ig u r e
D epartm ents and Sites
1 The revised project structure used to guide implementation o f the regional oncology information system (ois).
C urrent O ncology — Volume 21, N umber 5, O ctober 2014 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
227
EVANS etal.
table
n The project working groups and their roles in the implementation process Key working groups11
Rolesb
Operations working group
Ensure that the regional oncology information system (rois) is built and configured to specifications before Go Live events. Ensure that process changes reflect clinic and administrative requirements. Recommend nature, type, order, and sequencing of incremental Go Live events. Oversee multidisciplinary task groups to ensure sign-off on agreed decisions.
(owg )
Clinical Interface working group
Represent all areas of clinical activity. Ensure that the rois can be successfully adopted for use at all sites. Engage clinical leaders and communicate issues and decisions across clinical teams at all sites. Provide input to all clinical components of the system build and design, including readiness for Go Live events.
( ciwg )
Change Readiness working group
Ensure that each site and functional area is prepared to adopt and implement the Go Live elements of the rois . Provide guidance to sites and managers to prepare staff for Go Live events. Plan for change events. Develop change-readiness templates and materials. Use surveys, formal and informal interviews, liaison with key clinical and “superusers” to assess site readiness for Go Live events.
(crwg )
Communications working group
Develop and implement a communications plan for all point-of-service staff to ensure their awareness of the changes about to occur and the potential effects. Develop needs assessment tools. Develop a communications toolkit for information-sharing to all levels of each organization. Organize celebratory events.
(cwg )
Task groups (multidisciplinary teams that report to owg and ciwg )
Work on specific aspects of the rois configuration or workflow processes. Provide recommendations to the owg and ciw g for ratification.
a Provide oversight and facilitate decision-making for the system build and configuration. b Working group lead accountable through the Project Management Office to the Project Steering Committee. to o d o th e C h a n g e R e a d in e s s a n d C o m m u n ic a tio n s w o rk in g g ro u p s , b e c a u s e c o m m u n ic a tio n a n d m a n a g e m e n t o f th is m a jo r c h a n g e a re c ritic a l to a s u c c e s s fu l im p le m e n ta tio n . B e c a u s e c a n c e r c a re is d e liv e re d th r o u g h f u n c tio n a l d e p a rtm e n ts a t e a c h lo c a tio n , it is e sse n tia l th a t d ir e c to r s , m a n a g e rs , a n d s u p e rv is o rs a re s u ffic ie n tly p re p a r e d fo r G o L iv e e v e n ts. C h a n g e r e a d in e s s a s s e s s m e n ts a re p re p a re d p e rio d ic a lly , a n d d e p a rtm e n t a n d site m a n a g e rs a re a c c o u n ta b le fo r e n s u rin g th a t th e ir te a m s a re p r e p a re d fo r th e G o L iv e e v e n ts , in c lu d in g p ro c e s s c h a n g e s a n d d e v e lo p m e n t o f p o lic ie s a n d p ro c e d u re s .
3.
ROIS IMPLEMENTATION
3.1 Challenges and Lessons Learned T h e r o i s im p le m e n ta tio n is s till a w o rk in p ro g re ss. It h a s h a d s u c c e s s e s , b u t th e p ro je c t h a s a lso h a d its sh a re o f c h a lle n g e s a n d le s s o n s le a rn e d . A re g io n a l im p le m e n ta tio n p ro je c t in v o lv in g fo u r in s titu tio n s
228
C
urrent
O
ncology—
V o lu m e
a n d th e re p la c e m e n t o f tw o le g a c y s y s te m s , w ith u p g ra d e s to a n e x is tin g s y s te m , is a v e ry s ig n ific a n t u n d e r ta k in g , b u t th e m a g n itu d e w a s n o t fu lly u n d e r s to o d b y m a n y o f th e s ta k e h o ld e rs. A lth o u g h th e r a d ia tio n o n c o lo g y p r o g r a m at s ite A s u c c e s s f u lly c o m p le te d a m a jo r u p g r a d e w ith o u t in c id e n t, th e ir e x p e rie n c e m ig h t h a v e m a d e th e im p le m e n ta tio n te a m o v e rc o n fid e n t a b o u t th e p ro je c t’s s u b s e q u e n t im p le m e n ta tio n a c tiv itie s . T h is in itia l p h a se o f th e p ro je c t s u c c e e d e d b e c a u se a sm all g ro u p o f c lin ic a l a n d a d m in is tra tiv e s ta f f w e re p r e se le c te d as “ s u p e r u s e rs ” to c re a te a d e ta ile d u p g ra d e p la n , w ith th e su p p lie r c o n trib u tin g r e c o m m e n d a tio n s a n d s u b s ta n tia l re s o u rc e s . In a d d itio n , a c e n tra l p ro je c t te a m w o rk e d w ith a sm a ll g ro u p o f c lin ic a l s ta f f to c re a te a c h e c k lis t o f re q u ire d ta s k s . D u rin g th e G o L ive e v e n t, m e m b e rs o f b o th th e c lin ic a l a n d th e a d m in is tra tiv e s ta f f w e re p re s e n t o v e r a w e e k e n d fo r fin a l e d u c a tio n a n d te s tin g o f th e c lin ic a l sy ste m b e fo re th e s y s te m w a s s w itc h e d o n . I n f o r m a tio n te c h n o lo g y r e s o u rc e s w e re a v a ila b le fro m th e s u p p lie r, b o th o n site a n d re m o te ly , to a s s is t th e w e e k e n d
21 , N u m b e r 5, O c t o b e r 2014________________________________________________
Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
tu rn o v er. D u rin g th e first 2 w eeks o f the up g rad e, su p e r u s e rs a n d su p p lier re so u rc e s b o th p ro v id ed ad d itio n al o n -site and rem ote su p p o rt as required. A k ey lesso n learn ed fro m th at ex p erien ce w as th at a co m m itte d , eng ag ed, and acco u n tab le m u ltid isci p lin a ry team , co m b in ed w ith supplier reso u rces, can ach iev e p o sitiv e results. T he rad iatio n oncology upgrade w as a relatively sm all com ponent o f the m uch larger rois im plem enta tion plan. T he need for a com m on rois had been w idely em b raced , including th ese key elem ents for success: • • • • •
E sta b lish in g th e n eed for change E stab lish in g p h y sician lead ersh ip and en g ag e m en t in th e change p ro cess R evising w ork processes to optim ize use o f the ois Id e n tify in g all e d u c a tio n a n d re so u rc e n ee d s re q u ired for im p lem entation E s ta b lis h in g a p p ro p ria te in te rfa c e s to o th e r h o sp ital an d reg io n al system s to b e tte r in teg rate p atien t care
D e sp ite a c k n o w le d g in g th o se c ritic a l su ccess facto rs, n u m ero u s m isste p s occu rred .
3.2 Establishing the Need for Change T h e lite ratu re on change m anagem ent clearly defines th e n e e d for a “b u rn in g p latfo rm ” for change and effectiv e co m m u n icatio n o f th at n eed for change to all stak eh o ld ers11. T he tim e re q u ire d to id en tify and ch oose the m ost ap p ro p riate ois, to assess the effects on w orkflow p ro cesses, to p u rch ase and im plem ent the new sy stem , to learn the system , and to integrate it in to p ra c tic e c a n n o t b e u n d e rsta te d . A lack o f p artic ip a tio n in the d ecisio n -m ak in g p rocess for the p ro p o se d system ch ange can co n trib u te to change resistance. Inad eq u ate com m unication about w hy the o rg a n iz atio n has chosen to ad o p t a new inform ation system and ab o u t the potential benefits can resu lt in a lack o f co m m itm e n t to the change p rocess, w hich can , in tu rn , re su lt in a lack o f co m m itm e n t to new w o rk pro cesses. T h e fallout can be p o o r d ata entry, in creased tim e sp en t pro b lem -so lv in g and co m m u n icatin g solutions, and challenges in accessin g and re p o rtin g d a ta 12. C o m m u n ica tin g the “ b u rn in g p latfo rm ” to em ployees at m ultiple facilities is challenging. A lthough the a d m in istrativ e lead ersh ip w as fu lly co m m itted to th e v isio n o f a larg er in teg ra te d ca n cer system , th e clin ical lead ersh ip had co m p etin g persp ectiv es, in clu d in g a stro n g em o tio n al loyalty to th e ir host in stitu tio n w ith its em b edded p ro cesses and culture. T he poin t-o f-care s ta ff show ed m uch good w ill at the sta rt o f th e p roject, but little know ledge of, or rela tio n sh ip w ith , th eir co u n te rp a rts at the o th er in stitu tions. R elatio n sh ip s are im p o rtan t. M ore w ork could have b een done to en su re b e tte r co m m u n icatio n and o p p o rtu n itie s for successful collaboration, including
g ettin g to know one a n o th e r’s w ork en v iro n m en ts, so that, as problem s em erged, the focus w ould b e on jo in t problem -solving. D ecisions to freeze it in itia tives to su p p o rt the site B m ove created a “b u rn in g p la tfo rm ,” but one w ith in su fficie n t tim e for key initiatives such as d ata clean-up in the legacy system .
3.3 Establishing Physician Leadership and Engagement in the Change Process C hange in physician b eh a v io u r an d sh ifts in cu ltu re have b een extensively studied. L ack o f change re a d i n ess occu rs largely b ecau se ex p e rien ce d physician s and o th er health care w orkers have en tren ch ed w ork ing styles and routines that suit th eir personal practice styles. P ractitioners also develop w o rkarounds based on the real o r p erceived deficiencies o f a new p ro cess o r electronic system , and such p ra ctices can b e d if ficult to eradicate. T he value p ro p o sitio n o f an ois change has to em p h asize efficiencies an d b e tte r p atien t outcom es; how ever, p h y sician s a n d o th e r s ta f f m u st m ak e a m a te ria l an d h u m an re so u rce in v estm en t, a n d th e benefits o f ad o p tin g innovative technologies m u st be seen to outw eigh the w ork and costs o f im p lem en tation. I f eith er or bo th o f the change p ro cess an d tech n o lo g y are difficult to learn, adoption w ill likely be difficult. E nabling facto rs such as in tu itiv e u ser in terfa ces, co m m u n icatio n o f a clear u n d e rsta n d in g o f how w ork p ro c esses w ill change, an d p rovisio n o f education an d tra in in g in the new w o rk p ro cesses are essential. R ein fo rcin g a c u ltu re o f co n tin u o u s le a rn in g c a n also sig n ific a n tly im p ro v e su cc ess. P h y sician s are b u sy p ro fessio n als. L e a rn in g h o w to use new tech n o lo g ies re q u ires a significant tim e investm ent, and i f an u n d e rsta n d in g o f the valu e o f the tech n o lo g y and its p otential positive re tu rn on the in v estm en t o f th at tim e is not co m m u n icated , u p tak e can b e negatively affected. B enefits such as less tim e, efficient ac cess to test resu lts, availab ility o f in fo rm atio n reso u rces, and im proved q u ality o f p atien t care can be strong m otivators. R ecognition by p e e rs and im p ro v em en t in b illin g p ra ctice are also facto rs th at can co n trib u te to ad option in som e settin g s. S im ilar arg u m e n ts can be m ade for o th er clinical p roviders and for clerical and m an ag em en t staff. To facilitate uptake and optim al use, each group has to be co m fo rtab le w ith the new r o is . N egative attitu d e s tow ard the re p la cem e n t o f an ex istin g system , w ith ch an g es in w ork p ro c esses, have been rep o rted to be particularly com m on am ong physicians w ho are u n fa m ilia r w ith it and co m p u ter env iro n m en ts. T hose in dividuals req u ire leadersh ip , cham pions, p ra ctical education, follow -on supp o rt, and ongoing en c o u rag em en t to adopt the ch a n g e13. M an ag er and su p erv iso r c o m m itm e n t to review , develop, and sig n -o ff on new policies and p roced u res fa c ilita te s success. E sta b lish in g c le a r lin es o f a c co u n tab ility for leaders o f task groups and w ork in g
C urrent O ncology — V olume 21, N umber 5, O ctober 20141 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC). |
EVANS etal.
g ro u p s is also im p o rta n t for the success o f an y im p lem en tatio n project.
rois
3.4 Revising Work Processes to Optimize Use of the ROIS T he rois project involved significant evaluation o f cur rent p ro cesses and proposed fu tu re requirem ents. The tim e required to review and assess workflow, to choose ap p ropriate p ro gram s, to im plem ent and learn how to use the softw are, to enter current and past patient data, to scan relevant im ages, and to learn how to access inform ation all disrupted “norm al” workflow. The care p roviders, it staff, and inform ation system s supplier h ad to jo in tly co nsider how w ork w ould be done in the fu tu re to take best advantage o f the softw are for care d eliv ery an d p atient safety. T h at discussion should have been the catalyst for change w ith in the o rganiza tion. U nfortunately, relatively less w ork w as invested in pro cess re-en g in eerin g for the rois, and th at w ork w as not p erform ed collaboratively w ith all sites. Site A took the lead on process redesign because site B had less capacity. Sites C and D p articipated in the process review and red esign o f w ork processes, but as a result o f lim ited tim e and resources, requested to be left out o f the larger im plem entation process until closer to th eir G o Live dates. T he result w as that site B did not feel o w nership o f the new w ork-process m aps. W hen the new softw are w as im plem ented, som e o f the site B p erso n n el co n tinued to w ork as th o u g h th eir w ork pro cesses h ad not changed, o r they w orked around th e n ew so ftw a re ’s p erceiv ed in ad eq u acies, w hich negatively affected overall adoption o f the new system . F u rth e rm o re , c o n v e rtin g fro m a p a p e r-b a se d m ed ical re co rd (w ith som e level o f electro n ic reco rd use) to “p ap e rless” (or near-paperless) d eliv ery can be trem en d o u sly challenging. W ith p ap e r system s, ad ju stm en ts to care plans or p ra ctice no tes can be m ad e easily; by co n tra st, ac cessin g a co m p u ter and e n terin g d ata o n lin e w as ch allen g in g for som e u s ers. In ad d itio n , n o t everyone w ho w as ex p ected to w o rk w ith th e r o i s h ad the sam e level o f co m p u ter or k ey b o a rd in g skills. M oreover, a p ap er-b ased system ca n h ave in h ere n t p ro m p ts for the steps th at are to follow, w h ich m ig h t no longer ex ist in an electronic en v iro n m en t. T h u s, i f the O IS w orkflow does not in clude th e p ro m p ts w ith w h ich p ro v id ers are fam iliar, co n fu sio n m ay result, and the system u sers m ight not k n o w w h ich action or actions are to follow, even i f th e O IS inclu d es o th er prom pts to a le rt actions. In tro d u c tio n o f the r o i s into a p a p e r e n v iro n m e n t, a n d “ m ix e d e n v iro n m e n ts ” th a t c o m b in e b o th e le c tro n ic a n d p a p e r sy ste m s, re q u ire d th at old h ea lth re co rd s (or selected key d a ta elem ents) a n d so m e c u rre n t d o c u m e n ta tio n be s c a n n e d so th at th e c u rre n t p atien t reco rd w ould be com plete. S can n in g p ro c esses are labour- an d tim e-in ten siv e an d m ig h t h ave re su lted in u n n e c e ssa ry m aterials b ein g scan n ed . It is essen tial to create policies and IC
urrent
O
ncology—
p ro c ed u re s th at sp ec ify the d o cu m en ts th at have to b e sca n n ed and to pro v id e sufficient re so u rces for the w ork. T he p ro je c t’s lead ersh ip d e te rm in e d th at it w as not essential th at all h isto rical p atien t reco rd s be scanned, b u t th at selected patients (those cu rren tly on active trea tm e n t, for exam ple) have a core set o f clin ical in fo rm atio n sca n n ed , p a rtic u la rly i f th ere w as a n eed to tra n sfe r in fo rm atio n b etw e en p ro v id er sites. N e v erth eless, in fo rm atio n in scan n ed fo rm at ca n n o t be easily re trie v e d u sin g electro n ic search strateg ie s, a c irc u m sta n c e th at m ak es it v ita l th at stan d ard s be created for w h at is to be scan n ed and w h ere the scan n ed d o cu m en ts are to b e co n sisten tly m ain tain ed w ith in the electro n ic record.
3.5 Identifying Education and Resource Needs to Support Implementation T he early stage o f a new in fo rm atio n system im p le m e n ta tio n p ro c e ss g e n e ra lly re q u ire s su b s ta n tia l re so u rc e s for ed u c atio n in a c tiv ities such as how to log into the system and how to ty p e, scan, enter, an d tra n sfe r data; o th erw ise , accep tan ce ca n be in jeopardy. Table in su m m arize s the relevant education an d o th er re so u rce needs. A k ey learn in g w as th at so ftw are ap p licatio n s can be cu sto m ized to m eet n eed s, but th at sta n d a rd ized, co n sisten t edu catio n o f end u sers is essen tial. T he su p p lier’s ed u catio n s ta ff o ften p re sen ted too m an y d iffe re n t te c h n iq u e s for a c c o m p lish in g th e sam e end, w h ich u ltim ate ly c o n fu sed clin ician s and s ta ff and u n d erm in e d confidence in the q u ality o f the p ro c esses an d d ata th at w ould be captured . A n extensive evaluation process, including a s ta ff survey, w as u n d e rta k e n w ith s ta ff at sites A , C, and D to im prove upon th e u ser ex p erien ce en c o u n te red at site B. T he su rv e y w as p ilo t-tested w ith a g ro u p o f key in te rd isc ip lin a ry people. It w as then d istrib u te d to all staff. A n overall re sp o n se ra te o f 45% w as achieved. To en su re th at s ta ff w ere m ore com fortable w ith the changes bein g im p lem en ted , k ey learn in g s (h ig h lig h ted in Table iv) w ere used to focus the tra in ing and p ro c ess change tasks.
3.6 Establishing Interfaces with Other Information Systems T he goal o f in teg ra tio n o f the r o i s w ith o th er h i s s re q u ire d c o n sid e ra tio n o f how b e st to h a rm o n iz e p ro c e sse s, d a ta , a n d re p o rtin g . T h e k ey ste p s to ac h iev in g th at goal included • • •
d ev elo p in g a v isio n fo r th e in te g ra tio n o f th e in fo rm atio n system s. fu lly d o c u m e n tin g the sy stem s th a t w o u ld be re q u ire d for p o st-im p lem en tatio n integ ratio n . e n su rin g th e p re se n c e o f i t s ta f f e x p e rie n c e d w ith each system for w h ich an in terface w ould be req u ired .
V o l u m e 21, N u m b e r 5, O c t o b e r 2014
I Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
• • • • • •
harm onizing data transm ission protocols. perform ing system -w ide, integrated testing. ensuring com m unication and collaboration by resources across all sites. perform ing intensive data mapping.
it
creatin g o p eratio n al re p o rts to share across sites. re d esig n in g p ro c esses so th at the in terfaces su p p o rte d clin ical activity.
G iven th e fo u r sep arate his in stalla tio n s, signifi can t com plications arose w ith resp ect to a “com plete” in teg ra tio n ap p ro ach w ith th e rois. T he definition o f th e “ so u rce o f tru th ” for v alid atio n o f p atien ts across th e sep a rate sy stem s proved dau n tin g . F u rth er, the co sts o f p u rc h asin g the gen eric in terfa ces from the his su p p liers acro ss the fo u r sites, and the costs to cu sto m ize a n d m a in ta in th em w ere large. In the end, it w as d e te rm in e d th at the p rim e goal w as to en su re th a t th e a d m issio n , discharge, an d tra n sfe r his w ould send in fo rm atio n u n id ire c tio n a lly to the rois to keep it cu rren t.
table
M any system slow dow ns and “crashes” o cc u rre d b ecau se o f in tera ctio n s w ith o th er so ftw are ap p lica tio n s such as a n tiv iral softw are. T h o se o cc u rre n ces u n d e rs c o re d th e n e e d fo r b e tte r c o m m u n ic a tio n b etw e en it d ep a rtm en ts w ith re g ard to an y system changes th at m ig h t affect th e reg io n -w id e system . A lso, the region-w ide co n fig u ratio n w as h an d led by a c en tral group, w hich (it w as le a rn e d over tim e) h ad not been provided w ith com plete in fo rm atio n about all the system s o p eratin g at the v ario u s sites. F u r th erm o re, no m ech a n ism ex isted to stan d ard iz e the policies and p ro cesses for system co n fig u ratio n and im plem entation acro ss the region. T h e lack o f such a m ech a n ism w as not an rois issue, bu t b ecam e one because o f the in teractio n o f the rois w ith all the hiss. T he com plexity o f an im plem entation increases w hen interfaces are required across m ultiple sites w ith standalone technologies and w ith disparate policies on data ow nership, privacy, and confidentiality (such as w ho is responsible for releasing inform ation to a patient or to a provider). T he need for a higher level o f
m Education and other resources needed during the early stages of implementing a new information system Need
Required support
Key elements
Training sessions tailored to the specific needs of each professional group
Must use standardized methodology and format across all locations Training should be standardized based on workflow processes so that trainer-based fluctuations can be controlled
Process maps and education sessions for each professional group
Each professional has to understand their personal role—and those of others—in relation to orders, data entry, edit, and correction processes
Practice system for individual training and for interprofessional process simulation
A fully functional practice system for use well before “Go Live,” for training, education, and practice purposes
Regular reset of training system to allow for practice with data elements (for example, lab results for management) Realistic simulation of work processes that can manage data in a “learning laboratory”
Assessment of user readiness
Surveys, polls, and direct and indirect feedback with key user representatives and “super-users” to regularly assess the level of acceptance of the new system by the various user groups
Use of “traffic light” indicators to assess readiness of users and to focus training efforts with respect to the nature, type, and effort of training required for key groups
Supplier-based best-practice options
A test system based on initial site reviews and process re-engineering expertise that key users can work with to refine needs Refinements to the initial test system to meet regional requirements
Gain acceptance of key users for standardized protocols and work processes Disseminate best-practice learnings from other sites globally to for local use
Key internal resource development
Cooperation with the vendor to achieve full competency training for members of the key resource team who will be responsible for system maintenance and who will participate in ongoing system development
Create a multidisciplinary group of users from across all locations Determine the key resources—both technical (Information Technology) and developmental (reports, information flow)—to be trained
Login Data entry
Work processes
C urrent O ncology — V olume 21, N umber 5, O ctober 20141 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC). |
EVANS etal.
table
iv Key findings from the staff survey about the new regional oncology information system ( rois) implementation Finding
Comments
Potential increase in workload with new rois
Nurses, followed by physicians, were significantly more likely than other professional staff to believe that the rois implementation would increase workload.
Documentation for care improved
Nurses were significantly more likely than physicians and others to believe that the new rois functionality would improve how care was documented.
Improvements in efficiency with rois
Fewer than half the respondents (44%) believed that the rois would not improve their efficiency, but more than 50% indicated that the rois was an essential part of improving efficiency.
Perceptions
Identified needs Training a key requirement
Training was customized to each user group and presented in modules. Group training was identified as the preferred model, with one-on-one second and selfstudy third.
Cheat sheets and process maps
Resources were developed as part of the training program. A personalized “passport” for training, developed for each user type, highlighted the key learning requirements and provided “test” samples to gauge competency with the modules.
Access to computers
Migration from paper charts necessitated a significant increase in the number of devices available within the clinics and elsewhere to ensure that all appropriate staff could access the rois as required without delay.
Expert users
Cadres of “super-users” were developed to facilitate training and to act as reference staff during implementation. The super-users were drawn from clinical and nonclinical areas at all four sites.
Keyboarding skills
Simple Web-based keyboarding practice tools were provided to all staff who felt that their keyboarding skills were not up to par, a feeling that was both a psychological impedi ment to implementation and a practical one as older staff struggle with computerized technology.
Practice time outside of work hours
All staff were actively encouraged to practice their new skills on the training system outside of their work hours.
Computer response time
The amount of information available within the rois (20+ years of patient history) meant that processing time lags of 5-10 seconds occurred in some transactions.
coordination betw een the it departm ents o f the region’s hospitals w as a catalyst for the creation o f a regional ois g roup to w o rk collaboratively on resolving m any o f the in terface and system standardization issues.
4.
CONCLUSIONS
T he im p lem en tatio n o f a new in stitu tio n al in fo rm a tio n sy stem is ch a lle n g in g at th e b est o f tim es, but im p lem en tatio n on a region-w ide basis b rin g s new challenges. Strong leadership is req u ired to articulate a clear v isio n an d the n eed for m ajor change. A ctive p a rtic ip a tio n by all key leaders w ho w ill be affected b y th e ch an g e en su res ad eq u ate input to, an d o w n ersh ip of, th e w ork p ro cesses n e c e ssa ry to achieve th e d esired p ro jec t results. To fa cilitate co m m itm e n t to an im p lem en tatio n th at w ill u ltim ate ly affec t the fu tu re w o rk th ey w ill do an d how th ey w ill do it, the n eed for ch an g e m u st be clearly an d re p eated ly co m m u n icated to all front-line staff. A plan for the overall
232
im plem entation p ro cess and its subcom pon en ts m ust b e co m m u n icated in an u n d ersta n d ab le form to the relevant constituencies. A ll in s titu tio n s h av e th e ir o w n c u ltu re s an d w ays o f d o in g b u sin e ss, w h ich can be ch a lle n g in g w h e n it is e sse n tia l th a t th e y s ta n d a rd iz e th e ir w o rk p ro c e sse s to a sin g le ap p ro ach . S en io r ex e cu tiv es from each p a rtic ip a tin g in stitu tio n m u st jo in tly steer the p ro c ess and co m m it to a sta n d a rd iz e d ap p ro ach . C lin ic a l a n d p o in t-o f-c a re s t a f f fro m all p a r tn e r o rg a n iz a tio n s m u st a lso p a r tic ip a te in d e v e lo p in g th e n ew sta n d a rd w o rk -p ro ce ss m ap s, b ec au se th e sy stem w ill be co n fig u re d a c c o rd in g to th o se m ap s, and ed u c atio n m u st b e sta n d a rd iz e d to the new w o rk p ro c e sse s. In th is re sp e c t, stro n g su p p o rt fro m the supplier, w o rk in g in a close p a rtn e rs h ip w ith th e clien ts b o th in th e e d u c a tio n o f u se rs and in in fo rm in g the clien t ab o u t b e s t p ra c tic e s fro m p rio r im plem entations, can help to en su re the overall su cc ess o f a m u lti-site ois im p le m e n tatio n .
C urrent O ncology — V olume 21, N umber 5, O ctober 2014______________________________________ Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
IMPLEMENTING A REGIONAL ONCOLOGY INFORMATION SYSTEM
5. CONFLICT OF INTEREST DISCLOSURES W K E is th e fo rm er c e o o f the regional ca n cer p ro g ra m and h as no financial conflicts o f interest. A S is th e fo rm er p ro ject d ire c to r o f th e r o i s , h ired by th e reg io n a l ca n cer pro g ram . F D A an d JP are c o n su ltan ts c o n tra cted by the ois supplier and regional ca n cer p ro g ram to ev alu ate im plem entation o f the r o t s . G L H is the fo rm er general m an ag er and senior v ice-p resid en t, E lek ta, su p p lier o f the ois; he also ow n s sto ck in E lekta.
6. REFERENCES 1. Ashbury FD, Madlensky L, Raich P, et al. Antidepressant pre scribing in community cancer care. Supportive Care Cancer 2003;11:278-85. 2. Bates DW. Getting in step: electronic health records and their role in care coordination. J Gen Intern Med 2010;25:174-6. 3. Kralj B, Iverson DC, Hotz K, Ashbury FD. The impact of computerized clinical reminders on physician prescribing behavior. Am J Med Qual 2003;18:197-203. 4. Deutscher D, Hart DL, Dickstein R, Horn SD, Gutvirtz M. Implementing an integrated electronic outcomes and elec tronic health record process to create a foundation for clinical practice improvement. Phys Ther 2008;88:270-85. 5. Kanas G, Morimoto L, Mowat F, O’Malley C, Fryzek J, Nordyke R. Use of electronic medical records in oncology outcomes research. Clinicoecon Outcomes Res 2010;2:1-14. 6. Klumpp TR. Electronic medical records and quality of cancer care. Curr Oncol Rep 2013;15:588-94. 7. Dick RS, Steen EB, Dether DE, eds. The Computer-Based Patient Record: An Essential Technology fo r Health Care. 2nd ed. Washington DC: The National Academies Press; 1997. 8. Fletcher RD, Dayhoff RE, Wu CM, Graves A, Jones RE. Computerized medical records in the Department of Veterans Affairs. Cancer 2001;91(suppl):1603-6.
9. Englebardt SP, Nelson R. Health Care Informatics: An Inter disciplinary Approach. St. Louis, MO: Mosby; 2001. 10. Hillestad R, Bigelow J, Bower A, et al. Can electronic medical record systems transform health care? Potential health benefits, savings and costs. Health A ff (Millwood) 2005;24:1103—17. 11. Hotter JP. Leading change: why transformation efforts fail. Harv Bus Rev 2007;:123-31. 12. Lai JK, Lau F, Shaw N. A study of information technology use and implementation of electronic medical record systems in BC medical practices. B C M edJ 2009;51:114-21. 13. Terry AL, Thorpe CF, Giles G, etal. Implementing electronic health records: key factors in primary care. Can Fam Physician 2008;54:730-6.
C o rresp o n d en ce to: W illiam E vans, O n co sy n th esis C o n su ltin g Inc., 352 B ay S treet S outh, H a m ilto n , O n tario L8P 3J9. E -m a il: billev an s@ co g eco .ca *
D e p artm en t o f O ncology, F acu lty o f H ealth Sci ences, M cM aster U niversity, H a m ilto n , ON. f D alla L ana School o f P ublic H ealth, U niversity o f Toronto, Toronto, ON. t D ivision o f P reventive O ncology, U n iv ersity o f C algary, C algary, A B . § Illaw a rra H ealth and M edical R esearch Institu te, U n iv ersity o f W ollongong, N ew S outh W ales, A ustralia. II In tellig en t Im p ro v em en t C o n su lta n ts, Toronto, ON. # In sig h tfu l S olutions, E nglew ood, CO, U .S.A . ** A D S C o n su ltin g S olutions, A n caster, ON.
C urrent O ncology — V olume 21, N umber 5, O ctober 2014 Copyright © 2014 Multimed Inc. Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).
233
Copyright of Current Oncology is the property of Multimed Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
