Trial Review

The ANZCTR website will be unavailable from 1pm until 3pm (AEDT) on Wednesday the 30th of October for website maintenance. Please be sure to log out of the system in order to avoid any loss of data.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12617000017325
Ethics application status
Approved
Date submitted
16/12/2016
Date registered
5/01/2017
Date last updated
22/12/2021
Date data sharing statement initially provided
23/07/2019
Type of registration
Retrospectively registered

Titles & IDs
Public title
Evaluation of NetworkZ: Can multidisciplinary team-training improve the safety of surgery in New Zealand?
Scientific title
Evaluation of the NetworkZ programme: Can multidisciplinary surgical simulation team-training improve teamwork processes and the safety of surgery in New Zealand?
Secondary ID [1] 290450 0
Nil known
Universal Trial Number (UTN)
U1111-1189-3992
Trial acronym
NetworkZ
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Preventable surgical harm and death 300820 0
Condition category
Condition code
Surgery 300645 300645 0 0
Other surgery
Anaesthesiology 300646 300646 0 0
Other anaesthesiology

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
NetworkZ is team-training intervention designed to improve information sharing between team members, improve communication strategies and develop mutual respect and trust between Operating Room (OR) staff.
The educational framework used is the Salas Model of effective teams. Whole operating room teams participate in immersive, simulated surgical cases where the challenges to teamwork and communication are amplified in order to demand optimal processes and expose deficiencies. Each simulation is followed by a structured debrief, led by trained and accredited simulation instructors, exploring reactions, clarifying scenario details to ensure a shared understanding of what happened, exploring what actually happened and why, and drawing lessons on communication and teamwork to apply to clinical practice. The debrief lasts approximately 45 minutes. Simulation based training is supported by communication skills training comprising four separate 30 minute workshops on the principles of teamwork and shared mental model; speaking up; closed loop communication; and structured recap. Workshops include videos, slide presentations and small group discussion. The communication skills training is delivered by trained and accredited NetworkZ instructors at the DHB.
The simulator used is a 3G Laerdal Simman, set in a simulated or real operating room environment, and integrated with realistic custom-designed surgical models of trauma or pathology requiring a specific clinical response. All patient notes, laboratory investigations, equipment, drugs, fluids and consumables such as surgical drapes, gowns, gloves and masks will be provided as in a real theatre case.
The simulated cases will be modelled on real life cases, and include up to 25 different surgical scenarios in the disciplines of general surgery, ENT/ORL, urology, plastics and orthopaedics. A full team training intervention comprises 1 full day of simulations, debriefs and discussion. In each day, there will be 4-5 simulations and debriefs.
Simulations will be chosen from the 25 available scenarios based on the participants attending the session. As nursing, technician and anaesthesia staff work across all of the surgical disciplines portrayed in the scenarios. choice of scenario will be based on the discipline of the surgeon or surgeons attending any particular course, also ensuring that participants have not previously been exposed to that scenario.
The intervention will be delivered at each District Health Board (DHB) in New Zealand and may be run either in existing operating theatres or in a purpose built simulation facility. The intervention will be progressively rolled out across all 20 DHBs over a 5 year time frame, in four cohorts with five DHBs in each. Cohorts are matched for DHB size.
Cohort 1 training commenced on 3 March 2017. Cohort 2, 3, and 4 training will commence at 12 month intervals between 2018 and 2020. The intended start date for each new cohort is 1 February. In practice there may some delays in the start date in each cohort.
Delivery of the intervention is by trained, accredited instructors. The Project team will deliver the initial courses at each DHB and train local staff as instructors through a structured training programme with ongoing monitoring and support. Course quality will be monitored and assured through a central accreditation process, instructor networks and instructor exchange.
The NetworkZ team training days will be ongoing on a regular basis in each DHB, with an expectation that all OR staff will participate in a full course at least once every five years. We aim to train the majority of NZ OR staff during the course of this project and estimate that this will be over 6000 operating theatre staff.
The intervention will run from 3 March 2017, beginning in Cohort 1, and will continue until the training is running in all four Cohorts (i.e. all DHBs in New Zealand).
The pre-intervention period will be defined as the period of 12 months prior to the commencement of training in that Cohort. The implementation period will be 15 months following the commencement of training in that Cohort. The post-intervention period will be 12 months following the implementation period in that Cohort.
Our primary outcome measure, Days Alive and out of Hospital will be measure at 90 days post surgery for all patients undergoing surgery in New Zealand public hospitals. The measurement will be from 12 months prior to the intervention in Cohort 1 to 12 months after the implementation period in Cohort 4 (plus 90 days for surgical case data). Because of the 90 day post-surgery follow-up we will continue to collect data for each DHB for 90 days after the measurement period has ended
Intervention code [1] 296305 0
Other interventions
Comparator / control treatment
We are using a stepped wedge cluster design with staged roll-out of the intervention across the 20 DHBs in New Zealand in four cohorts at 12 month intervals, over a period of 4 years and including a transition phase (implementation period) of 15 months.
Using this design, the DHBs will act as their own controls as the intervention progressively rolls out across cohorts.
Control group
Active

Outcomes
Primary outcome [1] 300069 0
Days Alive and Out of Hospital at 90 Days. This data is downloaded from a national data base. All hospital admissions, discharges and readmissions are entered into the National Minimum Data Set (NMDS) as well as complete details of patient identity, hospital, diagnosis, patient co-morbidities, procedures undertaken. In addition, mortality is entered into another national database which can be linked to the NMDS. The files of all patients undergoing the surgical procedures of interest will be provided to the research team by the Ministry of Health, from which we will be able to calculate Days Alive and Out of Hospital.
Timepoint [1] 300069 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.

We also intend to undertake an interim analysis of the impact of the programme on Days Alive and Out of Hospital (DAOH) scores, based on data available at 17 March 2020. The interim analysis will inform funding applications and future programme planning. The outcome of the interim analysis will not alter the conduct of the trial or the final analysis. The final analysis will continue to follow the methodology described here and in the protocol publication in BMJ Open. The final analysis of the impact of the programme will be conducted in 2022, 12 months after the transition period for the final cohort (Cohort 4), as outlined in the original protocol description.
Primary outcome [2] 300225 0
Days out of hospital at 90 days post-surgery calculated from the NMDS.
Timepoint [2] 300225 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.

We also intend to undertake an interim analysis of the impact of the programme on Days Out of Hospital (DOH) scores, based on data available at 17 March 2020. The interim analysis will inform funding applications and future programme planning. The outcome of the interim analysis will not alter the conduct of the trial or the final analysis. The final analysis will continue to follow the methodology described here and in the protocol publication in BMJ Open. The final analysis of the impact of the programme will be conducted in 2022, 12 months after the transition period for the final cohort (Cohort 4), as outlined in the original protocol description.
Primary outcome [3] 300226 0
Mortality and days alive post-surgery calculated from data from the Ministry of Health Mortality Data Set, linked through patient National Health Index, to the National Minimum Data Set.
Timepoint [3] 300226 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.

We also intend to undertake an interim analysis of the impact of the programme on days alive scores, based on data available at 17 March 2020. The interim analysis will inform funding applications and future programme planning. The outcome of the interim analysis will not alter the conduct of the trial or the final analysis. The final analysis will continue to follow the methodology described here and in the protocol publication in BMJ Open. The final analysis of the impact of the programme will be conducted in 2022, 12 months after the transition period for the final cohort (Cohort 4), as outlined in the original protocol description.
Secondary outcome [1] 328936 0
Post-operative complications recorded in the National Minimum Data Set
Timepoint [1] 328936 0
90 days post surgery for all included surgical cases from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4. Data on postoperative complications is entered by coders at DHB level into the NMDS. This will be measured from day 1 post surgery to 90 days post-surgery.

We also intend to undertake an interim analysis of the impact of the programme on days alive scores, based on data available at 17 March 2020. The interim analysis will inform funding applications and future programme planning. The outcome of the interim analysis will not alter the conduct of the trial or the final analysis. The final analysis will continue to follow the methodology described here and in the protocol publication in BMJ Open. The final analysis of the impact of the programme will be conducted in 2022, 12 months after the transition period for the final cohort (Cohort 4), as outlined in the original protocol description.
Secondary outcome [2] 328937 0
This is a composite secondary outcome of Accident Compensation Corporation (ACC) treatment injury claims including volume, cost, and categorisation from ACC claims data base.
Timepoint [2] 328937 0
Measured continuously from one year to the start of training on Cohort 1, to one year after the end of faculty assisted training in Cohort 4. This covers the period for all included claims from 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.

We also intend to undertake an interim analysis of this data in mid 2020.
Secondary outcome [3] 329078 0
Change in scores in a custom-designed teamwork processes survey.
Timepoint [3] 329078 0
A pre-measure in each cohort, and post-measure 12-15 months after the start of the intervention

We also intend to run an interim analysis of the completed Cohort 1 and 2 Cohorts based on data available at mid March 2020.
Secondary outcome [4] 329079 0
Change in safety culture score measured by the Surgical Safety Culture Survey
Timepoint [4] 329079 0
This survey is being collected at a national scale in 2017, 2019 and 2021.

We intend to run an interim analysis comparing 2017 and 2019 data.
Secondary outcome [5] 329080 0
Change in Quality of administration of the WHO Safe Surgery Checklist, measured by World Health Organisation Behaviourally Anchored Rating Scale (WHOBARS).
Timepoint [5] 329080 0
A pre-intervention observational measure in each cohort up to eight months before the start of training; and a post-intervention observational measure in each cohort about 12-15 months after the start of training in each DHB in the cohort.

We intend to run an interim analysis of the completed Cohort 1 and 2 Cohorts based on the data available at mid March 2020.
Secondary outcome [6] 329087 0
Organisational implementation process (qualitative exploration and understanding)
Timepoint [6] 329087 0
Data collected from each DHB during the period from five to twelve months after the start of training.
Secondary outcome [7] 329115 0
NetworkZ effectiveness and application to clinical practice (qualitative exploration and understanding)
Timepoint [7] 329115 0
OR staff interview data, collected from each DHB during the period from three to six months post the start of training.
Secondary outcome [8] 329116 0
Uptake within the DHBs measured as the number and proportion of relevant staff who have received the training (adjusted by the number of trained staff transferred in and out of the DHB).
Timepoint [8] 329116 0
From 12 months prior to the start of training in Cohort 1 up until 12 months after the implementation period of the intervention in Cohort 4.

Eligibility
Key inclusion criteria
Eligibility for the training and staff surveys and observations: All staff working in ORs to provide general, orthopaedic, urological, plastics, and ENT/ORL surgery in NZ. The main staff groups who will be included are: surgeons, surgical trainees, anaesthetists, anaesthetic trainees, anaesthetic technicians, anaesthetic technician trainees, and theatre nurses. We will also include some staff who have a managerial interest in the implementation of NetworkZ training in their DHB, including directors of surgery, directors of nursing, and quality managers.
Eligibility for patient outcome measure: all patients having surgical procedures with anaesthesia that are listed in the procedural codes of the Health Quality and Safety Commission (HQSC) Perioperative Mortality Review Committee (PMRC), involve the previously defined surgical disciplines and are operated on in a public hospital operating room in New Zealand
Minimum age
No limit
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
Exclusion criteria for training and evaluation: Some junior medical staff that have not commenced specialised training in anaesthesia or surgery, such as house officers, may be present in the teams that we observe and rate, but we will not seek to gather survey information from them in relation to our two staff safety culture surveys. The same will apply to Health Care Assistants, who may or may not be involved in OR work.
Exclusion criteria for surgical cases: minor procedures (i.e. patients who had their operation under local infiltration only).

Study design
Purpose of the study
Educational / counselling / training
Allocation to intervention
Randomised controlled trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
The allocation of DHBs to Cohorts was as follows: The DHBs were allocated to five groups according to population. The first Cohort of five DHBs was selected based on convenience and to ensure inclusion of DHBs of appropriate sizes and geographical distribution. The remaining DHBs were randomly allocated to Cohorts 2, 3 and 4 using a stratified allocation to ensure each Cohort contained the same distribution of large to small DHBs. Due to the pragmatic nature of quality improvement interventions, DHBs unable to participate in the training at their allocated time are able to swap with a DHB of a similar size in a subsequent Cohort.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Randomised number generator in Excel for DHB allocation.
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Randomised number generator in Excel for DHB allocation
Masking / blinding
Open (masking not used)
Who is / are masked / blinded?



Intervention assignment
Other
Other design features
Stepped Wedge Cluster design
Phase
Not Applicable
Type of endpoint/s
Safety/efficacy
Statistical methods / analysis
The Primary Outcome measure (DAOH at 90 days), will be calculated from the National Minimum Data Set and the Mortality Data set.
For all patients alive at 90 days, the DAOH score is 90 minus the number of days in hospital from the date of the surgical procedure. Any part of a day in hospital will count as one day. The maximum score for any patient included in the study is 89 because the day of the procedure will count as one day in hospital. A death prior to leaving hospital will count as zero. A death within the 90 day period when the patient was discharged home from hospital will count as the number of days out of hospital during the 90 day period.
A second operation within the 90 day period will be treated as the same event with the exception of any situation where the first operation is a minor diagnostic procedure leading directly to the definitive procedure (e.g. biopsy under GA confirming Ca, followed by definitive surgery within the 90 day period).
We will include only patients in the surgical disciplines receiving the intervention, patients undergoing regional or general anaesthesia, and patients receiving an intervention listed as relevant to the concerns of the HQSC’s PMRC.

The study design will be the transition form of the stepped wedge cluster design [1}, where the transition phase will be a 15-month period of implementation of the intervention following the start of training. The total period of data collection for the DAOH analysis will be from 12 months prior to the first training day in the first DHB in Cohort 1 until 12 months after the end of the implementation period in Cohort 4. Data from the implementation (transition) phase will not be used in the analysis given that it cannot be assigned to either intervention or control arms.
Patients’ mean DAOH scores will be classified into pre-intervention (control) and post-intervention (intervention) for analysis.

We will use a non-parametric Mann Whitney rank sum tests as our primary test for an overall difference between distributions of DAOH90 before and after implementation of NetworkZ, specifying p <0.05 as significant. We will then perform quantile regression at the 0.1, 0,25, 0.75, 0.9 quantile to characterize any difference between the two distributions. This quantile regression will include time (month and year of procedure), cohort, DHB (within cohort) and intervention as factors in the model.

We will conduct a similar comparative analysis for Days Out of Hospital (for survivors); for post-surgery mortality; a defined set of post-surgery complications (for example wound infection, renal impairment, thrombotic event).

We will use a similar approach to the analysis of the ACC claims data base for treatment injury claims (number and costs).

[1] Hemming, K., et al., The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ : British Medical Journal, 2015. 350.

The teamwork perceptions surveys will be provided to participants with identifiable numbers, which will enable us to conduct paired t-tests of pre- and post-intervention survey responses from the same individuals.

For both the SSCS and teamwork perceptions surveys we will establish the "dose effect" of intervention by gathering and analysing data on which staff have received the training.

For each cohort we will analyse observational data collected via the WHOBARS tool. We will then conduct pre- and post-intervention t-tests on the WHOBARS scores. Because our observational and training records will enable us to identify the proportion of observed staff, and OR staff in each DHB, who have been trained. We will be able to investigate the relationship between these variables (i.e. the dose) and WHOBARS scores (the effect).

Quantitative measures will be compared between pre- and post-rollout phases.

Qualitative data from the interviews will undergo framework analysis to identify barriers and enablers to the implementation of NetworkZ training by DHBs, and improved communication by OR staff. The framework for this analysis will be the Organising for Quality model [2]. The themes identified will be analysed at the level of entire cohorts.

[2] Bate, P,. et al., Organising for Quality: The Improvement Journeys of Leading Hospitals in Europe and the United States. Oxford & New York, Radcliffe Publishing, 2008.

Recruitment
Recruitment status
Recruiting
Date of first participant enrolment
Anticipated
Actual
18/11/2016
Date of last participant enrolment
Anticipated
Actual
Date of last data collection
Anticipated
31/07/2022
Actual
Sample size
Target
6000
Accrual to date
1800
Final
Recruitment outside Australia
Country [1] 8368 0
New Zealand
State/province [1] 8368 0
nationwide

Funding & Sponsors
Funding source category [1] 294871 0
Government body
Name [1] 294871 0
Accident Compensation Corporation
Country [1] 294871 0
New Zealand
Funding source category [2] 298769 0
Government body
Name [2] 298769 0
Lotteries Health Research Fund New Zealand
Country [2] 298769 0
New Zealand
Funding source category [3] 299668 0
Other Collaborative groups
Name [3] 299668 0
Australia and New Zealand College of Anaesthetists
Country [3] 299668 0
Australia
Primary sponsor type
Individual
Name
Professor Jennifer Weller
Address
Centre for Medical and Health Sciences Education
School of Medicine
The University of Auckland
Private Bag 92019
Auckland 1142
Country
New Zealand
Secondary sponsor category [1] 293752 0
None
Name [1] 293752 0
Address [1] 293752 0
Country [1] 293752 0

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 296252 0
Health and Disability Ethics Committees
Ethics committee address [1] 296252 0
Ethics committee country [1] 296252 0
New Zealand
Date submitted for ethics approval [1] 296252 0
15/08/2016
Approval date [1] 296252 0
03/10/2016
Ethics approval number [1] 296252 0
16/NTB/143

Summary
Brief summary
Trial website
Trial related presentations / publications
Public notes
Attachments [1] 1230 1230 0 0
/AnzctrAttachments/371766-HDEC Letter 16NTB143 Approved FULL Application.pdf (Ethics approval)
Attachments [2] 2445 2445 0 0
/AnzctrAttachments/371766-Participant Information Sheet_v5.pdf (Participant information/consent)
Attachments [3] 2446 2446 0 0
/AnzctrAttachments/371766-Email invitation and consent.pdf (Participant information/consent)

Contacts
Principal investigator
Name 70122 0
Prof Jennifer Weller
Address 70122 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70122 0
New Zealand
Phone 70122 0
+64 9 923 9459
Fax 70122 0
+64 9 373 7641
Email 70122 0
[email protected]
Contact person for public queries
Name 70123 0
Jennifer Weller
Address 70123 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70123 0
New Zealand
Phone 70123 0
+64 9 923 9459
Fax 70123 0
+64 9 373 7641
Email 70123 0
[email protected]
Contact person for scientific queries
Name 70124 0
Jennifer Weller
Address 70124 0
Auckland City Hospital, 2 Park Road, Grafton (Support Building 599, Level 12)
Private Bag 92 019
Auckland 1142
Country 70124 0
New Zealand
Phone 70124 0
+64 9 923 9459
Fax 70124 0
+64 9 373 7641
Email 70124 0
[email protected]

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
The primary outcomes are based on hospital admissions data collected by a third party. The researchers working on this study do not have authorisation to share this data with other parties.


What supporting documents are/will be available?

Doc. No.TypeCitationLinkEmailOther DetailsAttachment
7898Study protocolWeller J, Long JA, Beaver P, et al. Evaluation of the effect of multidisciplinary simulation-based team training on patients, staff and organisations: protocol for a stepped-wedge cluster-mixed methods study of a national, insurer-funded initiative for surgical teams in New Zealand public hospitals. BMJ Open 2020;10:e032997. doi: 10.1136/bmjopen-2019-032997  



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
SourceTitleYear of PublicationDOI
EmbaseTowards a safer culture: implementing multidisciplinary simulation-based team training in New Zealand operating theatres - a framework analysis.2019https://dx.doi.org/10.1136/bmjopen-2018-027122
EmbaseEvaluation of the effect of multidisciplinary simulation-based team training on patients, staff and organisations: Protocol for a stepped-wedge cluster-mixed methods study of a national, insurer-funded initiative for surgical teams in New Zealand public hospitals.2020https://dx.doi.org/10.1136/bmjopen-2019-032997
N.B. These documents automatically identified may not have been verified by the study sponsor.