Trial Review

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12621001385831p
Ethics application status
Submitted, not yet approved
Date submitted
6/09/2021
Date registered
14/10/2021
Date last updated
14/10/2021
Date data sharing statement initially provided
14/10/2021
Type of registration
Prospectively registered

Titles & IDs
Public title
Machine Learning to Predict Disposition from Emergency Department Triage
Scientific title
The Accuracy of Machine Learning to Predict Disposition from Emergency Department Triage
Secondary ID [1] 305229 0
None
Universal Trial Number (UTN)
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Emergency Care 323509 0
Condition category
Condition code
Emergency medicine 321070 321070 0 0
Other emergency care

Intervention/exposure
Study type
Observational
Patient registry
False
Target follow-up duration
Target follow-up type
Description of intervention(s) / exposure
We will use historical Western Australia Health Emergency Department Information System (EDIS) data from 1st Jan 2010 to 1st September 2021 to train a machine learning model. We will adapt existing state of the art natural language processing models such as BERT (Bidirectional Encoder Representations from Transformers) for this project.[1] We will also test a trained model developed by Tahayori et al. on our data.[2] We will also apply other pre-exisiting machine learning algorithms such as XGBoost. Historical EDIS data will be split into a training and test group and validated in line with current best practices. Input variables will include all information collected at the time of triage. This includes patient age, time of presentation, mode of arrival, type of residence, Australasian Triage Scale (ATS) category, injury surveillance data, and free text triage notes. Outcome data will include disposition from the ED (such as admitted to ward, intensive care, or discharged).



References

1. Devlin J, Chang M, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics; 2019 Conference of the North American Chapter of the Association for Computational Linguistics; June 2-7, 2019; Minneapolis, MN. 2019. Jun, pp. 4171–4186.

2. Tahayori B, Chini-Foroush N, Akhlaghi H. Advanced natural language processing technique to predict patient disposition based on emergency triage notes [published online ahead of print, 2020 Oct 11]. Emerg Med Australas. 2020;10.1111/1742-6723.13656. doi:10.1111/1742-6723.13656
Intervention code [1] 321625 0
Not applicable
Comparator / control treatment
We will have emergency physicians of various levels of seniority assess a subset of the test data and compare their predictions (based their own personal experience and opinions) to the best performing machine learning model.
Control group
Active

Outcomes
Primary outcome [1] 328844 0
Machine learning model accuracy in classifying patients disposition. Accuracy will be assessed by comparing the disposition prediction of the machine learning model to the ground truth (patients actual disposition). Patient disposition will be defined as admission to ward, admission to intensive care, and discharge. Ground truth will be determined from medical records.
Timepoint [1] 328844 0
End of emergency department episode of care, or 24 hours following emergency department triage (whichever is earlier).
Secondary outcome [1] 400626 0
Machine learning model accuracy in classifying patients disposition compared to emergency physician. Accuracy will be assessed by comparing the disposition prediction of the machine learning model to predictions made by emergency physicians and ground truth (patient actual disposition). Ground truth will be determined from the medical records.
Timepoint [1] 400626 0
End of emergency department episode of care, or 24 hours following emergency department triage (whichever is earlier).

Eligibility
Key inclusion criteria
All patients who presented to an emergency department and were triaged will be eligible for inclusion.
Minimum age
No limit
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
No
Key exclusion criteria
Participants will be excluded if triage information is incomplete, or if they left the emergency department without being seen (“Did Not Wait”).

Study design
Purpose
Screening
Duration
Cross-sectional
Selection
Defined population
Timing
Retrospective
Statistical methods / analysis
We will provide descriptive statistics on the characteristic of the dataset used. We will report the predictive performance of our models and emergency physicians in terms of sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve. Confidence intervals and power calculations will be included where appropriate.

Recruitment
Recruitment status
Not yet recruiting
Date of first participant enrolment
Anticipated
1/11/2021
Actual
Date of last participant enrolment
Anticipated
1/11/2021
Actual
Date of last data collection
Anticipated
1/11/2021
Actual
Sample size
Target
1000000
Accrual to date
Final
Recruitment in Australia
Recruitment state(s)
WA

Funding & Sponsors
Funding source category [1] 309604 0
Other Collaborative groups
Name [1] 309604 0
Western Australian Health Translation Network
Country [1] 309604 0
Australia
Primary sponsor type
Individual
Name
Dr Jonathon Stewart
Address
Emergency Department
Fiona Stanley Hospital
11 Robin Warren Dr, Murdoch WA 6150
Country
Australia
Secondary sponsor category [1] 310618 0
Individual
Name [1] 310618 0
Dr Adrian Goudie
Address [1] 310618 0
Emergency Department
Fiona Stanley Hospital
11 Robin Warren Dr, Murdoch WA 6150
Country [1] 310618 0
Australia

Ethics approval
Ethics application status
Submitted, not yet approved
Ethics committee name [1] 309379 0
South Metropolitan Health Service Human Research Ethics Committee
Ethics committee address [1] 309379 0
South Metropolitan Health Service Executive
Level 2, Education Building, Fiona Stanley Hospital
14 Barry Marshall Parade
MURDOCH Western Australia 6150
Ethics committee country [1] 309379 0
Australia
Date submitted for ethics approval [1] 309379 0
30/09/2021
Approval date [1] 309379 0
Ethics approval number [1] 309379 0

Summary
Brief summary
Background
Patients who arrive at the emergency department are triaged by a trained triage nurse, but then may wait hours before being seen by an emergency doctor who decides if they need to be admitted to hospital. Early identification of patients requiring admission from the emergency department may help improve hospital efficiency. Previous research has suggested that machine learning may be able to be applied to triage data to predict if a patient will be admitted to hospital, however no research has been conducted in Western Australia.

Objective
Use machine learning to predict disposition for patients presenting to the emergency department based on data available at the time of triage.

Project plan
We will develop our dataset using retrospective triage data from Western Australian Emergency Departments. We will then use a portion of this dataset to train a machine learning model to predict emergency department disposition (such as admitted to ward, intensive care, or discharged). We will test the performance of our machine learning model on the remainder of the dataset and compare the prediction of the best performing machine learning model to the predictions of emergency doctors.
Trial website
Trial related presentations / publications
Public notes

Contacts
Principal investigator
Name 113950 0
Dr Jonathon Stewart
Address 113950 0
Emergency Department
Fiona Stanley Hospital
11 Robin Warren Dr, Murdoch WA 6150
Country 113950 0
Australia
Phone 113950 0
+61 435211352
Fax 113950 0
Email 113950 0
[email protected]
Contact person for public queries
Name 113951 0
Dr Jonathon Stewart
Address 113951 0
Emergency Department
Fiona Stanley Hospital
11 Robin Warren Dr, Murdoch WA 6150
Country 113951 0
Australia
Phone 113951 0
+61 435211352
Fax 113951 0
Email 113951 0
[email protected]
Contact person for scientific queries
Name 113952 0
Dr Jonathon Stewart
Address 113952 0
Emergency Department
Fiona Stanley Hospital
11 Robin Warren Dr, Murdoch WA 6150
Country 113952 0
Australia
Phone 113952 0
+61 435211352
Fax 113952 0
Email 113952 0
[email protected]

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
Reasonable privacy concerns prohibit sharing of this data.


What supporting documents are/will be available?

No Supporting Document Provided



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
No additional documents have been identified.