Trial Review

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12613001146785
Ethics application status
Approved
Date submitted
13/10/2013
Date registered
15/10/2013
Date last updated
17/03/2016
Type of registration
Prospectively registered

Titles & IDs
Public title
Consistency of Assessor Scoring on the Direct Observation of Procedural Skills Form
Scientific title
Do assessors using the Australian and New Zealand Direct Observation of Procedural Skills form for regional anaesthesia achieve consistency and reliability?
Secondary ID [1] 283394 0
nil
Universal Trial Number (UTN)
U1111-1149-0810
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Reliability of the Direct Observation of Procedural Skills Form 290291 0
Condition category
Condition code
Anaesthesiology 290689 290689 0 0
Anaesthetics
Public Health 290700 290700 0 0
Other public health

Intervention/exposure
Study type
Observational
Patient registry
False
Target follow-up duration
Target follow-up type
Description of intervention(s) / exposure
An observational study of consistency and reliability of scoring regional anaesthesia performance on the Direct Observation of Procedural Skills form.
ANZCA Fellows will be recruited and asked to view and score 2 videos of regional anaesthesia block performance. These are scripted videos of actors roleplaying the trainee, anaesthetic assistant, and patient. The process is contained in a 1.5 hour session.
The scores are based on the DOPS form introduced in the ANZCA new curriculum commencing 2013. The results from all participants are analysed to estimate the degree of reliability by using measures of correlation.
Intervention code [1] 288119 0
Not applicable
Comparator / control treatment
No control
Control group
Uncontrolled

Outcomes
Primary outcome [1] 290703 0
Correlation of scores across all assessors completing the Direct Observation of Procedural Skills form
Timepoint [1] 290703 0
after all assessors view an instance of regional anaesthesia performance by a trainee
Secondary outcome [1] 305070 0
Correlation of scores across all assessors completing the Direct Observation of Procedural Skills form
Timepoint [1] 305070 0
after all assessors view the second instance of regional anaesthesia performance by a separate trainee

Eligibility
Key inclusion criteria
Assessors are Fellows of the Australian and New Zealand College of Anaesthetists
Minimum age
18 Years
Maximum age
No limit
Sex
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
Those declining participation in study

Study design
Purpose
Screening
Duration
Cross-sectional
Selection
Convenience sample
Timing
Prospective
Statistical methods / analysis
Measures of correlation using intra-rater correlation coefficient and Chronbach's alpha analysis

Recruitment
Recruitment status
Completed
Date of first participant enrolment
Anticipated
25/02/2014
Actual
25/02/2014
Date of last participant enrolment
Anticipated
Actual
30/04/2014
Date of last data collection
Anticipated
Actual
Sample size
Target
40
Accrual to date
Final
54
Recruitment in Australia
Recruitment state(s)
NSW
Recruitment outside Australia
Country [1] 5496 0
New Zealand
State/province [1] 5496 0
North Island

Funding & Sponsors
Funding source category [1] 288116 0
Self funded/Unfunded
Name [1] 288116 0
Country [1] 288116 0
Primary sponsor type
University
Name
Macquarie University
Address
2 Technology Place
Macquarie University
NSW 2109
Country
Australia
Secondary sponsor category [1] 286837 0
None
Name [1] 286837 0
Address [1] 286837 0
Country [1] 286837 0

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 290039 0
Macquarie University HREC (Human Sciences)
Ethics committee address [1] 290039 0
Ethics Secretariat
Research Office
Level 3, Research Hub
Building C5C East
Macquarie University
NSW 2109
Ethics committee country [1] 290039 0
Australia
Date submitted for ethics approval [1] 290039 0
28/10/2013
Approval date [1] 290039 0
21/11/2013
Ethics approval number [1] 290039 0

Summary
Brief summary
Commencing in 2013, the Australian and New Zealand College of Anaesthetists (ANZCA) implemented the Direct Observation of Procedural Skills (DOPS) form as part of the new curriculum that uses workplace based assessments (WBA) as an integral part of trainee assessment. DOPS is a generic tool using a 10-item, 9-point Likert scale scoring system for all procedural skills performed in anaesthesia.

A vital aspect of confidence in the results generated by any assessment tool is the concept of reliability, which refers to the reproducibility of a test. This implies that a broad range of assessors, when viewing the same trainee performing a regional anaesthesia block, comes to a similar result.

Components which improve reliability of an assessment tool include objective criteria for assessment, training of the assessors, direct observation at the moment of performance (rather than retrospective), and ability of the tool to discriminate between different levels of performance.

As there is no published literature on the ANZCA DOPS reliability, we are performing this study to collect the consistency of scoring. The results will generate baseline data to inform future studies, such as whether training can improve reliability.

This research project will recruit ANZCA Fellows and ask them to view 2 videos of regional anaesthesia block performance. The videos are scripted, and the trainee, anaesthetic assistant, and patient are roleplayed. Participants will view each video and score on the DOPS form. The results from all participants are analysed to estimate the degree of consistency by using measures of correlation.
Trial website
Trial related presentations / publications
Chuan A et al. Reliability of the Direct Observation of Procedural Skills assessment tool for ultrasound-guided regional anaesthesia. In print, Anaethesia and Intensive Care 2016
Public notes

Contacts
Principal investigator
Name 43650 0
Dr Alwin Chuan
Address 43650 0
The Australian School of Advanced Medicine
Building F10A, Groundfloor,
2 Technology Place
Macquarie University NSW 2109
Country 43650 0
Australia
Phone 43650 0
+61 407743668
Fax 43650 0
Email 43650 0
[email protected]
Contact person for public queries
Name 43651 0
Dr Alwin Chuan
Address 43651 0
The Australian School of Advanced Medicine
Building F10A, Groundfloor,
2 Technology Place
Macquarie University NSW 2109
Country 43651 0
Australia
Phone 43651 0
+61 407743668
Fax 43651 0
Email 43651 0
[email protected]
Contact person for scientific queries
Name 43652 0
Dr Alwin Chuan
Address 43652 0
The Australian School of Advanced Medicine
Building F10A, Groundfloor,
2 Technology Place
Macquarie University NSW 2109
Country 43652 0
Australia
Phone 43652 0
+61 407743668
Fax 43652 0
Email 43652 0
[email protected]

No information has been provided regarding IPD availability


What supporting documents are/will be available?

No Supporting Document Provided



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
No additional documents have been identified.