Measuring evidence-based practice knowledge and skills in occupational therapy - a brief instrument

Buchanan, Helen

Measuring evidence-based practice knowledge and skills in occupational therapy - a brief instrument

Journal Article

2015-10-30

Journal Title

BMC Medical Education

Link to Journal

http://bmcmededuc.biomedcentral.com/

Publisher

Biomed Central

Publisher

University of Cape Town

Department

Department of Health and Rehabilitation Sciences

Faculty

Faculty of Health Sciences

License

http://creativecommons.org/licenses/by/4.0/

Abstract

Background: Valid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. The aims of this paper are to: 1) describe amendments to the Adapted Fresno Test of Competence in EBP (AFT), and 2) report the psychometric properties of the modified instrument when used with South African occupational therapists. Methods: The clinical utility of the AFT was evaluated for use with South African occupational therapists and modifications made. The modified AFT was used in two studies to assess its reliability and validity. In Study 1 a convenience sample of 26 occupational therapists in private practice or government-funded health facilities in a South African province were recruited to complete the modified AFT on two occasions 1 week apart. Completed questionnaires were scored independently by two raters. Inter-rater, test-retest reliability and internal consistency were determined. Study 2 was a pragmatic randomised controlled trial involving occupational therapists in four Western Cape Department of Health district municipalities (n = 58). Therapists were randomised in matched pairs to one of two educational interventions (interactive or didactic), and completed the modified AFT at baseline and 12 weeks after the intervention. An intention-to-treat analysis was performed. Data were not normally distributed, thus non-parametric statistics were used. Results: In Study 1, 21 of 26 participants completed the questionnaire twice. Test-retest (ICC = 0.95, 95 % CI = 0.88–0.98) and inter-rater reliability (Time 1: ICC = 0.995, 95 % CI = 0.99–0.998; Time 2: ICC = 0.99, 95 % CI = 0.97–0.995) were excellent for total scores. Internal consistency based on time 1 scores was satisfactory (α = 0.70). In Study 2, 28 participants received an interactive educational intervention and completed the modified AFT at baseline and 12 weeks later. Median total SAFT scores increased significantly from baseline to 12-weeks (Z = −4.078, p < 0.001) with a moderate effect size (r = 0.55). Conclusion: The modified AFT has demonstrated validity for detecting differences in EBP knowledge between two groups. It also has excellent test-retest and inter-rater reliability. The instrument is recommended for contexts where EBP is an emerging approach and time is at a premium.