Citation

Gittelman MA, Kincaid M, Denny S, Wervey Arnold M, Fitzgerald M, Carle AC, Mara CA. J. Trauma Acute Care Surg. 2016; 81(4 Suppl 1 Forging New Frontiers: The 20th Annual Conference of the Injury Free): S8-S13.

Affiliation

Professor, Division of Emergency Medicine Cincinnati Children's Hospital Medical Center, Cincinnati, OH Mike.gittelman@cchmc.org Graduate student/research assistant Brown School at Washington University in St. Louis kincaid.madeline@gmail.com Associate Clinical Professor, Division of Emergency Medicine Nationwide Children's Hospital, Columbus, OH Sarah.Denny@nationwidechildrens.org Executive Director, Ohio Chapter, American Academy of Pediatrics Ohio Chapter of the AAP, Worthington, OH marnold@ohioaap.org Assistant Professor, Division of Emergency Medicine Cincinnati Children's Hospital Medical Center, Cincinnati, OH Mike.Fitzgerald@cchmc.org Associate Professor, James M Anderson Ctr for Health Systems Excellence Cincinnati Children's Hospital Medical Center, Cincinnati, OH Adam.carle.cchmc@gmail.com Research Associate, Patient-Reported Outcomes James M Anderson Ctr for Health Systems Excellence Cincinnati Children's Hospital Medical Center, Cincinnati, OH Constance.Mara@cchmc.org.

Copyright

(Copyright © 2016, Lippincott Williams and Wilkins)

DOI

10.1097/TA.0000000000001182

PMID

27488487

Abstract

BACKGROUND: A standardized injury prevention (IP) screening tool can identify family risks and allow pediatricians to address behaviors. To assess behavior changes on later screens, the tool must be reliable for an individual and ideally between household members. Little research has examined the reliability of safety screening tool questions. This study utilized test-retest reliability of parent responses on an existing IP questionnaire and also compared responses between household parents.

METHODS: Investigators recruited parents of children 0-1 year during admission to a tertiary care children's hospital. When both parents were present, one was chosen as the "primary" respondent. Primary respondents completed the 30 question IP screening tool following consent and they were re-screened approximately 4 hours later to test individual reliability. The 'second' parent, when present, only completed the tool once. All participants received a 10 dollar gift card. Cohen's Kappa was used to estimate test-retest reliability and inter-rater agreement. Standard test-retest criteria consider Kappa values: 0.0-0.40 poor-fair, 0.41-0.60 moderate, 0.61 to 0.80 substantial, and 0.81-1.00 as almost perfect reliability.

RESULTS: 105 families participated, with 5 lost to follow-up. 32 (30.5%) parent dyads completed the tool. Primary respondents were generally mothers (88%) and Caucasian (72%). Test-retest of the primary respondents showed their responses to be almost perfect; average 0.82 (SD=0.13, range 0.49-1.00). 17 questions had almost perfect test-retest reliability and 11 had substantial reliability. However, inter-rater agreement between household members for 12 objective questions showed little agreement between responses; inter-rater agreement averaged 0. 35 (SD=0.34, range -0.19-1.00). One question had almost perfect inter-rater agreement and 2 had substantial inter-rater agreement.

CONCLUSIONS: The IP screening tool used by a single individual had excellent test-retest reliability for nearly all questions. However, when a reporter changes from pre to post-intervention, differences may reflect poor reliability or different subjective experiences rather than true change. LEVEL OF EVIDENCE: Level II, Prognostic and Epidemiological.

Language: en