Research paper
Accuracy of delirium assessments in critically ill children: A prospective, observational study during routine care

https://doi.org/10.1016/j.aucc.2020.07.012Get rights and content

Abstract

Objectives

The objectives of this study was to explore the accuracy of the Cornell Assessment for Pediatric Delirium (CAP-D), Pediatric Confusion Assessment Method for the Intensive Care Unit (pCAM-ICU), and Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM-ICU) when implemented in routine care as delirium screening tools, and to assess patient characteristics and clinical variables that may affect their validity.

Design

This is a prospective observational study.

Setting

The study was conducted in a 36-bed, mixed paediatric intensive care unit (PICU) at an Australian tertiary hospital.

Patients

The study included critically ill children developmentally aged 6 months to 17 years, with a PICU length of stay >18 h.

Interventions

No interventions were provided in the study.

Measurements and main results

Patients were screened for delirium by their bedside nurse (CAP-D and pCAM-ICU/psCAM-ICU) once daily, for up to 5 d. Delirium status identified using screening instruments was compared with delirium diagnosis using the diagnostic criteria for delirium (Diagnostic and Statistical Manual of Mental Disorders, 5th Edition). In this sample, the CAP-D retained its high sensitivity (91.3%) and good specificity (75.2%), whereas the psCAM-ICU and pCAM-ICU had moderate sensitivity (58.8% and 75.0%, respectively) and excellent specificity (89.8% and 84.9%, respectively). There was moderate agreement between the CAP-D and the psCAM-ICU (κ = 0.52, p < .001) and good agreement between the CAP-D and the pCAM-ICU (κ = 0.80, p < .01).

Conclusions

Although the CAP-D, psCAM-ICU, and pCAM-ICU all appear promising in their validation studies, when implemented in routine care, their performance can be variable. The CAP-D performed well in routine clinical practice, but follow-up diagnosis is required to confirm delirium. The psCAM-ICU and pCAM-ICU both provide valuable, objective assessments of delirium in critically ill children; however, further evaluation of their implementation in routine clinical practice is needed.

Introduction

Delirium is a serious neuropsychiatric complication of critical illness characterised by acute and fluctuating disturbances in attention and awareness and changes in baseline cognition.1 Paediatric delirium is common in children admitted to the paediatric intensive care unit (PICU), occurring in 25% of patients,2 and has been associated with increased healthcare expenditure, length of hospitalisation, duration of mechanical ventilation, and risk of mortality.[3], [4], [5] Emerging research on adults has linked early prevention, identification, and management of delirium with a reduced burden on nursing workload.6 Diagnosing delirium in children is challenging, however, as it requires clinicians to consider multiple factors (e.g., developmental regression, pain, anxiety, depression, iatrogenic withdrawal, and psychosis) that may have a similar or comorbid presentation.7,8 The variation in developmental age and cognitive and language skills in children admitted to the PICU, combined with the speculation that there is no lower age limit to delirium, further complicates paediatric delirium detection. In addition, the availability of personnel, patient needs, and the feasibility of implementing time- and resource-intensive evaluation of symptoms against the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) diagnostic criteria, add to the difficulty of delirium diagnosis.9 As such, clinical decision-making relies on the use of delirium screening instruments that address all of the aforementioned challenges.

Rapid and reliable screening of paediatric delirium is important in aiding accurate identification,10 ongoing assessment and reassessment (including confirming delirium status with follow-up diagnosis),8 and prevention and management strategies (e.g., implementing multicomponent nonpharmacological interventions).11 Accurate screening instruments are also necessary for better understanding the prevalence, risk factors, and outcomes of paediatric delirium.12 In recognition that bedside nurses are uniquely placed to detect symptoms of delirium and instigate early prevention, identification, and management strategies,13,14 there has been an increase in the availability of screening instruments that have been developed and validated for use at the PICU bedside with minimal training, including the Cornell Assessment for Pediatric Delirium (CAP-D),[15], [16], [17] the Pediatric Confusion Assessment Method for the Intensive Care Unit (pCAM-ICU),18 and the Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM-ICU).19 Although all three instruments are reliable and rapid tools for detecting delirium, they have primarily been validated when used by trained users; the CAP-D was initially validated by the attending paediatric intensivist and chief resident,15 whereas the pCAM-ICU and psCAM-ICU were validated by a study team of trained paediatric anaesthesiologists, intensivists, nurse practitioners, and registered nurses.18,19 Only the CAP-D has undergone validity testing with bedside nurses undertaking the screening assessment; however, this is yet to be replicated outside the developing institution.17 To date, this has facilitated standardisation of these screening instruments but does not ensure accuracy when used for routine bedside screening.

Additionally, few studies have accounted for patient and clinical variables that may impact the accuracy of delirium screening instruments. Factors such as receiving sedation, mechanical ventilation, age, and gender have been shown to impact the accuracy of the pCAM-ICU and its modified severity scale (sspCAM-ICU),20 whereas others have also speculated that developmental delay, younger age, and motoric subtype can all complicate the accuracy of delirium detection.7,9,17 Therefore, this study aimed to (i) evaluate the accuracy of each screening instrument, when implemented in routine care, against a reference standard assessment of delirium (clinical diagnosis using the DSM-5 criteria),1 and (ii) assess patient characteristics and clinical variables that may affect their validity.

Section snippets

Study design and participants

This prospective, observational study was conducted from November 2015 to April 2017 in a 36-bed, mixed medical and surgical PICU at an Australian tertiary hospital. The study was undertaken 12 months after commencing nursing education and the adoption of routine delirium screening using the psCAM-ICU/pCAM-ICU and CAP-D. This study was granted ethics approval by the Children's Health Queensland Human Research Ethics Committee (HREC/13/QRCH/105/AM4) and the University of Queensland Human

Patient characteristics

In total, 119 patients were enrolled and 186 reference standard assessments were conducted. Five patient assessments were excluded owing to >180 minutes separating the nursing assessment from the reference standard, and 18 reference assessments were missing the corresponding index tests (see Fig. 1 for the participant flow diagram).30 Patient characteristics are described in Table 2. The majority of the patients were females (n = 64; 54.0%) and had a primary diagnosis of ‘surgical, excluding

Discussion

To the best of our knowledge, the present study is the first to evaluate the accuracy of paediatric delirium screening instruments implemented in routine care. This study found that the CAP-D was highly sensitive, with results similar to its validation studies,15,16 and had reasonable specificity. The pCAM-ICU had sensitivity scores slightly lower than those previously reported, but maintained excellent specificity.18,20 Conversely, the psCAM-ICU was not as sensitive as previously reported,19

Conclusions

To date, limited research that evaluates paediatric screening instruments outside of their developing centres exists. This prospective observational study aimed to evaluate the psychometric performance of three delirium screening instruments, the CAP-D, the psCAM-ICU, and the pCAM-ICU, as part of routine clinical practice. As a screening instrument, the CAP-D performs well, although all patients with ‘delirium-positive’ screens should be closely monitored, with follow-up diagnosis required to

Fundings

All phases of this study were supported by the University of Queensland, Australia; Griffith University, Australia; and Children's Health Queensland, Australia. R.S.P. is supported by an Australian Government Research Training Program (RTP) Scholarship (full-time base RTP Stipend Scholarship) and the Children's Hospital Foundation (Children's Hospital Foundation PhD Scholarship Top-Up Grant: RPCPHD0032017). D.A.L. was awarded a Queensland Health Nursing and Midwifery Research Fellowship as

Financial disclosure

The authors have indicated they have no financial relationships relevant to this article to disclose.

CRediT authorship contribution statement

Rebecca S. Paterson: Conceptualisation, Methodology, Formal analysis, Investigation, Writing - original draft, Funding acquisition. Justin A. Kenardy: Conceptualisation, Methodology, Writing - review & editing, Supervision. Belinda L. Dow: Conceptualisation, Methodology, Writing - review & editing, Supervision. Alexandra C. De Young: Conceptualisation, Methodology, Writing - review & editing, Supervision. Kylie Pearson: Methodology, Investigation, Writing - review & editing. Leanne M. Aitken:

Conflict of interest

The authors have indicated they have no potential conflicts of interest to disclose.

Acknowledgements

The authors gratefully acknowledge the contributions of Tara Williams, Jessica Schults, Kerry Johnson, and Rebecca Doyle for their assistance as clinical nurse researchers, Dr. Ronan McKenna and Dr. Tavey Dorofaeff for their active roles as medical champions for the study, and all PICU staff for their support for the project. The authors additionally thank Dr. Anne Bernard for her review of the statistical analyses conducted in this manuscript and Nicholas Gailer for his contribution to

References (40)

  • R.S. Paterson et al.

    Delirium in the critically ill child: assessment and sequelae

    Dev Neuropsychol

    (2017)
  • J. Harris et al.

    Clinical recommendations for pain, sedation, withdrawal and delirium assessment in critically ill infants and children: an ESPNIC position statement for healthcare professionals

    Intensive Care Med

    (2016)
  • G. Gesin et al.

    Impact of a delirium screening tool and multifaceted education on nurses' knowledge of delirium and ability to evaluate it correctly

    Am J Crit Care

    (2012)
  • T.T. Hshieh et al.

    Effectiveness of multicomponent nonpharmacological delirium interventions: a meta-analysis

    JAMA Intern Med

    (2015)
  • A. Daoud et al.

    Alberta Sepsis N. Diagnostic accuracy of delirium diagnosis in pediatric intensive care: a systematic review

    Crit Care

    (2014)
  • J.A. Dammeyer et al.

    Nurse-led change: a statewide multidisciplinary collaboration targeting intensive care unit delirium

    Crit Care Nurs Q

    (2012)
  • M.C. Balas et al.

    Critical care nurses' role in implementing the "ABCDE bundle" into practice

    Crit Care Nurse

    (2012)
  • G. Silver et al.

    Detecting pediatric delirium: development of a rapid observational assessment tool

    Intensive Care Med

    (2012)
  • G. Silver et al.

    Delirium screening anchored in child development: the Cornell assessment for pediatric delirium

    Palliat Support Care

    (2015)
  • C. Traube et al.

    Cornell Assessment of Pediatric Delirium: a valid, rapid, observational tool for screening delirium in the PICU

    Crit Care Med

    (2014)
  • Cited by (12)

    • Updates and Clinical Implications of Pediatric Delirium

      2023, Critical Care Nursing Clinics of North America
    • Inter-subject prediction of pediatric emergence delirium using feature selection and classification from spontaneous EEG signals

      2023, Biomedical Signal Processing and Control
      Citation Excerpt :

      Therefore, there are emerging needs to detect and even predict the occurrence of PED in pediatric patients so that rapid and timely treatment can be provided to reduce adverse outcomes. Currently, a number of scales have been applied for delirium detection in clinical practice, such as the 5th edition of diagnostic and statistical manual of mental disorders (gold standard for delirium detection), the richmond agitation sedation scale [4], the pediatric confusion assessment method for the intensive care unit [5], the preschool confusion assessment method for the ICU [6], the PAED scale [1], and the cornell assessment of pediatric delirium [7]. These scales are subjective delirium tests based on scoring consciousness or behavior of pediatric patients after postoperative awakening, hence not applicable to deeply sedated or unaware pediatric patients for PED prediction.

    • Postoperative opioid administration and post-traumatic stress symptoms in preschool children after cardiac surgery

      2023, Journal of Pediatric Nursing
      Citation Excerpt :

      We felt this was an acceptable limitation given the primary aims of the current study. However, we initially hoped to generate future analysis by recording the number of patients who met criteria for delirium according to the Pediatric Confusion Assessment Method for the ICU (pCAM-ICU) and the Preschool Confusion Assessment Method for the ICU (psCAM-ICU) (Paterson et al., 2021; Smith et al., 2016). Only five children in the current study met this criteria for delirium, thereby limiting the ability to generate meaningful statistical analysis.

    • A prospective cohort study of emergence delirium and its clinical correlates in a pediatric intensive care unit in North India

      2022, Asian Journal of Psychiatry
      Citation Excerpt :

      A Colombian study which utilized p-CAM-ICU, reported that the hypoactive subtype (50%) was common type of delirium followed by hyperactive (26.9%) and mixed subtype (19.2%) (Cano Londoño et al., 2018). Similar findings have been reported in some other studies which have utilized the p-CAM-ICU scale to evaluate pediatric delirium (Ricardo Ramirez et al., 2019; Paterson et al., 2021). However, a previous Indian study had reported hyperactive delirium to be most common subtype (53%) in children and adolescents (Grover et al., 2014).

    View all citing articles on Scopus

    This study was conducted at the Queensland Children’s Hospital, Brisbane, Queensland, Australia.

    View full text