March 27, 2006

Evidence-based Practice

Filed under: Uncategorized — mental1 @ 6:19 pm

Ovid: HAMILTON: J Am Acad Child Adolesc Psychiatry, Volume 45(3).March 2006.364-370

Evidence-Based Practice for Outpatient Clinical Teams

Dr. Hamilton is with The Permanente Medical Group of California, Inc., Sacramento, CA, and Kellogg College, University of Oxford.
Accepted September 1, 2005.
Robin Weersing, Ph.D., John Lyons, Ph.D., and Michael Jellinek, M.D., were helpful in conceptualizing and drafting these ideas.
Correspondence to Dr. John D. Hamilton, 2025 Morse Avenue, Sacramento, CA. 95825; e-mail:
Disclosure: The author has no financial relationships to disclose.

This column focuses on evidence-based practice (EBP) within multidisciplinary outpatient settings, but, first, some definitions. Besides EBP (Burns and Hoagwood, 2005; Guyatt and Rennie, 2002), there are also evidence-based medicine (EBM; March et al., 2005), evidence-based service (EBS; Chorpita et al., 2002), and evidence-based treatment (EBT; Kazdin, 2005). The term empirically supported treatment (EST) is often used interchangeably with EBT (Chorpita, 2003).

The common denominator in these terms is a commitment to using the evidence of empirical results to guide care in a clinically sensitive way. To oversimplify, there are three groups. The first group (EBM) is associated with medication issues and child and adolescent psychiatrists. The second (EBT) is associated with psychologists, psychosocial treatments, and the American Psychological Association. The third group (EBS) is associated with systems trying to make better use of empirical results to improve outcomes. In this column, when we refer to EBP, we are using it as an umbrella term to include EBM, EBT, and EBS, and valuing patient preference and clinical expertise as well. While accommodating both medication and psychosocial interventions, EBP is neutral in choosing between them. This is a major advantage in choosing an approach acceptable to a multidisciplinary team.

EBM focuses on the practitioner, EBT focuses on the therapy, and EBS focuses on a large system. Yet it is often the multidisciplinary team serving outpatients that is the natural unit of organization in delivering services. Consider, therefore, two central questions.

First, how does a child and adolescent outpatient team function if it is committed to a sophisticated EBP that values clinical expertise and patient values? Which processes in EBP differ from functioning “as usual” and in what way?

Second, why bother? Without definitive evidence that efforts to adopt EBPs in real-world settings will generate better outcomes than proceeding as usual, this is an important question without a definitive answer (Dulcan, 2005; Weisz et al., 2004). Yet there are reasons to bother. First, real-world practice settings have minimal to no effect compared with a control group for child and adolescent psychotherapy (Bickman et al., 1999; Weisz and Jensen, 1999). Second, demonstration projects to improve care by simply offering more care have also shown minimal effect (Bickman et al., 1999). Third, in Hawaii’s system of care for youths, the rate of improvement (reduced symptoms and improved functioning) as reported by teacher, parent, and clinician all improved significantly between 2002 and 2004, the time period following the introduction of EBP (Daleiden, 2004; Daleiden and Chorpita, 2005). These initial reports indicate that systemwide efforts to introduce EBPs have made a difference in Hawaii.

Table 1 lists a chain of clinical processes fundamental to EBP and highlights differences between the EBP approach and usual practice in defining the clinical population, in choosing an intervention, and in evaluating its effects relative to a comparison or control group. This order is the familiar PICO format derived from epidemiology: population, intervention (or exposure), control (or comparison), and outcome. Implementation of such an ambitious list of processes has only barely begun at my own workplace.

[Help with image viewing]
[Email Jumpstart To Image] TABLE 1 Functions and Processes: Treatment as Usual Versus EBP
Diagnostic Assessment of the Youth’s Psychopathology and Functioning Is Sufficient to Hold Up in the Court of Scientific Opinion Yet Balanced With Clinical Sensitivity

At least one component of the diagnostic assessment needs to generate results that even a skeptic would acknowledge define subsets of youths represented in the psychiatric treatment literature. A diagnostic process so arbitrary that its results are truly idiosyncratic to the agency and its interviewers is a disadvantage. For example, many agencies have their own intake forms, and clinicians routinely assign youths DSM-IV-TR diagnoses based on a single unstructured interview, a process with significant limitations (Jensen and Weisz, 2002): it becomes more difficult to link treatment planning to the literature and to compare results with known benchmarks. DSM-IV-TR, however, does not have a special reserved place within EBP, and this is an advantage on a multidisciplinary team because DSM is associated with a single professional association. Continuous scales empirically derived using factor analysis such as the Achenbach System of Empirically Based Assessment (ASEBA), for example, can compete with DSM as legitimate broad measures of psychopathology and functioning.

A clinical interview and formulation can complement a feasible, broad-based instrument with known reliability and concurrent and predictive validity (Jensen, 2005; Jellinek and McDermott, 2004). First, inventories of broad arrays of symptoms free up the interviewer’s limited time. He or she can use clinical expertise to focus on whatever appears most significant in the encounter with the youth, knowing that the inventory, delivered as a questionnaire or via computer, will search broadly for symptoms. The interviewer is free to build rapport and refine initial information gathered from the inventory as well as address discrepancies among reporters (parent, youth, teacher). A useful analogy is how a pediatrician connects to an asthmatic child with a personal interview, but uses a peak flow meter as a measure of functioning.

A structured, broad assessment and a clinical interview complement each other because only the latter can provide the highly case-specific information often useful in clinical treatment, as well as a context for understanding the results of the structured assessment. For example, the divorced parents of Evan, an adolescent with severe attention-deficit/hyperactivity disorder (ADHD) and oppositional defiant disorder on the parent-based Diagnostic Interview Schedule for Children, gave a detailed history of their current multiple, intense, and unpleasant arguments related to their marriage’s dissolution. This information provided a current context for Evan’s oppositional behavior as rated on the structured instrument.

Both broadbased and highly specific diagnostic instruments complement the therapist’s efforts to establish a strong therapeutic alliance with a clinical interview. There indeed may be an optimal mix in combining well-defined instruments with more spontaneous information-gathering approaches such as a play therapy session with a school-age child and a clinical interview with an adolescent focusing on the developmental tasks and dynamics of adolescence. Values, ideals, and aspirations, for example, as well as models for identification, sexual experiences, and strengths and accomplishments may all be more adequately discussed in a clinical interview.

Although defined instruments in a time-pressured setting may invite overreliance on their results, pushing an assembly-line atmosphere and undervaluing clinical sensitivity, wisely used technological innovations like the computerized Diagnostic Interview Schedule for Children (Version 4.0) and Achenbach’s checklist system can capture data efficiently and liberate clinicians to develop a solid alliance with youths and parents. The goal is an evaluation summary that includes both a rich and sophisticated formulation, a good alliance, and results from established diagnostic instruments.
Diagnostic Assessment Includes Instruments Able to Function as Outcome Measures

The heart of EBP is empirical results, both from the site itself and from the literature. Hence, the need to begin collecting data at intake with feasible instruments that can be used to measure outcome. For example, the Screen for Child Anxiety Related Emotional Disorders, or SCARED, is helpful in defining the level of anxiety symptoms in clinical youths (Birmaher et al., 1999). It is feasible because it is a brief self-report measure for parents and youths in the public domain. Another feasible instrument in tracking outcome results for ADHD is the SNAP-ADHD rating scale, composed of 18 items that closely parallel the DSM-IV-TR listing of ADHD symptoms (Gaub and Carlson, 1997; Swanson, 1992; Swanson et al., 1999). It also has been widely used in treatment studies (e.g., Wigal et al., 2004). Measures such as the SCARED and the SNAP-ADHD scale, administered at intake, can help in developing outcome databases for individual youths, for individual clinics, or for a whole system. Many other feasible instruments sensitive to change and in the public domain exist for specific indications, including the Child Stress Disorders Checklist for observer-report of acute stress disorder and posttraumatic stress disorder symptoms (Saxe et al., 2003).
Pyramid of Evidence Is Central to the Team’s Functioning

In the team committed to EBP, practitioners are familiar with the pyramid of evidence (Guyatt et al., 2002) and use it to offer feasible-to-implement treatments showing the most benefit and the least harm in the most credible studies. In the State of Hawaii’s mental health division, for example, a unifying task for the clinical staff is to biannually review the evidence base for treating the most common disorders. They post their conclusions on the Internet as an agency-wide “menu”. Psychosocial treatments are rated according to the American Psychological Association’s rating system, and medications are given a letter grade based on similar principles (Chorpita and Viesselman, 2005).

The pyramid of evidence is a neutral and useful referee in debates centered on medication versus psychosocial interventions, debates easily fueled by the irrational. The pyramid of evidence then becomes a central unifying element of the team’s functioning, linking providers, youths, and their parents to both psychosocial and medication treatment options. Active discussion of the best evidence with families decreases the gap between what clinicians know about treatment and what youths and parents know, an information gap that handicaps consumers (Lyons, 2004).

Finally, the pyramid needs to be built on a human scale rather than as a marble monument. The clinician’s own expertise and patient preferences temper and soften the pyramid without dulling its crisp edges. An overly precise approach discarding subtle, astute clinical observations becomes artificial and only discourages EBP.
Issues Related to Efficacy Versus Effectiveness Are Taken Seriously

A psychotherapy shown to be efficacious in a controlled study in a delineated population is often referred to as an EBT, especially in the psychological literature. The problems exporting an EBT from university centers using specialized clinicians with small caseloads to real-world settings are well known. How much can real-world psychotherapists adapt an EBT to better suit their own site without destroying key essential components and affecting outcome? How much supervision or live monitoring is necessary to ensure adequate treatment fidelity? Because clinicians control EBP implementation for their clients, what motivational and learning strategies go into changing individual therapists’ behaviors? These serious questions are unresolved at present with a large impact on the “bottom line” of improved youth outcomes (Riemer et al., 2005). A team committed to EBP takes interest in these issues.

Exporting medication therapies from highly specialized research teams into real-world settings is also complex. In the Multimodal Treatment of ADHD Study, for example, 56% of the youths with ADHD receiving a carefully crafted intervention of psychostimulants achieved a good outcome (defined as SNAP-IVPT


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

%d bloggers like this: