C 2006), pp. 115–129 Journal of Child and Family Studies, Vol. 15, No. 1, February 2006 (
DOI: 10.1007/s10826-005-9008-7

The Role of Fidelity and Feedback in the Wraparound Approach Benjamin M. Ogles, Ph.D.,1,7 David Carlston, Ph.D.,2 Derek Hatfield, M.S.,3 Gregorio Melendez, Ph.D.,4 Kathy Dowell, Ph.D.,5 and Scott A. Fields, Ph.D.6 Published online: 30 December 2005

Wraparound approaches are being implemented with children in many mental health systems around the country. Evidence for the effectiveness of the wraparound approach, however, is limited. In addition, the degree to which wraparound interventions adhere to the principles of wraparound has rarely been assessed. We examined the influence of adherence to wraparound principles and outcome feedback within the wraparound approach. Children participating in family team meetings were enrolled in a feedback or no feedback condition. Teams receiving feedback were given a brief report regarding outcome progress four times over a three-month period. In addition, adherence to wraparound principles was assessed in the initial team meeting and examined in relationship to outcome at three months and nine months. Although youth in both feedback and nonfeedback groups improved with intervention, there were few differences between the groups based on outcome feedback. Similarly, adherence was uniformly high and did not influence the outcome for individual cases. Although the wraparound approach was helpful for youth in our sample, outcome feedback and adherence to wraparound principles had limited influence on these effects. KEY WORDS: wraparound; feedback; adherence; outcome; mental health.

1 Professor and Chair, Department of Psychology, Ohio University, Athens, OH. 2 Assistant Professor, Department of Psychology, Midwestern State University, Wichita Falls, TX. 3 Doctoral Candidate, Department of Psychology, Ohio University, Athens, OH. 4 MST Consultant, MST Services, Mt. Pleasant, SC. 5 Assistant Professor, University of Minnesota at Duluth, Duluth, MN. 6 Assistant Professor, Director of Behavioral Sciences, Department of Family Medicine, West Virginia

University School of Medicine, Charleston Division, Charleston, WV. should be directed to Benjamin M. Ogles, Department of Psychology, 200 Porter Hall, Ohio University, Athens, OH 45701; e-mail: [email protected].

7 Correspondence

115 C 2006 Springer Science+Business Media, Inc. 1062-1024/06/0200-0115/1


116

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields

The recent shift toward community based interventions for youth is partially the result of the observation that children who are removed from their home environment for treatment and then returned home are unlikely to benefit from treatment and are often placed in unnecessarily restrictive settings (Stroul & Friedman, 1986). Home-based interventions are thought to be more cost effective and intuitively better for the child’s quality of life. Since the child remains at home, any change that occurs has a better likelihood of being maintained (Burchard & Clark, 1990). As a result, home-based interventions are being employed more frequently. One broad-based approach to intervening with families in the community is called wraparound services. This approach uses input from multiple stakeholders who serve on a family team in order to develop a highly individualized service plan for the child. The wraparound process is viewed as a potential alternative to out-ofhome placement for children with serious emotional disturbances because a unique package of supports and services can be tailored to each situation (Eber & Nelson, 1997). The theory behind the wraparound approach suggests that individualized services that are family-centered and child-focused are more appropriate for a child’s specific needs and more likely to maintain the youth in the community than categorical approaches to mental health care. The wraparound approach has grown in popularity over the past decade and is now considered an integral component of the system of care for children (Burns, Schoenwald, Burchard, Faw, & Santos, 2000; Stroul & Friedman, 1986). If wraparound is to become more than a trend in the mental health treatment of children, however, outcome studies must demonstrate its effectiveness (VanDenBerg & Grealish, 1996). Similarly, data must be collected to demonstrate the superiority of wraparound to other approaches. Finally, the wraparound process is used inconsistently (Rosenblatt, 1996). As a result, studies are needed to investigate the implications of adherence to the wraparound philosophy or to investigate the various components of an ill-defined intervention. Several outcome studies supported the wraparound approach as an alternative to categorical approaches of mental health treatment for youth (Bruns, Burchard, & Yoe, 1995; Clark, Lee, Prange, & McDonald, 1996; Eber, Osuch, & Redditt, 1996; Toffalo, 2000). In general, outcomes suggest that children treated from the wraparound approach demonstrate improved behavioral and emotional functioning on various measures. Another potentially attractive finding has been the savings in costs for services that are attributed to the wraparound approach (Brown & Hill, 1996). One more recent study, however, found that wraparound services produced similar outcomes to treatment as usual, but provided more services at a greater cost (Bickman, Smith, Lambert, & Andrade, 2003). Wraparound services vary from child to child with some commonalities. The following elements provide an important philosophical base for wraparound, and must be adhered to in order to maximize the likelihood of success for children with

Feedback within Wraparound

117

serious emotional disturbances (VanDenBerg & Grealish, 1996): (a) wraparound services must be based in the community; (b) services and supports must be individualized to meet the needs of the children and families, and not the priorities of the service systems; (c) parents must be included at every level of development in the process; (d) agencies must have access to non-categorized, flexible funds; (e) services must be unconditional, in that changes in the needs of the child and family should be accompanied by changes in services; (f) the process must be implemented on an inter-agency basis and must be owned by the community; (g) outcomes of every service must be measured; (h) services must be built on the unique cultural values, cultural strengths, and the racial make-up of children and their families. Although the research base lends support to this innovative approach, due to the variability in wraparound approaches, it is often difficult to know if researchers are evaluating the same treatment models (Myaard, 2000; Rosenblatt, 1996). One way to evaluate various wraparound interventions would be to compare them to existing criteria such as those listed above. With this in mind, an instrument to measure the implementation of the wraparound process, the Wraparound Observation Form was developed to compare wraparound practices with the theoretically important components (WOF; Epstein et al., 1998). Our study investigates the relationship of adherence to wraparound principles (using the WOF) to child and family outcomes and satisfaction. It was expected that better adherence would result in increased family satisfaction with services and better outcomes. One of the core components of the wraparound approach is the focus on specified outcomes and the inclusion of outcome assessment (VanDenBurg & Grealish, 1996). However, little research has investigated the impact of outcome feedback and no study has investigated the importance of tracking outcome and providing feedback within the wraparound approach. Researchers have long studied the effects of behavioral monitoring on the successful achievement of behavioral goals (Kanfer & Gaelick-Buys, 1991). Specifically, therapeutic goals have been a variable of interest, and it is clear that a heightened awareness of goals and progress increases the accountability of those involved in the therapeutic process (Hart, 1978). Similarly, the increased focus on outcome monitoring and management has resulted in recent studies that investigate the influence of feedback on patient progress in outpatient therapy with adults (e.g., Lambert, 2001; Lueger, 1998). While self-monitoring and feedback have been studied in therapy settings, no study has examined the implications of tracking outcome within the wraparound process. Our study was designed to provide information about monitoring outcome within the family team approach. Feedback was provided to team members (including the parent) regarding the progress of the child. It was hypothesized that increased feedback to team members would be met with a heightened sense of accountability, more timely interventions, and better outcomes of services.

118

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields Table I. Initial scores on the Ohio Scales for parents and youth Feedback Measure Parent Problem Severity Parent Functioning Parent Hopefulness Youth Hopefulness Youth Problem Severity Youth Functioning

No Feedback

Mean

SD

Mean

SD

26.15 47.97 10.97 9.50 26.47 49.14

15.36 17.76 4.44 4.25 18.68 24.89

26.06 48.77 11.42 9.11 24.87 58.84

15.76 16.65 5.06 4.32 13.96 12.26

No significant differences between feedback groups.

The purpose of our study was to investigate the influence of treatment fidelity and outcome feedback on outcomes for children and families receiving wraparound services. Specific hypotheses included: (1) Youth receiving wraparound services instituted through a family team would have decreased problems and improved functioning over time; (2) Family team meetings which had greater adherence to wraparound principles would result in better outcomes for youth and increased family satisfaction with services; (3) Teams that received outcome feedback about the progress of the youth would be more satisfied with interventions and would be better informed regarding the case; (4) Families that received feedback regarding their progress would have youth who improved more than families that did not receive feedback.

METHOD An experimental design with repeated measures was used to compare the outcomes of clients assigned to wraparound services plus feedback versus wraparound services without feedback.

Participants A total of 72 families attended initial wraparound team meetings and agreed to participate in the study. The children in the sample consisted of 38 males and 34 females who were an average age of 13.31 (sd = 3.04) years old. Thirtyseven youth were assigned to the feedback condition and 35 to the non-feedback condition. Initial analyses indicated that the two feedback groups did not differ at the time of the first team meeting and that participants, on average, had significant levels of problems as indicated by scores on the Ohio Scales problem severity, Ohio Scales functioning, Vanderbilt Functioning Index, and Target Complaints (Ogles et al., 2002; see Table I). By the three-month interval, 12 of the participants

Feedback within Wraparound

119

had dropped out leaving 60 participants for the main analyses. Drop-out rates at 3 months were not significantly different between the two feedback groups.

Instruments To assess youth outcomes, several measures with established psychometric properties were administered: The Ohio Scales (Ogles, Melendez, Lunnen, & Davis, 2001), Target Complaints (Battle et al., 1966), and achievement of individually defined goals using Goal Attainment Scaling (GAS; Kiresuk, Smith, & Cardillo, 1994). Family outcomes were assessed using the Family Adaptability and Cohesion Scale (FACES III; Olson, Portner, & Lavee, 1985). Members of the team (including parents) were queried regarding their satisfaction with the outcomes of intervention and the degree to which they felt informed about the case using items developed by the research team. Adherence to the wraparound principles was assessed using the Wraparound Observation Form (Epstein et al., 1998).

Procedures Parents and children scheduled for a family team meeting associated with a given agency were asked by a Parent Research Assistant (PRA) if they would be willing to participate in the study just prior to their initial team meeting. Families that agreed to participate in the study met briefly with a PRA prior to the meeting to complete consent forms. The PRA then attended and rated the family team meeting using the wraparound observation form. The PRA was not a participant on the team, but rather an independent observer of the meeting. Following the meeting, the PRA interviewed the child and parent using the parent interview form. At this time, the families completed the Parent and Youth rated Ohio Scales, the Target Complaints, and the FACES-III along with identifying three primary goals. For youth under age 12, only the parent completed the measures. All families were then contacted 2 weeks, 4 weeks, 8 weeks, 12 weeks, and 9 months following the initial team meeting to complete a set of specified forms depending on the time point. Both parents and children who completed the forms were compensated for their participation. The first two youth were deliberately assigned to the non-feedback group in order to work out the details of the assessment process. From that point on, youth were assigned to the feedback condition based on a matched randomization procedure where an equal number of feedback and non-feedback slips of paper were placed in a hat and selected one at a time to make the group assignment for families that enrolled in the study. A total of 37 were assigned to feedback with 35 in the non-feedback condition with more than four slips of paper remaining in the hat.

120

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields Table II. Initial status for youth in the feedback and no feedback groups Feedback Measure Age Grade Suspensions (last year) Arrests (last year) AWOL overnight (last year) Hospitalizations (last year) Days in out of home placement (last year) Hours of Counseling (last year)

No Feedback

Mean

SD

Mean

SD

13.45 7.11 .64 .27 1.39 .21 27.18 18.91

2.68 2.67 1.49 .67 4.5 .54 63.03 18.11

13.17 6.63 .59 .45 .06 .41 19.09 29.63

3.41 3.51 1.27 .97 .25 1.78 49.11 73.84

No significant differences between feedback groups.

For youth assigned to the feedback condition, a feedback report was faxed to the PRA after each of 4 data collection points (2 week, 4 week, 8 week, and 12 week). The PRA distributed the report to the team including the parent(s). Participants in the non-feedback group received no information back concerning the completed measures.

RESULTS A total of 72 youth were enrolled in the study. Thirty-seven were assigned to the feedback condition and 35 to the non-feedback condition. Initial scores for the two groups on the Ohio Scales Problem Severity, Functioning, and Hopefulness Scales as rated by the youth and parent are displayed in Table I. As can be seen, the two feedback groups did not differ at the time of the initial meeting. In addition, Tables II and III display the means and standard deviations for the groups on other measures or variables of interest. Again, there were no significant differences between the feedback groups. This suggests that the matched randomization procedure was successful at creating two approximately equivalent groups. In terms of the initial status of the youth, average scores on the Ohio Scales indicated that the average youth was exhibiting problems within the clinical range and typical of youth receiving mental health services (Ogles, Melendez, Lunnen, & Davis, 2001). Similarly, many of the youth had a history of previous hospitalization, suspension in school, arrest, placement in a foster home, group home or other placement, or leaving home without permission over night. Only 11 of the youth had not participated in counseling in the year prior to the team meeting. Scores on the Vanderbilt Functioning Index also indicated that youth had a history (past six months) of involvement in an average

Feedback within Wraparound

121

Table III. Initial scores on other measures for youth in the feedback and no feedback groups Feedback Measure Wraparound Observation Form FACES – III Total FACES – III Adaptability FACES – III Cohesion Vanderbilt Functioning Index Target Complaints – Problem 1 Target Complaints – Problem 2 Target Complaints – Problem 3

No Feedback

Mean

SD

Mean

SD

28.81 59.63 25.67 33.97 7.70 3.27 3.33 3.36

3.30 11.20 5.60 7.04 3.88 .98 .82 .65

28.50 58.80 24.74 34.06 7.12 3.45 3.52 3.50

4.15 12.18 7.72 9.23 3.30 .91 .67 .84

No significant differences between feedback groups.

of 7 of 24 problem behaviors (e.g., fights, illegal behaviors) or critical events (e.g., self-harm attempt). Parent ratings of the three target complaints were completed using a fivepoint scale (0 = absent, 1 = trivial, 2 = mild, 3 = moderate, and 4 = severe). As can be seen in Table III, the average rating for all three complaints at the time of the initial meeting fell between moderate and severe. In fact, 53% of the parents rated the youth as severe on the primary target problem with an additional 26% rating the youth at the moderate level on the first problem. Similar proportions were evident for the second and third target complaints. Scores on the wraparound observation form indicated that the Parent Research Assistant viewed the teams as being consistent with an average of 28.65 (sd = 3.72) of the 34 wraparound principle statements. This is an indication of a high degree of consistency with wraparound principles or fidelity to the model within the initial team meeting and as operationalized by the wraparound observation form.

Outcome Evaluation While the average child participant evidenced problem behavior in the clinical range on the problem severity scale, a subset of participants (n = 15 of the 60 completers) had initial problem severity scores that were quite low (below 13). Because these participants had floor effects preventing them from making large decreases in problem severity, we divided the sample into two groups for further analyses (clinical level of problems at intake and non-clinical level of problems at intake). In order to evaluate the effectiveness of wraparound services and feedback on symptom change over time, a 2 (intake, post-treatment) × 2 (feedback, no feedback) × 2 (initial significant problems, initial non-significant problems) was

122

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields Table IV. Parent ratings of Youth Problem Severity, Functioning, Target Complaints and Family Functioning at the Initial Team Meeting and 3 Months Later for youth with Clinical and Sub-clinical Levels of Problems at the Initial Team Meeting Clinical problems Variable Problem Severity (Time 1) Problem Severity (3 months) Functioning (Time 1) Functioning (3 months) FACES Total (Time 1) FACES Total (3 Months) Target Complaints (Time 1) Target Complaints (3 months)

Sub-clinical problems

Mean

SD

Mean

SD

37.40 26.67 39.59 49.81 56.25 56.84 3.54 2.85

1.41 2.46 2.81 2.55 2.39 2.63 .11 .21

11.04 15.58 60.24 59.71 59.99 55.92 3.39 2.53

1.53 2.68 3.15 2.86 2.78 3.06 .11 .22

conducted using the Ohio Scales Problem Severity totals, Ohio Scales Functioning totals, the average target complaints, and the Family Adaptability and Cohesion total scores. These analyses were conducted to evaluate change at three months (when feedback ended) and nine months following intake (a six month follow-up after feedback ended). N’s for the analyses vary somewhat depending on missing data.

Change at Three Months When evaluating change using the parent rated Ohio Scales Problem Severity Scale, a significant interaction was obtained between initial clinical status and symptom change, F(1,56) = 13.32, p = .001). Follow-up analyses indicate that children who evidenced clinical levels of problems at intake showed significant reductions in symptom severity, t(45) = 3.16, p < .01, while children who evidenced problems in the normal range at intake showed a significant increase in symptom severity, t(15) = −2.55, p = .05. No other main effects or interactions among time, feedback, or initial severity groupings were significant. Means and standard deviations for the clinical and sub-clinical groups on problem severity and each of the following variables at 3 months are displayed in Table IV. When evaluating change as reported on the Ohio Scales Functioning Scale, a significant three-way interaction was identified (feedback X initial clinical status X change in functioning), F(1,37) = 19.21, p = .01. Follow-up analyses indicated that children who evidenced clinical levels of problems at intake within the both the feedback and no-feedback groups showed significant gains in functioning, t(30) = −4.35, p = .001. However, children who did not evidence clinical levels of problems at intake did not improve, t(9) = 1.61, p = .14.

Feedback within Wraparound

123

When evaluating change in target complaints (average for the three complaints at each time), wraparound services were found to produce significant decreases in reported levels of target complaints over time, F(1,47) = 18.95, p < .001. No other significant interactions or main effects were obtained among initial clinical status, feedback, and time. Finally, when evaluating change in family functioning using the FACES-III, wraparound services were found to produce no significant increases in reported levels of family functioning at three months. Parental reports on the Goal Attainment Scale indicate the degree to which services produced expected change. Parental ratings were coded into two levels – less than anticipated change and anticipated/greater than anticipated change. Feedback was found to be significantly associated with parental perceptions of change at three months (X2 (42) = 5.87, p = .015). Specifically, a greater proportion (69% versus 31%) of those parents who were in the feedback group felt that treatment produced expected or greater than expected levels of change. Change at Nine Months When evaluating change using the Ohio Scales Problem Severity totals, a significant two-way interaction (time X clinical status) was observed, F(1,32) = 4.36, p < .05). Follow-up analyses were not significant (probably as a result of low power due to decreased numbers of participants), but mimicked findings for the 3-month data. Youth who evidenced clinical levels of problems at intake showed reductions in symptom severity, while children who evidenced problems in the normal range at intake showed increases in symptom severity. No other main effects or interactions among time, feedback, or initial severity groupings were significant. When evaluating change using the Ohio Scales Functioning totals, none of the interactions or main effects were significant. This indicates that children showing initial improvement in functioning at 3 months did not maintain those improvements. This finding must be tempered by the relatively large number of dropouts between 3 and 9 months. When evaluating change in target complaints at nine months, wraparound services produced significant decreases in reported levels of target complaints over time, F(1,30) = 29.66, p < .001). No other main effects or interactions were significant. When evaluating change in family functioning, a significant interaction between time and feedback was observed, F(1,28) = 4.34, p < .05). Follow-up analyses indicated that parental perceptions of family functioning decreased in the 6 months following participation in the feedback group, t(16) = 2.60, p < .05. In contrast, parental perceptions of family functioning did not

124

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields

change for families that were in the non-feedback group. Importantly, levels of family functioning at 9-months for both feedback and non-feedback groups were not significantly different, x = 53 and x = 51 for the non-feedback and feedback groups respectively. Finally, feedback was not significantly related to parental perceptions of goal attainment at nine months. The majority of parents, however, did view the treatment as having achieved or exceeded expectations for goals identified at the initial meeting (57%).

Wraparound Adherence In order to evaluate the impact of wraparound adherence on treatment gains, wraparound adherence was correlated with each measure of change – total problem severity, functioning, target complaints, family functioning and goal attainment for both three and nine months. Given that a variable set of items on the Wraparound Adherence Form were applicable to individual dyads, the percentage of applicable items endorsed was used to represent adherence. Goal attainment at 3 months was significantly correlated with adherence, r(43) = .43, p < .05. This relationship was not maintained at 9 months. In addition, adherence was negatively correlated with family functioning at follow-up, r(35) = −.35, p < .05. None of the remaining 8 correlations (4 at 3 months and 4 at follow-up) were significant. Therefore, it would seem to indicate that the degree of adherence to wraparound principles is unrelated or even negatively related to treatment outcome. However, these findings may be affected by the significantly restricted range of wraparound adherence ratings. Indeed, the parent research assistant’s (PRA) ratings averaged 96% (sd = .04). This indicates a very high degree of consistency with the relevant principles listed on the Wrap Around Observation Form and provides little variability that might be correlated with measures of change.

Parent and Team Member Perceptions Parents rated 5 items on a 1 (not at all) to 5 (very much) scale regarding their perceptions of outcome (2 items) and the team’s awareness of the child’s progress (3 items). Parents rated these items at 1 month, 2 months, and 3 months following the initial team meeting. When averaging across all times, parents in the feedback condition (x = 3.09, sd = .81) felt their children made more progress than parents in the non-feedback group (x = 2.82, sd = .56), F(1163) = 5.54, p < .05. Ratings of team involvement, however, were not significantly different for feedback and non-feedback groups respectively, x = 4.28, sd = .92; x = 4.27, sd = .86.

Feedback within Wraparound

125

Team members rated 10 items on a 1 (not at all) to 5 (very much) scale regarding the degree to which they felt informed about the case (4 items), their perception of outcome (4 items), and the degree to which they participated in the case (2 items). Team members for both family teams that received feedback and did not receive feedback felt well-informed, x = 4.16, sd = .93; x = 4.31 sd = .65, respectively. Similarly, both feedback and non-feedback teams viewed their participation as significant, x = 4.57, sd = .93; x = 4.57, sd = .87. In addition, the average outcome rating of team members was at the midpoint and not different between the feedback groups, respectively, x = 3.10, sd = .91; x = 3.39, sd = .86. In short, team member’s perspectives of amount of information available, team member participation, and outcome of cases did not differ between feedback and non-feedback groups. DISCUSSION While wraparound services are becoming more popular as an individualized and coordinated alternative to categorical and separated services, few studies have established the benefits of these services. Similarly, few studies have examined the relationship between wraparound adherence and outcome and no studies have examined the relationship of feedback to outcome. We set out to investigate the relationships among these variables — adherence, feedback, and outcome – within the wraparound approach. Our first hypothesis suggested that youth receiving wraparound services would have decreased problems and improved functioning. In fact, wraparound services delivered to youth who had clinical levels of problems at the initial team meeting did produce positive changes in problem severity, functioning, target complaints, and goal attainment over a three month time period. Analyses of nine-month data also indicated that changes were maintained in problem severity, target complaints, and goal attainment. Changes in parent rated youth functioning were not maintained at 9 months. Parental perceptions of family functioning did not change over the initial 3 months of services, but for those who continued in the study, parental perceptions of family functioning decreased significantly for youth in the feedback group. Our findings are similar to those presented in the few extant studies of wraparound services. Toffalo (2000), for example, found that children who participated in a wraparound treatment program showed improvement in target behaviors. Similarly, Bickman et al. (2003) found that both wraparound services and treatment as usual produced improved functioning and decreased problems in children. The children in the wraparound group also received more wraparound services (e.g., case management, in-home treatment, and non-traditional services) and were provided “better continuity of care” at a greater cost (p. 135). Our study adds to this growing body of literature that suggests that wraparound services do result in improved outcomes for children.

126

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields

Interestingly, there was a sub-sample of youth participants for which parents made initial ratings indicating low levels of problems. Post study conversations with the parent research assistants suggested that some of the youth with low ratings were children who had been referred by the local child welfare agency as part of a reunification plan. In these circumstances, the wraparound team met at the end of the youth’s out-of-home placement in order to coordinate services as the child returned home. Parent ratings of the children at that point in time (after an extended period of time away from the child) were lower. And, as evidenced in the analyses, these children also were rated as having increased problems and poorer functioning as the study continued. It should be noted that the problems increased on average only slightly and into the mild range. As a result, wraparound services may have served the purpose of easing the transition and preventing more serious increases in problems. Nevertheless, the point of time at which the team is engaged with the family may need to be considered in future research in order to further investigate this finding. Importantly, Parent Research Assistant ratings of the wraparound teams indicated that the teams were adhering to the guiding principles of the wraparound process. This high degree of perceived adherence left little opportunity to identify correlates of adherence with various outcome measures. Nevertheless, wraparound adherence was significantly correlated with and accounted for 10% of the variance in goal attainment at 3 months. Similarly, adherence accounted for approximately 10% of the variance in parent ratings of family functioning at follow-up using the FACES. However, greater adherence was related to poorer functioning. Together these findings provide minimal and contradictory evidence regarding the relationship between wraparound adherence and outcome. As outcomes varied significantly within the groups, three potential explanations for our findings seem most relevant. First, wraparound services in this study were conducted in a consistent fashion with the guiding principles, but consistency does not guarantee the outcome (e.g., adherence is unrelated to outcome). Second, problems with the measurement of adherence (i.e., parent research assistants were overly generous in their rating of team meetings, the wraparound observation form is not sufficiently sensitive to the operations of team meetings, etc.) distorted the relationship between adherence and outcome. Third, adherence is related to outcome, but a more heterogeneous set of team meetings with varying levels of adherence will be needed to establish the relationship. Our study is not the first to find an inconsistent relationship between adherence and outcome. Toffalo (2000) in a study of adherence and outcome found no relationship between fidelity and behavioral outcomes. Fidelity in this study, however, was defined quite differently from the current study – namely as the ratio of mean actual hours of treatment provided to mean number of hours

Feedback within Wraparound

127

prescribed. Nevertheless, two studies now suggest that adherence may not be key to the provision of wraparound services. Contrary to our hypothesis that parents and family team members who received feedback would feel better informed regarding the case, parents and team members in both the feedback and non-feedback teams felt well-informed. Similarly, team members in both conditions had similar views of the outcome of the cases and their own participation. This may be an indication that outcome feedback from a standardized measure did not noticeably add to the information that is already shared among family team members in the wraparound approach. Because the wraparound approach specifically attempts to increase the communication among participating team members feedback may be redundant to the intervention (Burns et al., 2000). In addition, we found little evidence that feedback about outcome progress delivered to the team at 2 weeks, 4 weeks, 8 weeks, and 12 weeks provided any additional benefit to wraparound services. Parents in the feedback group were more likely to indicate that their child had met or exceeded initial goals when compared to parents of children in the non-feedback group. In addition, parent retrospective ratings of outcome indicated a slight advantage for participants in the feedback group over parents in the non-feedback group. No other differences in measures of problem severity, functioning, target complaints, or family functioning, were observed between feedback and non-feedback groups. Perhaps the coordinated nature of wraparound services with planned team meetings provides sufficient feedback so that a formal mechanism for providing standardized outcome is unnecessary. Alternatively, the feedback intervention may not have been sufficiently powerful. Four episodes of feedback over a 10-week period may not be frequent enough to monitor progress. In studies of outpatient therapy with adults, every session feedback produced significant improvements for clients identified as potential treatment failures (Lambert et al., 2001). If feedback produces a similar finding in treatments for children, the feedback may need to be more frequent and may be limited in benefit to those cases in which initial deterioration is evident. To test this effect, a much larger sample would be needed to have a sufficient number of potential treatment failures. Wraparound services will continue to be used as a method of coordinating intense services for youth involved with multiple systems. Our study provides additional evidence that these services can be beneficial for the youth who enter the service with clinical levels of problems. Whether team adherence to general guiding principles of wraparound services produces any additional benefit for the child must remain an open question. Our study found a high degree of perceived adherence, but outcome varied significantly and independently of adherence. Formalized feedback about progress using standardized outcome measures did not substantially improve the wraparound services delivered in this study. Only parent perceptions of goal attainment and retrospective outcome differed between

128

Ogles, Carlston, Hatfield, Melendez, Dowell, and Fields

feedback and non-feedback groups. Wraparound services may rely on such high levels of cooperation and communication that standardized feedback adds only a limited benefit. Continued efforts to explore the nuances of wraparound services will add empirical evidence that is necessary to back the claims associated with wraparound theory and practice.

ACKNOWLEDGMENTS This study was funded by a grant from the Ohio Department of Mental Health, Office of Program Evaluation and Research, #00.1139A

REFERENCES Battle, C. C., Imber, S. D., Hoehn-Saric, R., Stone, A. R., Nash, E. H., & Frank, J. D. (1966). Target complaints as a criteria of improvement. American Journal of Psychotherapy, 20, 184–192. Bickman, L., Smith, D. M., Lambert, E. W., & Andrade, M. R. (2003). Evaluation of a congressionally mandated wraparound demonstration. Journal of Child and Family Studies, 12, 135–157. Brown, R. A., & Hill, B. A. (1996). Opportunity for change: Exploring an alternative to residential treatment. Child Welfare, 75, 35–57. Burchard, J. D., & Clark, R. T. (1990). The role of individualized care in a service delivery system for children and adolescents with severely maladjusted behavior. The Journal of Mental Health Administration, 17, 48–60. Burns, E. J., Burchard, J. D., & Yoe, J. T. (1995). Evaluating the Vermont system of care: Outcomes associated with community-based wraparound services. Journal of Child and Family Studies, 4, 321–339. Burns, B. J., Schoenwald, S. K., Burchard, J. D., Faw, L., & Santos, A. B. (2000). Comprehensive community-based interventions for youth with severe emotional disorders: Multisystemic therapy and the wraparound approach. Journal of Child and Family Studies, 9, 283–315. Clark, H. B., Lee, B., Prange, M. E., & McDonald, B. A. (1996). Children lost within the foster care system: Can wraparound service strategies improve placement outcomes? Journal of Child and Family Studies, 5, 39–54. Eber, L., Osuch, R., & Redditt, C. A. (1996). School-based applications of the wraparound process: Early results on service provision and student outcomes. Journal of Child and Family Studies, 5, 83–99. Epstein, M. H., Jayanthi, M., McKelvey, J., Frankenberry, E., Hardy, R., & Dennis, K. (1998). Reliability of the Wraparound Observation Form: An instrument to measure the wraparound process. Journal of Child and Family Studies, 7, 161–170. Hart, R. R. (1978). Therapeutic effectiveness of setting and monitoring goals. Journal of Consulting and Clinical Psychology, 46, 1242–1245. Kanfer, F. H., & Gaelick-Buys, L. (1991). Self-management methods. In F. H. Kanfer & A. P. Goldstein (Eds.). Helping people change (4th ed.; pp. 305–360). New York: Pergamon. Kiresuk, T. J., Smith, A., & Cardillo, J. E. (Eds.). (1994). Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum. Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11, 49–68. Lueger, R. J. (1998). Using feedback on patient progress to predict the outcome of psychotherapy. Journal of Clinical Psychology, 54, 383–393. Myaard, M. J. (2000). Applying behavior analysis within the wraparound process: A multiple baseline study. Journal of Emotional and Behavioral Disorders, 8, 216–229.

Feedback within Wraparound

129

Ogles, B. M., Hatfield, D., Carlston, D., Fields, S. A., Dowell, K., & Melendez, G. (2002). The role of fidelity and feedback in the wraparound approach: initial data. New Research in Mental Health: 2000-2001 Biennium, 15, 240–245. Ogles, B. M., Melendez, G., Davis, D. C., & Lunnen, K. M. (2001). The Ohio Scales: Practical outcome assessment. Journal of Child and Family Studies, 10, 199–212. Olson, D. H., Portner, J., & Lavee, Y. (1985). FACES-III. St. Paul, MN: University of Minnesota. Rosenblatt, A. (1996). Bows and ribbons, tape and twine: Wrapping the wraparound process for children with multi-system needs. Journal of Child and Family Studies, 5, 101–116. Stroul, B. A., & Friedman, R. M. (1986). A system of care for children and youth with severe emotionally disturbances (Rev. Ed.). Washington, D. C.: Georgetown University Child Development Center. Toffalo, D. A. D. (2000). An investigation of treatment integrity and outcomes in wraparound services. Journal of Child and Family Studies, 9, 351–361. VanDenBerg, J. E., & Grealish, E. M. (1996). Individualized services and supports through the wraparound process: Philosophies and procedures. Journal of Child and Family Studies, 5, 7–21.