The Ecodevelopmental Model and Methodological Questions

1. Ecodevelopmental and Systemic Modeling and Implementing High Fidelity Interventions in Real World Settings Guillermo Prado1,2, Hilda Pantin1, Seth Scwartz1, Jose Szapocznik1 & Daniel J. Feaster 1,2 1Center for Family Studies, University of Miami 2 Stempel School of Public Health, Florida International University

2. The Ecodevelopmental Model and Methodological Questions

3. The Center for Family Studies is interested in the role of ecodevelopmental context in the prevention and treatment of adolescent behavior problems, drug abuse, and HIV/AIDS.

4. Ecodevelopmental Theory • Incorporates three primary, integrated components: • Social Ecological Theory • (b) Developmental Theory • (c) Emphasis on Social Interactions

5. Social Ecological Theory Bronfenbrenner (1979, 1986) • Highlights the Multiple Influences on Adolescent Development, including: • Macrosystems(e.g., cultural & societal values) • (b) Exosystems(e.g., parents’ exosystemic stressors and social support for parents) • (c) Mesosystems(e.g., parental monitoring of peers and collaboration with youth’s school) • (d) Microsystems(e.g. family functioning)

6. Developmental Component Emphasizes the changing nature of youth, contexts, & their interdependenceover time e.g., family functioning is influenced not only by parents’ current social support & work stress, but also by previous levels of social support & work stress e.g., current family functioning in turn influences both present and future levels of adolescent behavior problems

7. Social-Interactional Component Risk and protection are expressed in the patterns of direct transactions between individuals within and across the different contextual levels e.g., when parents engage in supportive interactions with individuals outside the family, they are more likely to parent their children in supportive rather than harsh ways.

8. An Ecodevelopmental Perspective on Prevention

9. Context of Adolescent Behavior Problems, Drug Use, and Risky Sexual Behavior Social-Cultural Context Parental Resources/Stressors Parents’ Social Support Parents’ Work Stress Family Microsystem Parent-adolescent communication about sex Positive Parenting Parent-Adolescent Communication Marital Conflict Parental Involvement Family support Cultural Mismatch Language Problems Family-School Relations Parental involvement in school Monitoring homework Family-Peer Relations Parental monitoring of peers Supervision of situations of sexual possibility School School Bonding Academic Achievement Peers Substance use w/ friends Sexually active friends Immigration Policy Poverty

10. Parental-Adolescent Communication about Sex Parental Monitoring of Peers Parent Acculturation Social Support for Parents Peer Sexual Behavior Adolescent Acculturation Ecodevelopmental Model of Problem Behaviors Family Exosystemic Stressors Peer Substance Use Family Relations Early Adolescent Sexual Initiation Early Adolescent Substance Use Early Adolescent Problem Behaviors Social Cognitive Mediators re.Drug Use Social Cognitive Mediators re. Sex

11. Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves multiple levels of nesting • Repeated Observations  Individuals Families  Ecosystemic levels across developmental stages • Statistically, data are non-independent—Substantively people and their social contexts are interdependent

12. Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves longitudinal processes • Ecodevelopmental processes are interrelated and influence each other over time • How do we model these interrelationships over time? • Ecodevelopmental processes develop over time, and a snapshot of such a process (e.g., family functioning at baseline) is not accurate • How do we account for this in our model?

13. Implications of Ecodevelopmental Theory for Methodological Development Ecodevelopmental research involves influences from multiple systems • Ecodevelopmental models involve two and three level interactions • How do we model these moderation effects when the variables are observed? • How do we model these moderation effects when the variables are latent?

14. Multiple levels of nesting in effectiveness trials Time is nested within adolescents who are nested within families who are nested with therapist who are nested both within treatment & site Treatment crosses site

15. Implementing a High Fidelity Systemic Treatment in Drug Abuse Treatment Centers

16. Plan of Section • Clinical Trials Network (CTN) • BSFT • BSFT Design Issues—Level of Control • BSFT Experience in High Fidelity Implementation

17. Clinical Trials Network • NIDA funded network to test the effectiveness of efficacious treatments in real world settings • As of 9/05: • Nodes = 17 • States = 34+Puerto Rico • CTPs = 152 • Protocols = 27 (11 closed to enrollment, 5 in development) • Currently 88 CTPs involved in 11 open studies • Mission: To implement science-based efficacious treatments in community settings AND to show that these implementations are an improvement over current practice

18. Brief Strategic Family Therapy—BSFT, CTN0014 • Brief Strategic Family Therapy is a systemic, process-focused family therapy • 4 months with weekly sessions • Up to 8 booster sessions over 8 months • Focus on changing repetitive (maladaptive) interactions within the family • Note that the focus is on underlying processes, not crises • May use a crisis as a content focus, but therapy addresses underlying “everyday” processes • BSFT has been the focus of over 30 years of research at the Center for Family Studies

19. BSFT-Therapy Components Three major techniques • Joining (Engaging Participants into Treatment) • Balanced across all family members • Must join with most powerful • Diagnosis (Family Relationships and Roles) • Restructuring (Implementing the Treatment Plan) • Work in Present—Enactments • Reframing • Shifting boundaries

20. BSFT CTN0014 • 8 Sites • 60 participants per site on average • 480 adolescent participants • Drug Use assessed monthly for 12 months • Full assessments at Baseline, 4 months, 8 months and 12 months • Delinquent Behaviors, Conduct Problems, Sexual risk behaviors, Adolescent Pro-social Activities, & Family Functioning

21. BSFT—Randomization • Participants will be randomized to BSFT or the clinic’s standard outpatient treatment • Note, randomization is at the individual level, not at the clinic level

22. Design Considerations for Effectiveness Trials • Level of Control • Homogeneity of Study Population (Szapocznik) • Standardization & Monitoring of Treatment (Szapocznik) • Standardization & Monitoring of Control (Feaster) • Handling of Site Variance (Feaster) • Efficacy study—Fixed Effect • Effectiveness—Random Effect

23. BSFT: Heterogeneous Treatment Population • Inclusion Criteria: • 12-17 years of age • Any illicit drug use in last 30 days • Lives or is expected to with “family” • Reside in the same geographic area as CTP • Signed consent & assent • Exclusion Criteria: • Not living with family (halfway house, institution, etc.) • Suicidal or homicidal risk must be stabilized, first • Current or pending severe criminal charges if likely to lead to incarceration • If already receiving drug treatment services

24. Choice for BSFT:Full Control of Experimental Condition • Extensive Training and Supervision • 5 months of training • Weekly supervision • Training and Supervision are considered integral to the BSFT model

25. Multiple Levels of Clinical Supervision & Adherence Monitoring • Clinical Supervision Weekly conference calls with a BSFT supervisor. Supervision includes: • videotape review • case discussion and planning. • National Clinical Supervisor • weekly face-to-face sessions with each clinical supervisor • Regularly sits in on selected supervision calls with sites • Adherence Ratings of Videotapes • Randomly selected sessions • trained independent raters in Miami • Failure to adhere to the BSFT model • Definition-<70% adherence for 3 consecutive sessions • Consequences & Corrective Action • no new cases until 80% in two consecutive sessions • Increased supervision & retraining until meets criteria • If criteria not met before conclusion of current cases withdrawn from study (at discretion of clinical supervisor)

26. Therapist Consent and Selection Process • Identification of volunteers • Consent • Demographics • Views of Adolescent Drug Abuse – Q-sort • Selection • Interviews (Site PI, National Study Director/ Coordinator, and BSFT Head Training Supervisor) • Family Session • Randomization • Academic training • Years of clinical experience

27. Training Phase for BSFT Therapists • Five-month clinical training program • Workshops • Four 3-day workshops • Week 1 (Workshop 1) *Miami • Week 3 (Workshop 2) • Week 5 or 6 (Workshop 3) • Week 13 (Workshop 4) • Supervision • Weekly group supervision • Each therapist will have ½ hour for videotape review and ½ for case discussion • Pilot Cases: 2-4 cases for each therapist • Certification

28. Implementation Phase • Active Cases • Caseload builds over time • Study caseload will range from 2-8 cases • (minimum = 0; maximum = 10) • Supervision • Weekly group supervision • Review of videotapes • Review of clinical forms • Treatment planning • Each therapist (2 active) will have 30-45 minutes for videotape review and 30-45 minutes for case discussion

29. Adherence Ratings By Site Good Adherence is  3

30. Adherence Ratings • Show adequate adherence • Some variability across sites • Supervisors’ ratings uniformly higher than than independent raters’

32. Some Statistical Approaches and Areas for Further Research in Systemic Modeling and Design of Effectiveness Trials

33. Plan of Section: • Specification of Site Effects & Bounds of Inference • Power analysis and Trial Planning • Time Structure of Models & Reciprocal Effects • Need for Simulation Research • Questions about Mplus & Simulation

34. Design Considerations for Effectiveness Trials • Level of Control • Homogeneity of Study Population (Szapocznik) • Standardization & Monitoring of Treatment (Szapocznik) • Standardization & Monitoring of Control (Feaster) • Handling of Site Variance (Feaster) • Efficacy study—Fixed Effect • Effectiveness—Random Effect

35. Considerations for Choice of Comparison Condition • Standardized Control Group • Smaller sample size (sites & participants) • High internal validity • Lacks ecological validity for CTPs • Treatment As Usual • Larger sample size (sites & participants) • High external validity • Highly variable across sites • Minimum site size— within site comparisons

36. Choice for BSFT:Treatment As Usual Compare BSFT to the population of treatments in the community Currently drug abuse treatment has considerable variability in treatment approach and implementation

37. Handling of Site Variance • Fixed Effect (control for site & remove variance from error): Cannot generalize statistically • Random Effect: Can generalize effect beyond the clinics included • Both site & site X treatment are random • Implications on power of study

38. Choice for BSFT: TAU • Differences in TAU at each site implies larger variance of the random Site X Treatment effect • Sites need not be randomly assigned, but need to describe the generality of clinics (if not randomly selected)

39. Fidelity of Implementation Variability in fidelity across sites will increase the site and site by treatment effects

40. Specifics of BSFT Statistical Plan Hypothesis 1: BSFT will be significantly more effective than TAU in reducing adolescent drug abuse, defined as the percentage of days with positive drugs use.

41. Example of Expected Trajectories

42. Secondary Hypotheses BSFT will be significantly more effective than TAU in : • Reducing: • Delinquent behaviors & conduct problems • Sexually risky behaviors • Increasing: • Prosocial activities (school, employment) • Family functioning (parenting, parent-adolescent communication)

43. Analysis Strategy • Randomization at the Individual Client Level • Multi-level Growth Curve with 3 levels • Level 1 is within person—Time • Level 2 is between persons (and incorporates treatment assignment) • Level 3 is between site • Controlling for baseline drug use (ANCOVA-type specification) • Note: not accounting for all levels of nesting in primary hypotheses (therapist effects examined in post-hoc analyses)

44. Proposed Model Level 1—Time: Time, a, is centered at Month 4 Level 2—Between Adolescents: Level 3—Between Sites:

45. Multisite Power Analyses(Raudenbush & Liu, 2001) Procedure assumes single post test (not growth curve) We have 12 monthly drug use assessments However, drug use is not normally distributed

46. Methodological Issues • Most research on design implications of site effects has been done by statisticians with drug trial experience • Aim is to prevent site effects or justifying ignoring them if they exist • Therefore, little prior evidence published on site variability & particularly on the site by treatment interaction

47. Methodological Issues • Power Analysis • Lack of prior info makes difficult • Lack of software (Raudenbush & Liu, 2001) • Simulation in M-Plus • Have been doing simulations • In SAS, have difficulty with 8 sites identifying the site & site X treatment variance in growth curve framework • Specifically, it may not be possible to identify the covariance between the site and site X treatment random components • Simulations do not exactly match Raudenbush & Liu

48. Resource Allocation Issue • Random site X treatment effect with variability in Treatment as Usual requires a large sample • Necessity of effectiveness research if to generalize to new clinics • Some disagree (due in part to costs) • Simulation to show Type 1 error • Look at potential mistakes of policy based on inappropriate overly precise estimates

49. Reciprocal Effects • Person-specific cross-lagged panel model • Could be formulated in a latent difference score framework • Model is important for systemic phenomenon like family interactions • Illustrated within a person using coping and distress data

50. “SETA-Fam” Hypotheses Target Woman’s Outcomes b Family Functioning SET a d e c Family Members’ Outcomes Dotted Line Refers to Hypotheses in SETA