EVALUATION OF HOME VISITATION: The first series of evaluation and research studies described below are on home visitation services provided for high-risk mothers and fathers. Evaluation and research conducted over the past 16 years has focused on program theory, practices and services, implementation processes, belief systems of program staff, circumstances and characteristics of the families being served, and organizational development. A portfolio of research and evaluation design and methods have been utilized including qualitative, annual pre-post outcome evaluations, ethnography, and a randomized control study.
2009 to 2012: NFN Depression Improvement Study, Grant award, Children’s Trust Fund, State of Connecticut. Clinical trial of In-Home Cognitive Behavior Therapy (IH-CBT), replication of an evidence-based treatment model.
Currently conducting a clinical trial of In-Home Cognitive Behavioral Therapy (IH-CBT) for depressed mothers who are participating in the Nurturing Families Network Home Visitation Program. This study replicates “Treatment of Depression in Home Visitation,” conducted by Robert Ammerman, Principal Investigator, and Frank Putnam, Co-Principal Investigator, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH. Eligible mothers are randomized to either IH-CBT or home visiting as usual (HVAU). Pre, post, and 3-month follow-up assessments are conducted with both groups to determine changes in depression status.
The specific objectives of the study are to: (1) Compare the efficacy of IH-CBT in contrast to home visitation “as usual.” (2) Test the generalizability of the treatment model by examining differential effects of IH-CBT on two home visitation models: Ohio’s Early Childhood Succeeds and Connecticut’s Nurturing Families Network (NFN) program. See presentation: “Nurturing Families Network Depression Improvement Study.”
2009-2012: Depression Improvement Study: Impact on Child Social Emotional Outcomes, Grant award, Child Health and Development Institute, CT.
A randomized control trial comparing child outcomes for depressed mothers in an experimental group receiving In Home-Cognitive Behavior Therapy in addition to home visitation services (see above study). The clinical trial is being conducted over a 3 year period. Assessment of child social-emotional behavior is collected as a measure of outcome at or just after the child’s 1 year birthday. See presentation: “Depression Improvement Study.”
The Center worked with program administrators at the Nurturing Families Network home visitation program and expert consultants to develop a web-based data collection system scheduled to be fully implemented in October 2011. This will establish a cross-site data management infrastructure that will provide agencies with a monitoring system and regular performance management reports on program implementation and process measures. It will also allow for home visitors to manage their caseloads and schedules and managers and supervisors to oversee the quality of services to families. The data from the system will be exported to the Center staff for statistical analyses of program implementation and effectiveness.
2007-2010: Nurturing Families Network Fatherhood Involvement Study. Nurturing Families Network, Children’s Trust Fund, Department of Social Services.
Conducted a qualitative research study and analyses that examined a broad scope of issues affecting the lives of socially and economically marginalized fathers in CT and their relationships with their children. A series of interviews, 4 in total, were conducted with a racially diverse group of fathers (N=35) residing in 16 cities and towns between 2007 and 2009.
Analysis also explored fathers’ views on home visiting and support groups and was used to inform program practices for providing home visitation to fathers.
1995 to Present: Nurturing Families Network Annual Outcome Evaluation, CT Children’s Trust Fund, Department of Social Services
The Nurturing Families Network (NFN) is a statewide intensive home visiting program designed to promote positive parenting and reduce incidences of maltreatment. Home visitation is offered to high-risk, first-time mothers at or before birth to a child’s 5th birthday. There are forty-two NFN program sites in the service areas of all twenty-nine birthing hospitals that are housed within both public and non-profit service centers, from neighborhood-based community programs to large hospital and clinic settings.
Since the model’s inception in 1995 as Healthy Families Connecticut, evaluation and research conducted by the Center for Social Research (CSR) have been used to inform and refine program practices. For this statewide, multi-site research initiative, it is necessary to rely on program staff to assist in data collection efforts. Consequently, the CSR puts considerable resources into providing technical assistance to program staff and in developing relationships with providers. Site visits occur regularly and feedback on data collection performance is provided.
Data collection protocols and forms and standardized instruments are administered at the time services begin and at regular intervals (i.e., at 6 months, and annually) during program participation. Descriptive and pre-post outcome data are routinely analyzed and compiled in both yearly and quarterly reports, and used for monitoring changes occurring in areas that the program is trying to impact. Outcome and process data have been used to inform program development and provide direction for ongoing research and evaluation.
In order to monitor program sites’ delivery of services, data on a number of process and outcome variables are compared across program sites using a Quality Assurance (QA) measure. The QA chart gives program leaders a mechanism for routinely monitoring program services across the state, assessing implementation information, and ensuring model fidelity. It also allows program leaders and other stakeholders to examine variation in critical outcomes.
Research Initiatives. In addition to annual outcome evaluations, research initiatives during the past 15 years on program practices, implementation, and on the characteristics and circumstances of the families served are listed below in chronological order:
- “Reflections on a program” (1996): Organizational practices and daily activities of the program were documented to inform and refine conceptualization of the program intervention (Black & Steir, 2007).
- “Process Evaluation of a Home Visitation Program to Enhance Positive Parenting and Reduce Child Maltreatment” (1998-2000): Qualitative analyses documented program practices and a theoretical rationale for the paraprofessional model (Black & Markson, 2001).
- Study Circles (July-October 2001): An action research design was employed to promote dialogue among practitioners and propose practices, policies, and solutions to concerns identified in the prior process evaluation (Diehl, 2001).
- Continuous Quality Improvement (CQI) (2002 and ongoing): A Continuous Quality Improvement Team (CQI) emerged from the Study Circles and is charged with making policy recommendations that govern the services of NFN and overseeing the practices (Damboise, Powell, & Black, 2003).
- “Life Stories of Vulnerable Families in Connecticut” (2002-2004): Analysis of 171 interviews with program participants conducted in 2002 and 2003 identified four patterns of vulnerability among mothers at risk for poor parenting (Black, Erdmans, & Dickinson, 2004).
- Expanded analysis of child abuse and neglect reports (annually since 2004): These analyses compare rates of abuse and neglect between NFN program sites and other studies of parents who have similar high-risk factors but who did not receive an intervention (Damboise & Black, 2004).
- Hartford NFN, Neighborhood analysis (2005) and New Haven NFN, Neighborhood analysis (2008): Analysis of social and economic context of program and family neighborhoods.
- NFN Special Report, “Nurturing Families Network, Leading the Way in Connecticut: Where we’ve been, what we’ve learned, where we’re going” (2008): A critical analysis conducted by the NFN research team, comparing the NFN/PAT program with national models (Hughes et al., 2008).
- “Revisiting the Cultural Broker Model” (2007-2008): Focus group analysis designed to elucidate the decision making processes of home visitors in identifying family needs, and helping families connect to community resources (Hughes, 2008).
- Development of a web-based data collection system (2008-2011): A cross-site data management infrastructure that provides agencies with a monitoring system and performance management reports on program implementation, process and outcome measures.
- Clinical trial of In-Home Cognitive Behavioral Therapy (IH-CBT) for NFN/PAT first-time mothers who meet the criteria for major depression: Clinical trial of In-Home Cognitive Behavior Therapy (IH-CBT) for mothers who meet criteria for major depression, replication of an evidence-based treatment model (Hughes et al.).
- Fatherhood Involvement Study (2008-2010): Examined a broad scope of issues that affect the lives of vulnerable fathers, their relationships with their children, and their participation and outcomes in the NFN program (Black, Walker, & Keyes, 2010).
2000 to Present: Evaluation of Family-School Connection, CT Children’s Trust Fund, Department of Social Services
The Family School Connection program, located in five communities around Connecticut, provides home visitation and support services for families of children who have been identified as having truancy, academic, behavioral issues or other indicators of educational neglect. The goals of the program are to improve parenting skills and help families become more involved with their children’s educational experiences.
Evaluation of the Family School Connection (FSC) program includes both outcome and process measures. Process measures include enrollment data, demographic profiles of program participants, and utilization of services. In addition, data on participant outcomes show if, and how, families change on measures of parenting, and community and school involvement over the course of their participation in the FSC program. See report: “Family School Connection 2010 Process and Outcome Evaluation Report”