CURRENT AND RECENT PROJECTS
CSR is supported by a large reputable university and has a demonstrated record in tracking large numbers of program participants to determine program effectiveness, in using multi-method strategies to determine program outcomes and to strengthen program services, in providing technical assistance and developing relationships with multiple stakeholders, and designing research that addresses public policy and programmatic issues.
In statewide, multi-site research initiatives, it is often necessary to rely on program staff to assist in data collection efforts. Consequently, the CSR puts considerable resources into providing technical assistance to program staff and in developing relationships with providers. Site visits occur regularly and feedback on data collection performance is provided.
Descriptions of research and evaluation projects are organized by 1) size or scope of the project, 2) program model or population being served, and 3) dates, from most to least recent project.
EVALUATION OF HOME VISITATION: The first series of evaluation and research studies described below are on home visitation services provided for high-risk mothers and fathers. Evaluation and research conducted over the past 16 years has focused on program theory, practices and services, implementation processes, belief systems of program staff, circumstances and characteristics of the families being served, and organizational development. A portfolio of research and evaluation design and methods have been utilized including qualitative, annual pre-post outcome evaluations, ethnography, and a randomized control study.
2009 to 2012: NFN Depression Improvement Study, Grant award, Children’s Trust Fund, State of Connecticut. Clinical trial of In-Home Cognitive Behavior Therapy (IH-CBT), replication of an evidence-based treatment model.
Currently conducting a clinical trial of In-Home Cognitive Behavioral Therapy (IH-CBT) for depressed mothers who are participating in the Nurturing Families Network Home Visitation Program. This study replicates “Treatment of Depression in Home Visitation,” conducted by Robert Ammerman, Principal Investigator, and Frank Putnam, Co-Principal Investigator, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH. Eligible mothers are randomized to either IH-CBT or home visiting as usual (HVAU). Pre, post, and 3-month follow-up assessments are conducted with both groups to determine changes in depression status.
The specific objectives of the study are to: (1) Compare the efficacy of IH-CBT in contrast to home visitation “as usual.” (2) Test the generalizability of the treatment model by examining differential effects of IH-CBT on two home visitation models: Ohio’s Early Childhood Succeeds and Connecticut’s Nurturing Families Network (NFN) program. See presentation: “Nurturing Families Network Depression Improvement Study.”
2009-2012: Depression Improvement Study: Impact on Child Social Emotional Outcomes, Grant award, Child Health and Development Institute, CT.
A randomized control trial comparing child outcomes for depressed mothers in an experimental group receiving In Home-Cognitive Behavior Therapy in addition to home visitation services (see above study). The clinical trial is being conducted over a 3 year period. Assessment of child social-emotional behavior is collected as a measure of outcome at or just after the child’s 1 year birthday. See presentation: “Depression Improvement Study.”
The Center worked with program administrators at the Nurturing Families Network home visitation program and expert consultants to develop a web-based data collection system scheduled to be fully implemented in October 2011. This will establish a cross-site data management infrastructure that will provide agencies with a monitoring system and regular performance management reports on program implementation and process measures. It will also allow for home visitors to manage their caseloads and schedules and managers and supervisors to oversee the quality of services to families. The data from the system will be exported to the Center staff for statistical analyses of program implementation and effectiveness.
2007-2010: Nurturing Families Network Fatherhood Involvement Study. Nurturing Families Network, Children’s Trust Fund, Department of Social Services.
Conducted a qualitative research study and analyses that examined a broad scope of issues affecting the lives of socially and economically marginalized fathers in CT and their relationships with their children. A series of interviews, 4 in total, were conducted with a racially diverse group of fathers (N=35) residing in 16 cities and towns between 2007 and 2009.
Analysis also explored fathers’ views on home visiting and support groups and was used to inform program practices for providing home visitation to fathers.
1995 to Present: Nurturing Families Network Annual Outcome Evaluation, CT Children’s Trust Fund, Department of Social Services
The Nurturing Families Network (NFN) is a statewide intensive home visiting program designed to promote positive parenting and reduce incidences of maltreatment. Home visitation is offered to high-risk, first-time mothers at or before birth to a child’s 5th birthday. There are forty-two NFN program sites in the service areas of all twenty-nine birthing hospitals that are housed within both public and non-profit service centers, from neighborhood-based community programs to large hospital and clinic settings.
Since the model’s inception in 1995 as Healthy Families Connecticut, evaluation and research conducted by the Center for Social Research (CSR) have been used to inform and refine program practices. For this statewide, multi-site research initiative, it is necessary to rely on program staff to assist in data collection efforts. Consequently, the CSR puts considerable resources into providing technical assistance to program staff and in developing relationships with providers. Site visits occur regularly and feedback on data collection performance is provided.
Data collection protocols and forms and standardized instruments are administered at the time services begin and at regular intervals (i.e., at 6 months, and annually) during program participation. Descriptive and pre-post outcome data are routinely analyzed and compiled in both yearly and quarterly reports, and used for monitoring changes occurring in areas that the program is trying to impact. Outcome and process data have been used to inform program development and provide direction for ongoing research and evaluation.
In order to monitor program sites’ delivery of services, data on a number of process and outcome variables are compared across program sites using a Quality Assurance (QA) measure. The QA chart gives program leaders a mechanism for routinely monitoring program services across the state, assessing implementation information, and ensuring model fidelity. It also allows program leaders and other stakeholders to examine variation in critical outcomes.
Research Initiatives. In addition to annual outcome evaluations, research initiatives during the past 15 years on program practices, implementation, and on the characteristics and circumstances of the families served are listed below in chronological order:
- “Reflections on a program” (1996): Organizational practices and daily activities of the program were documented to inform and refine conceptualization of the program intervention (Black & Steir, 2007).
- “Process Evaluation of a Home Visitation Program to Enhance Positive Parenting and Reduce Child Maltreatment” (1998-2000): Qualitative analyses documented program practices and a theoretical rationale for the paraprofessional model (Black & Markson, 2001).
- Study Circles (July-October 2001): An action research design was employed to promote dialogue among practitioners and propose practices, policies, and solutions to concerns identified in the prior process evaluation (Diehl, 2001).
- Continuous Quality Improvement (CQI) (2002 and ongoing): A Continuous Quality Improvement Team (CQI) emerged from the Study Circles and is charged with making policy recommendations that govern the services of NFN and overseeing the practices (Damboise, Powell, & Black, 2003).
- “Life Stories of Vulnerable Families in Connecticut” (2002-2004): Analysis of 171 interviews with program participants conducted in 2002 and 2003 identified four patterns of vulnerability among mothers at risk for poor parenting (Black, Erdmans, & Dickinson, 2004).
- Expanded analysis of child abuse and neglect reports (annually since 2004): These analyses compare rates of abuse and neglect between NFN program sites and other studies of parents who have similar high-risk factors but who did not receive an intervention (Damboise & Black, 2004).
- Hartford NFN, Neighborhood analysis (2005) and New Haven NFN, Neighborhood analysis (2008): Analysis of social and economic context of program and family neighborhoods.
- NFN Special Report, “Nurturing Families Network, Leading the Way in Connecticut: Where we’ve been, what we’ve learned, where we’re going” (2008): A critical analysis conducted by the NFN research team, comparing the NFN/PAT program with national models (Hughes et al., 2008).
- “Revisiting the Cultural Broker Model” (2007-2008): Focus group analysis designed to elucidate the decision making processes of home visitors in identifying family needs, and helping families connect to community resources (Hughes, 2008).
- Development of a web-based data collection system (2008-2011): A cross-site data management infrastructure that provides agencies with a monitoring system and performance management reports on program implementation, process and outcome measures.
- Clinical trial of In-Home Cognitive Behavioral Therapy (IH-CBT) for NFN/PAT first-time mothers who meet the criteria for major depression: Clinical trial of In-Home Cognitive Behavior Therapy (IH-CBT) for mothers who meet criteria for major depression, replication of an evidence-based treatment model (Hughes et al.).
- Fatherhood Involvement Study (2008-2010): Examined a broad scope of issues that affect the lives of vulnerable fathers, their relationships with their children, and their participation and outcomes in the NFN program (Black, Walker, & Keyes, 2010).
2000 to Present: Evaluation of Family-School Connection, CT Children’s Trust Fund, Department of Social Services
The Family School Connection program, located in five communities around Connecticut, provides home visitation and support services for families of children who have been identified as having truancy, academic, behavioral issues or other indicators of educational neglect. The goals of the program are to improve parenting skills and help families become more involved with their children’s educational experiences.
Evaluation of the Family School Connection (FSC) program includes both outcome and process measures. Process measures include enrollment data, demographic profiles of program participants, and utilization of services. In addition, data on participant outcomes show if, and how, families change on measures of parenting, and community and school involvement over the course of their participation in the FSC program. See report: “Family School Connection 2010 Process and Outcome Evaluation Report”
EARLY CHILDHOOD SERVICE SYSTEMS: The series of studies, evaluation, and technical assistance work described below focus on early childhood systems of care in Connecticut and in other states and a program model that connects children with developmental or behavioral problems and their families to community-based services.
2010 to 2013: Help Me Grow, National Technical Assistance Center, CT Children’s Medical Center, Kellogg Foundation.
Provide technical support and consultation as needed for building evaluation capacity (data collection, monitoring, reporting) and designing research studies within Help Me Grow (HMG) National Technical Assistance Center and within and across HMG replication sites (see below projects for description of HMG model).
In collaboration with affiliates, conduct a research study that will 1) help states, private donors, policy makers, child health care providers, and partner organizations to better understand how HMG systems support families and; 2) add new information to the resilience-based approach, specifically inform prevention and intervention efforts using a protective factor framework.
2008-2010: Project Evaluation and Technical Assistance. Help Me Grow National Replication Project, CT Children’s Medical Center, Commonwealth Foundation.
A process evaluation was conducted to document and analyze the replication of the Help Me Grow model first developed in the state of Connecticut in 2002 (Dworkin & Bogin, 2006; Dworkin & Honigfeld, 2009). The model promotes the early detection of young children at risk for adverse developmental and behavioral outcomes by placing health services within a broader system with other sectors and linking children and their families to community-based programs and services through a single point of entry. Information on project implementation in the five replication sites was gathered through individual interviews with the technical assistance (TA) team, focus group discussions with project leaders, and progress reports and other related written material completed by the project liaisons. See evaluation report: “Process Evaluation: Help Me Grow Replication Project.”
2009-2010: Help Me Grow Orange County 2009 Annual Evaluation Report, Help Me Grow, Orange County, University of California.
Help Me Grow Orange County (HMG-OC), a model developed in Connecticut and replicated in Orange County, California since 2005, is a comprehensive, coordinated system designed to assist child health care providers, other professionals, and families in improving developmental outcomes for children, birth through five. Core components of the system include a centralized call center staffed by care coordinators who assist families and professionals in connecting children to appropriate programs and services, community outreach and networking to facilitate access to and collaboration among professionals, nonprofit organizations, and government agencies, education and training for pediatric and child care providers, and training on developmental screening and monitoring using the Ages & Stages Questionnaire.
Results-Based Accountability (RBA, Freidman, 2005) provides a framework for the analysis: Analysis shows change in trends over the first 3 years of the program’s operation (January 2007 through September 2009). Performance measures are organized according to the following: “How much did Help Me Grow do?” (i.e., utilization of the program and related data), “How well is Help Me Grow doing?” (i.e., family referrals for services and community outreach efforts), and “Is anyone better off as a result of utilizing Help Me Grow?” (i.e., final disposition of cases and outcomes). In addition, qualitative analyses of individual case notes and other data collected by care coordinators were conducted by the research team. See report: “Help Me Grow Orange County 2009 Annual Evaluation Report”
2007-2008: Help Me Grow Hard to Reach Project, Connecticut Children’s Trust Fund, Kellogg Foundation
Evaluation of a pilot project designed to build on the Help Me Grow and Health Outreach for Medical Equality (HOME) models. The primary goal of the project was to connect hard-to-reach children and families living in the neighborhoods of the North-end of Hartford, CT to health care and community-based services as needed.
Interviews, on-site observations, and quantitative data analyses were used to assess: 1) progress and issues related to start-up and implementation, 2) project activities and outcomes including family service needs, referrals, and results, 3) the extent to which program goals were achieved, 4) penetration of the services within the community, and 5) families response to services. See report on Help Me Grow ‘Hard to Reach’ Project, provided to W.K. Kellogg Foundation.
2002 to Present: Annual Outcome Evaluation of the Help Me Grow Program, CT Children’s Trust Fund, Department of Social Services
The CT Help Me Grow statewide system is designed to identify children at-risk for poor developmental outcomes as early as possible and connect them to community resources and local programs. Since the inception of CT Help Me Grow in 2002, thousands of parents in Connecticut needing help with their child have called Child Development Infoline/Help Me Grow with questions about a developmental or behavioral problem. The phone is answered by one of six care coordinators who have a broad knowledge base of child development issues, available resources and services in Connecticut, agency and professional roles, and where to find information. Also since 2002, Help Me Grow staff have been providing trainings on developmental screening and monitoring to health care providers and other professionals in child care and social services. In addition, these community development liaisons bring professionals together from a wide range of agencies through “Network Breakfasts” in order to share information, widen connections, and provide each other support.
Connecticut General Assembly and Results-Based Accountability: In accordance with Connecticut’s General Assembly Appropriation Committee, results-based accountability (RBA, Freidman, 2005) provides a framework for analysis; that is, data – or indicators of performance and results – are presented to show where the program has been and a forecast of where the program is going. Data show trends over time. Performance measures are organized according to the following: 1) “How much did Help Me Grow do?” (i.e., utilization of the program and related data including information on where callers reside, presenting issues and program service needs, and rates of calls per case ); 2) “How well is Help Me Grow doing?” (i.e., family referrals for services and community outreach efforts to both pediatricians and community support programs); and 3) “Is anyone better off as a result of utilizing Help Me Grow?” (i.e., outcomes or rate of connection to services, and final disposition of cases).
Analysis of data by the “Five Connecticuts:” While Connecticut is a prosperous state with the highest per capita income in the country, it also has areas throughout the state with very high concentrations of low-income populations. Because income and wealth in Connecticut is proportionately unequal, and because children living in poverty are at increased risk for developmental and behavioral problems, we compare utilization and outcomes of the program between five distinct Connecticut town groups: Wealthy CT, Suburban CT, Rural CT, Urban Periphery CT, and Urban Core CT (see The Changing Demographics of Connecticut-1990-2000, Center for Population Research, University of Connecticut, 2004).
REENTRY PROGRAMS, INCARCERATION TO COMMUNITY: The below studies use qualitative methods to evaluate PROGRAM process and implementation of reentry programs for people returning to the community following incarceration.
2010 to 2012: Implementation Evaluation of Department of Correction Initiatives: Program Integration and DOC Reentry Program, Two-year grant award, Connecticut State Department of Correction.
Evaluate two initiatives of the Department of Correction (DOC): CT DOC Reentry Workbook (pre-release) program and the Program Fidelity project (post release services) in order to 1) determine if programs are being implemented with fidelity to the model, 2) identify challenges and competing issues to implementation, 3) assess the extent to which program participants (inmates, ex-offenders) utilize these resources and what they find to be helpful, 4) inform program development and training, and 5) develop capacity to evaluate effectiveness of these initiatives (i.e., to ensure successful reentry for offenders and reduce recidivism).
Researchers gather and analyze information and data on implementation, utilization, and effectiveness of the two initiatives. Methodology includes interviews, focus groups, surveys, on-site observations, and review of program documentation. See presentation: “Implementation Evaluation of Two DOC Initiatives: Reentry Workbook Program and Program Fidelity Project”
2008-2009: Process Evaluation, Bridgeport Reentry Initiative, Connecticut State Department of Correction.
Conducted an implementation evaluation of the Bridgeport Reentry Initiative (BRI) grant. The evaluator worked collaboratively with a range of program staff – conducted an in-depth investigation of the program, learned about the program model and developments, explicated the objectives of the different elements of the program specific to creating change, and identified the framework or strategies for achieving the objectives. The final outcome of the implementation evaluation was the development of a logic model that: (1) Described the components and functions of the multi-agent program, (2) Established a shared understanding and a set of expectations among staff and key stakeholders of how the program will create change, and (3) Provided an approach for tracking participation and progress within programs over time and tracking outcomes for different participant groups (based on assessment of needs and risks and engagement in program activities). See presentation: “Bridgeport Reentry Initiative, Program Model for Creating Change.”
INTERNATIONAL DEVELOPMENT WORK: Evaluation and assessment studies described below are on community participation and implementation processes and social and interdisciplinary aspects of developmental work conducted in rural areas of North India and Western Kenya.
2009 to Present: Evaluating Implementation Projects in India, Engineers Without Borders, University of Hartford Student and Hartford Professional Team.
Implementation evaluation of environmental development work carried out by faculty-student teams on the University of Hartford’s Engineers Without Borders “Water for Abheypur” project, Abheypur, India. Information and data are collected through structured interviews with a range of stakeholders in the village to better understand level of interest, understanding, and engagement in Engineer Without Borders projects in relation to decision making and influence on implementation. By analyzing participant motivation, information, and influence, the model (Contextual Interaction Theory, Bressers & Xue, 2006) evaluates the likelihood that a project will “get off the ground,” and the likelihood that it will adequately address the problem. See presentation: “Water Implementation Projects In Rural North India: Evaluating implementation ‘on the ground’”
2008 to Present: Program Evaluation of University of Hartford’s Engineers Without Borders “Water for Abheypur” project, Abheypur, India.
The University of Hartford’s Student-Professional Engineers Without Borders (EWB) team has been working with villagers in Abheypur, a rural village in Haryana state in Northern India to establish sustainable and dependable water sources in areas of the village that have historically been dependent on neighboring communities. The purpose of the evaluation work, led by faculty-student field researchers, is to 1) conduct needs assessments to determine community priorities, 2) assess the impact of EWB efforts (i.e., positive change) on the daily lives and routines of the villagers, and 3) engage and involve local people in EWB activities and use what is learned to inform project development and ongoing management and maintenance for sustainability.
Evaluation activities include: on-site information gathering and observations of water systems and related infrastructures; reviewing available data and reports; and conducting meetings, interviews, focus groups, and household surveys with a cross-section of relevant villagers and key stakeholders such as village leaders, community members, women’s self-help groups, farmers, health care professionals, and educators.
2009-2010: Service Learning Workshops in lieu of Village Based Project. Grant award, Connecticut Campus Compact.
Evaluated the impact of service learning workshops in preparing students for field experiences in community development work. Students were taking courses at their own college to prepare them for their specific role in an Engineers Without Borders effort, then came together to participate in four service learning workshops prior to their fieldwork. The workshops were conducted during the fall semester, 2009 and work in Abheypur, a rural village in India, took place during the winter session, 2010. See report: “Summary Evaluation of Service Learning Workshops In lieu of Village Based Project.”
2010 to 2011: Promoting Health and Economic Opportunities in the Western Kenya Lake Region through Entrepreneurship, Grant award, Women’s Education and Leadership Fund, University of Hartford.
This initiative builds on a partnership with the Kenya Agriculture and Research Institute (KARI) to establish amaranth grain as a cash crop and a new food staple within Gem District in rural Western Kenya. One objective of the partnership is to improve the livelihood of Alour, a group of HIV-positive women who have organized into a farming collective focused on “Living Positively.” A student-faculty team worked collaboratively with extension workers from KARI and with farmers from Alour on the co-development of business practices for marketing of amaranth and assessment of new technologies for production of amaranth. Student-faculty teams conducted interviews, surveys, and observations to collect information and feedback from individuals involved in the collaboration, assessed the receptivity of the local people, and assessed the resources and materials available for production and distribution of amaranth. Interviews also gave an understanding of how the various stakeholders ‘on the ground’ consider and evaluate projects and make decisions. Business practices and strategies (package design and marketing materials) and new innovative technology solutions (human powered thresher and a mechanical seed planter) were designed, piloted, and further customized based on the marketplace, the receptivity of the local people, and the resources in the region. See presentation: “Kenya 2010: Working in Partnership with Farming Collectives.”
2009 to 2011: Project Development: US-Kenya (ACESS) partnership promoting research on clean water, appropriate technologies, and sustainable agribusiness, University of Hartford, University of Rhode Island, Mount Holyoke College, African Center for Engineering Social Solutions, and Kenya Agricultural and Research Institute.
Developed university-community partnerships based on a model of action research that provides faculty and students with opportunities for problem solving community-identified issues and conducting research designed for social improvement. Facilitated communication and kept process in motion for developing interdisciplinary collaboratives, fostering both team work and individual contributions, and advancing the goals of community groups. See presentation: “University Partnership Supporting Africa Center for Engineering Social Solutions.”
OTHER RESEARCH INITIATIVES: The below described evaluation and research projects represent a range of studies in terms of purpose, design, program population, and communities.
2007 to 2011: Evaluation of the Diversity Program, Discover Center, Farmington, CT
Develop an evaluation tool that will assess students’ attitudes, perceptions, and stereotypes about race and ethnicity pre and post experience at the Discovery Center. The program “partners” urban and suburban schools together for three years and includes a combination of academic lessons and residential camp experiences. A process evaluation was conducted during the 2007-2008 public school year in order to explicate how the program was designed to create a change in attitudes and behavior (i.e., towards others of different race, ethnicity, or family backgrounds). See report: “Preliminary Report/Overview of Evaluation Activities, May, 2008.” A preliminary survey was developed using the findings from the process evaluation and a review of relevant literature and piloted to small groups of program participants (both pre and post program participation) during the fall of 2008. Based on the oral feedback from students and analyses of items (descriptive information), the survey was revised. See report: “Development of an Evaluation Tool To Assess Students’ Attitudes, Perceptions, and Stereotypes about Race and Ethnicity Pre and Post Experience at the Discovery Center.” Currently (2010 and 2011) administering it to a large sample of participants (400-500 students). Data from this sizable sample will be used to (1) conduct a thorough assessment of the instrument’s validity and reliability, (2) finalize the scales for each of the domains of experiences and outcomes, and (3) obtain preliminary data from the instrument for pre-post analyses.
2010 to 2014: Meriden Partnership For Success and Middletown Best Practices Strategic Prevention Framework, Rushford Center Inc, Substance Abuse Prevention for Teens, Department of Mental Health Addition Services.
Provide technical support for evaluation and development to program staff and Community Task Force (Meriden and Middletown programs). Help to monitor both process and outcomes of initiatives and bring evaluation logic to programming to assist with setting up monitoring and feedback systems. Design evaluation with the coalition staff that focuses on short term outcomes.
2009-2010: Program Evaluation, Project Horizon. Process evaluation of a community outreach project of the University of Hartford’s College of Education, Nursing, and Health Professions.
Project Horizon is a community outreach project of the University of Hartford’s College of Education, Nursing and Health Professions. Nurse-students are required to complete a one-year internship (fall/spring semester) in homeless shelters in the city of Hartford. The purpose of the evaluation was to assess the impact the nurse-students have on the homeless population in the city of Hartford and, in turn, to assess the impact the experience has on the nurse-students. On-site observations, one-to-one and group discussions and interviews with nurse-students and program staff, and a review of program data and other materials and documentation were used for analyses of how well Project Horizon met its intended goals. See reports: “Preliminary Assessment” and “The Impact of Nurse-students on the Homeless and the Impact of the Homeless on the Nurse-students: An Evaluation of Project Horizon” Developed and helped facilitate implementation of a measurement tool (modification of measurement tool developed by George et al., 2007) to assess participatory action research conducted by the nurse-students. See measurement tool: “Appraisal of Participatory Research”
2008 to 2009: Program Evaluation of Educational Main Street, University of Hartford, 21st Century Community Learning Center Grant, University of Hartford and Hartford Public Schools
Interviews, focus group data, and direct observations were used to assess the collaboration process among the grant partners. The evaluator attended program planning sessions, after-school activities, and also facilitated a focus group discussion (on the purpose, process, and progress of the after school program) and conducted individual interviews with teachers, the program administrator, several of the tutors, and the school principal. The information gathered for analyses focused on collaboration between the partners, specifically the progress and issues related to communication, decision-making, and the development and implementation of the after-school program.