Evaluating Our Children’s Mental Health Initiative
Recently, we conducted an evaluation of our children’s mental health initiative, because we’re committed to assessing the success of our work. Although we’re only two years into the project, results from the evaluation show that our project design is working. More importantly, more children are being identified early and being referred to treatment that can improve their mental health and prevent them from developing bigger issues that can lead to their entry into the juvenile justice system.
Before we share our lessons learned, here’s a little background. Our goal in the area of children’s mental health is:
“Reduce the number of at-risk children ages 6-14 entering intensive treatment and/or the juvenile justice system due to mental health issues.”
In 2008, we made this one of our strategic objectives:
“Promote a community-based system of early identification and intervention for children at risk of developing mental health issues.”
Our grant-making strategy had two phases. In 2007, in the first phase, 11 communities developed a plan for instituting a system of early identification and intervention for children who appear to be having behavioral health issues
In 2009, we selected 4 of the 11 communities for the second phase of implementation. Each of the communities—Derby, Manchester, Middletown, and Waterbury—based their mental health identification system within an elementary school.
We asked Dr. Megan Smith of Yale University to conduct an evaluation of how the grant initiative and the 4 communities are doing. While we encouraged each site to create a system individually tailored to their community, they used a standard assessment measure across all sites, the OHIO scales, so that we could compare apples to apples. We also liked this scale because it incorporates parental satisfaction.
- All 4 grantees have successfully implemented a system of mental health early identification, assessment, referral, and intervention for children ages 6-14.
- Since February 2010, a total of 1,337 children have been screened for early behavioral and emotional symptoms. Of these children, 17 percent met criteria for further assessment to determine appropriate interventions, and received CT Health-funded interventions.
- Although we are early in the initiative and thus small numbers limit definitive conclusions, of the children who received CT-Health-funded services, trends in improvements in children’s functioning and behavior have been noted. Teachers and parents report:
- The severity of behavioral and emotional problems in children has decreased.
- Children’s overall functioning has increased.
- Noticeable increases in children’s motivation and self-direction.
- Decreases in parenting stress.
- Institutional level and community level systems change efforts are underway in each community.
This is great news, and it would seem that our approach to our goal is working. With all evaluations come lessons learned and recommendations. Here’s the list:
- Deploy technical assistance to grantees to maximize the role of project coordinators and community advisory committees in systems change efforts. What we’ve learned is that program implementation requires a different approach and skills to implementing systems change. We can provide technical assistance to help these coordinators understand what we’re trying to achieve.
- Foster collaborative learning efforts across the communities focused on systems change, policy change, and sustainability. CT Health hosted “learning labs” that were a big hit with grantees. Why? Because rather than pitting grantee against grantee, we encouraged grantees to share and learn together. We’ll continue to do this.
- Consider the feasibility and acceptability of obtaining data elements for initiatives around the prevention of children’s mental illness. This was a real eye-opener for us. We want to reduce the number of children entering juvenile justice, but there is a gap of several years between this program and the age when kids enter the juvenile justice system. Additionally, there are issues with maintaining privacy for these children.
- Focus grantees on the importance of utilizing data to promote systems change. Currently, all four communities are required to collect relevant data in their system. Ideally, all grantees should systematically review this data to improve the performance of their model.
We presented these findings to our board of directors in June, and are moving now towards how we can take these recommendations and help us improve this grant-making initiative.
Have you received similar feedback from any initiatives you’ve undertaken? What did you do with it? How did you learn from it?