Development of Outcomes and Measures of Success

Essential to an effective continuous quality improvement (CQI) system is accurately identifying outcome and systemic areas to be measured and tracked that will assess the status and ongoing progress of an agency’s practices, programs, and services. In its August 27, 2012, Information Memorandum, ACYF-CB-IM-12-07 (available online at: http://www.acf.hhs.gov/programs/cb/resource/im1207), the Children’s Bureau stated that it intends to publish a specific set of monitoring measures in the future. Until those are known, however, concerns that have been identified in a State’s Child and Family Services Review (CFSR) and Program Improvement Plan, unless already sufficiently addressed, are a recommended beginning point for measurement toward desired outcomes.

Generally, CFSR results revealed that all States were challenged in assessing and meeting well-being needs of children and families, as well as achieving timely permanency. Specifically, many States struggle to ensure the social and emotional well-being of children and youth. Improvement in this area can have profound positive effects on several permanency issues that reflect the essence of a State child welfare program. Thus, these should receive strong consideration as areas needing to be tracked and assessed.

Outcomes are key in assessing and refining program delivery and supporting organization-wide quality improvement. When determining outcomes, or intended results of a program or initiative, agencies should strive to measure what works and does not work. In other words, rather than defining an outcome by whether or not a child or parent was the recipient of a practice or service, it should be measured instead by whether or not service participation improved functioning or the chances of success. For instance, rather than measuring success by whether or not a parent completed parent training, success (outcome) could be gauged by whether or not the training improved skills and capacities of that parent, perhaps measured by whether another incident of child maltreatment occurred within a specified period of time.

Outcomes capture the “what” and the “who,” and are written as "change statements." In other words, in defining outcomes, the details of the targeted initiative should be considered, as well as the recipients, intended impact, and change desired. For instance, in attempting to strengthen its youth independent living program, an agency, rather than defining its outcome goal as “prepare youth to live independently,” might consider instead the following as outcomes:

  1. Increased high school graduation rates of youth in foster care, and/or
  2. Decreased instances of youth in foster care being involved with juvenile justice

Since outcomes are broad in nature, performance indicators or measures serve as a bridge connecting intended outcomes and data collected. Measures are specific pieces of information that describe observable – or otherwise captured – characteristics or changes in factors. They are indicators that can be counted, reported, observed, or somehow detailed from data collected.

In composing measures, agencies should first calculate a baseline, or initial data that allow a comparison with subsequent data for assessing impact, and then identify targets, or the level of achievement (quantifiable goals) it hopes to achieve. Measures drawn up should be as simple as possible, while still being meaningful and useful. Performance measures enable an organization to use factual data it has gathered to determine whether its programs, practices, and CQI system as a whole have had a measurable impact on consumers and whether programmatic goals have been met.