Defining CQI and Building a CQI Framework

Continuous quality improvement (CQI) is the complete process of identifying, describing, and analyzing strengths and problems and then testing, implementing, learning from, and revising solutions. CQI relies on a proactive organizational and/or system culture that supports continuous learning; it must be firmly grounded in the overall mission, vision, and values of a child welfare agency or system. Perhaps most important, CQI depends upon the active inclusion and participation of staff at all levels of the agency/system, children, youth, families, and stakeholders throughout the process (Using CQI to Improve Child Welfare Practice, 2005).

More simply defined, CQI is an organization-wide system that involves the identification, dissemination, and measurement of best practices, as well as enhancement of processes and systems, to improve overall agency functioning and ensure more positive outcomes.

A key principle of CQI is that those who are closest to the work are the true experts in the field. Consumers, such as parents, children, and youth, and external stakeholders, such as courts, Tribes, and service providers, have much to contribute from completely different perspectives and should be incorporated by agency leaders into all phases of the CQI process. Youth, birth parents, and foster parents should be assigned to working CQI teams or committees to ensure a holistic perspective.

Some State child welfare agencies are very open and transparent, and fully involve stakeholders, families, and youth in their assessments and strategic planning activities, including their Child and Family Services Plan and Program Improvement Plan assessment and feedback. Consumers and stakeholders are made to feel such a part of these agencies that when an agency celebrates its successes, the stakeholders and families feel a special pride and that they, too, have succeeded.

This section discusses the activities that make up a well-functioning CQI system, focusing on the importance of aligning CQI with an agency's mission, values, vision, and practice. It also outlines distinct differences between quality assurance (QA) and CQI activities, and discusses the critical importance of integrating staff, external stakeholders, and consumers into CQI processes. Finally, it defines and explains the development of outcomes and measures of success that should be used to realistically measure the results and outcomes of the CQI initiative's various components and determine whether improvements are needed to improve practice and processes. It also stresses the importance of using measures as a “bridge” to connect data and intended outcomes.

Aligning CQI and an Agency's Mission, Vision, Values, and Practice

Continuous quality improvement (CQI) systems help align the agency’s practices, procedures, policies, training, and services with mission, vision, and values. The agency should focus on specific practice standards and outcomes that are grounded in its mission and values. For example, an agency whose mission reflects a strong focus on family engagement may be receiving consistent feedback from consumers that families do not feel valued and included. To ensure that the agency is adhering to its mission, practice would need to be changed to be more family-centered with a strong focus on promoting family engagement.

Other elements necessary to effectively carry out the mission and vision include:

  • Involving all key staff, external stakeholders, and consumer groups in the determination of change initiatives and measures
  • Providing maximum data/information access to stakeholders and consumers and all levels of staff
  • Using data to inform all major decisions
  • Using results to continually improve services
  • Integrating CQI activities into all aspects of systems and services, and into the agency’s core beliefs and values
  • Ensuring that child and family outcomes and measures are continually assessed

Developing and implementing a practice model, or a conceptual map of how an agency will operate and partner with consumers and stakeholders in its services, has provided an opportunity for some States to align their mission, vision, values, policies, and practice. At the same time, these States might examine their existing CQI program and, as needed, expand or modify it to ensure that their entire child welfare system coincides with the agency’s current values and standards and that all critical factors are aligned and in sync.

For instance, an agency’s practice standards might state that workers should, in visits with children, see and visit with the child alone, observe the child’s environment to make certain he or she is safe, ask foster parents if they have needs, etc. If practice is closely aligned with mission, vision, and values, agency standards in both policy and procedures will reflect those factors; management will be focused on meeting articulated standards; and the agency will be training and coaching toward achieving better practice in those areas. While standards alone do not ensure quality practice, they are a framework that reflects the agency’s values, principles, and approaches that help ensure positive outcomes for families and children.

When mission, values, vision, policies, and practice are synchronized, what is in writing, what is being coached, what is included in training, and what is assessed in case record reviews is aligned and the same. There is consistent messaging to all staff across all programs, as well as external stakeholders and consumers, which is critically important in making needed changes.

CQI provides the means to reach the goal of organizational excellence and encompasses the pursuit of knowledge and skills necessary to effectively accomplish the agency’s mission. To that end, the mission, values, and vision statements should be revisited periodically to ensure that policies, practices, services, and intended outcomes are in accord with tenets articulated in the agency mission.

Quality Assurance and CQI Activities

In many child welfare agencies, quality assurance (QA) case record reviews, along with the collection and review of aggregate data, may be the only or primary components of the States’ continuous quality improvement (CQI) systems. However, an agency's CQI program can and should become much broader to include many activities at many levels where case practice is reviewed, recommendations are made and carried out with the goal of achieving better outcomes for families and children, and data are generated. States should be creative in determining their practice elements that fit this criterion.

Normally, in the QA case review process, individual or paired staff members serve as case reviewers, and they periodically examine a group of selected cases in different areas of the State. They provide feedback about their findings, and this feedback most immediately impacts caseworkers, supervisors, and the next level of management. Over the years, with the implementation of the Child and Family Services Reviews, many agencies have moved from primarily monitoring compliance in their QA case reviews to assessing quality of services and child and family outcomes.

Some agencies with well-functioning QA case review systems may question why an expanded CQI program is needed, now that they have progressed beyond compliance monitoring. It is important to remember that a full CQI system involves analyzing data from case reviews and numerous other sources to identify what is working well and what is not so that the agency can constantly improve practice. With the relatively new focus on CQI in child welfare, agencies are broadening their improvement efforts to permeate every aspect of the agency. Thus, QA case reviews should represent an important, but not necessarily the primary, component of States’ multi-faceted, agency-wide CQI efforts that drive needed changes on a continual basis.

The table below is adapted from Alan Dever's 2003 book, Public Health Practice and Continuous Quality Improvement, and outlines the major differences between QA and CQI:

Differences Between QA and CQI

QA

CQI

is a separate activity

is an integrated activity

is reactive

is proactive

is “top down”

bridges both horizontally and vertically

improves the performance of those whose cases are being reviewed 

improves performance agency-wide

focuses on meeting specific compliance and outcome criteria 

focuses on improving multiple processes and outcomes

measures standards that are established by professionals 

uses fluid, constantly changing standards that are established by stakeholders and consumers working alongside professionals 

is event based 

is based on an ongoing process 

is management focused (directing) 

is employee, stakeholder, and consumer focused (involving) 

involves selected staff and functions

is agency-wide and crosses all functions

Thus, QA uses standards established by professionals that define acceptable or unacceptable levels of performance. As a reactive or retrospective process, it assesses practice that has already occurred. QA results direct the behavior and practice of child welfare practitioners toward improved outcomes “after the fact” through a specific event, such as a scheduled case review with selected staff, that occurs outside of other improvement processes.

QA case reviews are a necessary element of a child welfare agency’s overall CQI system. Although both QA and CQI seek to improve quality, CQI proactively tracks, analyzes, and corrects ongoing, interrelated, and interconnected processes (including QA case reviews) in an effort to constantly improve systems and practices. In CQI, it is these multiple processes and systems, not the performance of specific practitioners, that are the focus of improvement. A comprehensive CQI process sends a strong signal to agency staff, external stakeholders, and consumers that their involvement is crucial to the agency’s continued learning, exploration of new ways of doing things, and improvement.

Consequently, a well-functioning CQI system encompasses a wide range of processes and facilitates the launching of targeted activities to meet identified needs. These may be major initiatives or smaller-scale projects. When data show concerns in specific practice areas or parts of the State, the agency may choose to implement family team meetings in a targeted area, or large-scale statewide initiatives such as trauma-informed care. In another situation, mentoring and coaching frontline staff on higher quality worker-child visits statewide might have a significant positive impact on safety, placement stability, timely permanency, and other areas. Varied, smaller-scale activities can occur concurrently, as long as they are being overseen, assessed, and well-managed through an analysis of the data they are generating. 

Development of Outcomes and Measures of Success

Essential to an effective continuous quality improvement (CQI) system is accurately identifying outcome and systemic areas to be measured and tracked that will assess the status and ongoing progress of an agency’s practices, programs, and services. In its August 27, 2012, Information Memorandum, ACYF-CB-IM-12-07 (available online at: http://www.acf.hhs.gov/programs/cb/resource/im1207), the Children’s Bureau stated that it intends to publish a specific set of monitoring measures in the future. Until those are known, however, concerns that have been identified in a State’s Child and Family Services Review (CFSR) and Program Improvement Plan, unless already sufficiently addressed, are a recommended beginning point for measurement toward desired outcomes.

Generally, CFSR results revealed that all States were challenged in assessing and meeting well-being needs of children and families, as well as achieving timely permanency. Specifically, many States struggle to ensure the social and emotional well-being of children and youth. Improvement in this area can have profound positive effects on several permanency issues that reflect the essence of a State child welfare program. Thus, these should receive strong consideration as areas needing to be tracked and assessed.

Outcomes are key in assessing and refining program delivery and supporting organization-wide quality improvement. When determining outcomes, or intended results of a program or initiative, agencies should strive to measure what works and does not work. In other words, rather than defining an outcome by whether or not a child or parent was the recipient of a practice or service, it should be measured instead by whether or not service participation improved functioning or the chances of success. For instance, rather than measuring success by whether or not a parent completed parent training, success (outcome) could be gauged by whether or not the training improved skills and capacities of that parent, perhaps measured by whether another incident of child maltreatment occurred within a specified period of time.

Outcomes capture the “what” and the “who,” and are written as "change statements." In other words, in defining outcomes, the details of the targeted initiative should be considered, as well as the recipients, intended impact, and change desired. For instance, in attempting to strengthen its youth independent living program, an agency, rather than defining its outcome goal as “prepare youth to live independently,” might consider instead the following as outcomes:

  1. Increased high school graduation rates of youth in foster care, and/or
  2. Decreased instances of youth in foster care being involved with juvenile justice

Since outcomes are broad in nature, performance indicators or measures serve as a bridge connecting intended outcomes and data collected. Measures are specific pieces of information that describe observable – or otherwise captured – characteristics or changes in factors. They are indicators that can be counted, reported, observed, or somehow detailed from data collected.

In composing measures, agencies should first calculate a baseline, or initial data that allow a comparison with subsequent data for assessing impact, and then identify targets, or the level of achievement (quantifiable goals) it hopes to achieve. Measures drawn up should be as simple as possible, while still being meaningful and useful. Performance measures enable an organization to use factual data it has gathered to determine whether its programs, practices, and CQI system as a whole have had a measurable impact on consumers and whether programmatic goals have been met.