Implementing and Sustaining Systems Change

Once a well-functioning continuous quality improvement (CQI) system is in place, it will provide information and data that can be used to identify the State child welfare system’s strengths and weaknesses. This information is the foundation for a strong strategic planning process. Child and Family Services Plans are developed through the State’s strategic planning process, and the elements of the Plan should be informed by data from the CQI system and  feedback from staff, consumers and external stakeholders.

The process for effective systems change has been the subject of much research and scholarly writing in recent years. During Rounds 1 and 2 of the Child and Family Services Reviews (CFSRs), many States adopted ambitious Program Improvement Plans, yet in spite of commendable efforts, the intended improvements were not always realized. The research on effective systems change is instructive as we look to more effective systemic improvement in child welfare.

Achieving positive change requires a thorough assessment of the agency’s strengths and weaknesses based on comprehensive data. From this analysis, goals are identified and appropriate interventions selected. The interventions must then be fully implemented. A strong intervention that is inadequately implemented will not have the intended result.

A relatively new field of research, implementation science, has evolved around the study of the process of implementing new or improved practice innovations or programs. A specific knowledge base has emerged, articulated in various models, that applies broadly to many industries and settings. The models identify and describe proven, research-based steps for properly implementing new programs and systemic changes. Agencies can implement new initiatives in more sound ways by drawing on this rich body of implementation research in order to develop an Implementation Framework. A well-developed framework, in turn, can be a critical element in sustaining system change.

Implementation Framework

The Children’s Bureau has developed a three-phase model for system improvement in consultation with the National Implementation Research Network (NIRN). The three phases are:

  • Foundational Phase:  During this phase, an implementation team is formed, data are analyzed and decisions are made about what goals and initiatives will be the focus of the systems change.
  • Planning Phase:  This is the phase in which implementation is planned, needed infrastructure is developed and monitoring and feedback loops are designed.
  • Action Phase:  During this phase, the plans are executed and the implementation team is engaged in continuous monitoring and improvement of the change effort.

For more information on NIRN, visit http://nirn.fpg.unc.edu/.

It is important to remember that the major activities of practice and system improvement are ongoing and cyclical. One activity shapes and leads to the next, although steps are not always sequential; sometimes agencies must “backtrack” to a previous activity to re-evaluate, adjust, and even complete (or partially complete) the activity again.

Note that the implementation steps discussed in the links below relate to new practice innovations, although the concepts of implementation research have comparable application to the implementation of systemic improvements to programs and infrastructure, such as continuous quality improvement.

Note also that there are a number of models for implementing effective systems change. The Systems Change section of the Additional Resources section contains several resources for further reading. 

Foundational Phase

The foundational phase of an implementation framework includes the following elements:

  • Implementation Team
  • Assessment
  • Goal-Setting
  • Selection of Strategies
  • Readiness

Details on each element are provided below.

Implementation Team: The strategic planning process begins with the establishment of an implementation team, similar to the CQI teams discussed elsewhere in this module. The implementation team should have the authority, skills and support to lead the change effort. The team should include key staff, courts, Tribes and external stakeholders. The Team is empowered by leadership to guide the change process from the beginning through full implementation.

The implementation team enlists support for the strategic improvement process from internal and external stakeholders and develops a communication plan for bi-directional communication with stakeholders throughout the process.  Communication and collaboration with stakeholders should begin with the development of the agency’s vision and mission, and include review of the data and assessment of agency strengths and concerns, selection of priority areas for the change effort, identification of strategies and assessment, and adjustment of strategies throughout the implementation period.

Assessment: The purpose of this step is to use data to get a precise picture of strengths, needs and challenges. Change efforts that begin with a well-completed, thorough assessment phase have been shown to have a much higher probability of success. Data from multiple sources should be used to assess need, identify cross-cutting issues, and determine goals for the State. By “drilling down” in the data, or examining data at deeper and deeper levels to identify patterns, needs can become more clear and issues and target populations can be identified.

For example, in a situation where a State’s close examination of placement data reveals that older foster youth have more placements than younger children, the agency should ask “Why?” and proceed from there to answer questions such as the following:

  • What placement types show the most/least transition?
  • Are there enough and the right type of placement resources?
  • At what ages did older children come into care?

Often times, the answer to one question will lead to another question. Exploration of each question leads to a more complete understanding of the issue and possible solutions.

Goal-setting: After more questions are asked and answered, the next steps should involve pinpointing needs, identifying goals and desired outcomes to know what success will look like, and defining measures of success. As more information becomes apparent through deeper data analysis, a multi-faceted initiative may be developed with the goal of increasing placement stability for older youth in care.

Goals, both long-term and short-term, provide motivation and direction for agencies to be more focused and productive. States should use data on an ongoing basis to pinpoint areas of concern or poor outcomes for families and children, analyze why these poor outcomes may be occurring, and then set goals for what the outcome or outcomes should be. States can begin by asking the following general questions:

  • What changes will most improve outcomes for children and families?
  • What do we, as an agency, need to accomplish?
  • What are the most critical things to change initially?

An agency should attack its highest priority first, selecting the goal that will have the biggest impact on enhancing outcomes within that priority area. Then, when initiatives are identified or in place to achieve that goal, it should move on to its next prioritized goal. For instance, an agency may have a need to reunify more children in its care within 12 months, as well as a need to increase safety of children in care. While both goals are important, the agency’s priority goal would likely be to improve safety of children in care. At some point after work has begun toward meeting the safety goal, the agency can begin working to enhance reunifications.

Goals selected should be measurable, attainable, and relevant. Furthermore, they should correlate with and more clearly define the agency’s mission, values, and vision, where the agency is headed, and how it will get there.

Selection of Strategies: Once it has established the goals that it wants to address, the agency and its implementation team must begin the process of strategically identifying and planning those initiatives or innovations required to accomplish these goals. As a general policy, no more than three major innovations should be implemented with overlapping activities during the same time period. By the time the priority innovation begins full implementation, the agency may be in the assessment or planning stage of its next initiative. If an agency attempts to drive too many changes simultaneously, field staff may become overwhelmed, experiencing increased stress, competing priorities, and more difficulty coping with change. From a list of many possible initiatives, leaders should choose those that will have the greatest impact and improve multiple child and family outcomes.

For example, an agency may choose to implement a trauma-informed system of care. If successfully implemented, those practice changes should result in progress toward meeting multiple goals of children in care, such as fewer moves, enhanced connections, greater educational success, and diminished behavioral or mental health issues. In selecting specific innovations, research should be done on current evidence-based or evidence-informed practice, or practice and services where effectiveness, in terms of meeting outcomes, has been demonstrated.

Note that the California Evidence-Based Clearinghouse for Child Welfare (http://www.cebc4cw.org/) maintains a list and descriptions of evidence-based and evidence-informed child welfare practices to help inform and guide agencies in their selection of practice interventions.

Once potential innovations have been identified, States should determine which are a good fit for the agency, the communities and areas being considered, the State as a whole, and the values of culturally diverse groups. Agencies should ask themselves practical questions, such as:

  • How will the innovations fit with other initiatives already underway?
  • Can the State afford to implement and sustain this initiative?
  • Are training, technical assistance, and coaching available to support implementation of the innovation?
  • Does the State have the infrastructure to support the proposed innovation?

An example of an ill-fitting initiative might be a largely rural state that wanted to implement an evidence-based home visitation program using nurses despite a chronic shortage of nurses in many parts of the State.

Readiness: The purpose of this step is to identify the technical assistance required to implement each strategy and to ensure before beginning that there is internal and external support for this effort. 

Once a strategy has been selected, the next step, which is critically important, is assessing readiness for implementation and gaining buy-in. The implementation teams and agency leadership should ensure that there is support among stakeholders about the issues, concerns, and needs. The implementation team and leadership should share data with stakeholders, get their input, and use that input in planning. Stakeholder feedback, as well as adjustments based on that feedback, should be continual throughout the process. If internal and external stakeholders do not support and embrace the identified issue, they will not be an effective part of the solution. Carefully and thoroughly determining readiness for a project prior to implementation dramatically increases the chances of its success.

Another critical step will be obtaining training and technical assistance. The State will want to ask practical questions, including:

  • Do we have the resources to access training, coaching, and technical assistance, initially and on an ongoing basis?
  • What kinds of training and technical assistance are needed and who are the potential experts in this innovation?
  • Who will assist in evaluating the implementation and outcomes of the innovation?

Planning Phase

Before implementation of a new initiative can begin, agency leadership and the implementation team must collaborate to develop an implementation plan. This plan will serve as a roadmap to guide the project. Decision-makers must decide on, and plan for, the specifics of how the new initiative will unfold, including:

  1. A rollout plan, or the scope of activities
  2. How rollout activities will be sequenced
  3. Whether specific areas (e.g., transformation zones or innovation counties) will be designated for rollout
  4. How areas will be selected
  5. Projected date for full implementation
  6. Timelines for the various activities

The implementation plan will be a living document that reflects necessary and perhaps unanticipated changes and adaptations made as activities progress, and as continuous quality improvement (CQI) teams learn by doing and feed information back to the planners. The plan will ultimately reflect the details of the project’s implementation and ongoing adjustment process.

Other elements of the planning phase include supports for implementation and feedback loops for communication, monitoring and improvement.

Supports for Implementation: The purpose of this step is to put in place the supports and infrastructure for the new initiatives and to develop methods for measuring progress.

Decision-makers must further think through other organizational supports, including staffing, stakeholder and community support, training, space, equipment, funding, supervision, policies, and data processes. In determining staffing levels needed, staff expertise required, and other issues, leadership should ask questions such as:

  • What qualifications should be required of staff?
  • What level of staffing will be needed, and are resources available to support increased levels?
  • What coaching activities will be needed, and for how long and by whom?
  • Is funding available to support all facets of the project initially and in the long term?
  • Is there space available for staff and related activities, as well as technology, equipment, and supplies
  • What policies need to be changed or developed to support the project?

For each strategy, appropriate supports must be put in place to assure successful implementation of the initiative.

Feedback Loops for Communication, Monitoring and Improvement:  The purpose of this step is to update communication protocols, develop progress measures and establish feedback loops that will provide information on whether the intervention is operating as intended and having the desired impact on outcomes.

The implementation team should review and update the communication plan to assure that effective processes are in place to communicate findings and progress and obtain ongoing feedback from agency staff as well as consumers and external stakeholders, including a process for reporting any barriers to implementation and the plan for addressing those barriers.

A key part of the implementation plan at this stage is determining the quantitative and qualitative data needed for assessment and evaluation of project implementation and effectiveness, as well as clear measures of the progress of the initiative. Measurement of implementation and effectiveness includes both process measures (Did training occur as planned? and Is coaching ongoing?) and outcome measures (Is placement stability improving with the target population in the innovation county?).

It is essential that the project be implemented as it was intended, or with fidelity and faithfulness to the model. Assuring that changes are implemented properly can be just as important as determining whether the new initiatives are effective. If an agency veers from the intended project design in its implementation, then results and outcomes will be less predictable, may be inconsistent, and likely will not be sustainable.

Action Phase

The action phase of an implementation framework includes the following elements:

  • Implement, Monitor, and Adjust Interventions
  • Improve and Adjust Interventions
  • Scale-up

Details on each are provided below:

Implement, Monitor, and Adjust Interventions: The purpose of this step is to fully execute plans, review the data on progress of implementation and impact of the interventions, and make adjustments to improve outcomes.

During this stage, the plans are fully executed, including new procedures, guidelines, and practices. The implementation team is gathering information on how implementation is progressing, and is asking important questions such as “Is the model being implemented as intended (with fidelity)?” and “Are additional supports, like training and technical assistance, needed?”

Improve and Adjust Interventions: The purpose of this step is to assess whether the intervention is effective and to make adjustments, as necessary.

Based upon the initial effectiveness of the innovation and other staff and stakeholder feedback, adjustments will be made by the agency to improve the impact of the innovation, eliminate barriers, and increase fidelity. For instance, an adjustment might be needed when information reveals that caseworkers in a new initiative are having to work a great deal in the evenings, but agency policies do not allow for staff to work flexible hours. The State could implement flex-time policies for front-line staff before rolling the initiative out to other areas.

Data from tracking and monitoring the activities and results of the initiative should be reviewed on a regular basis. Ongoing assessment and analysis of findings will validate effective practice, identify trends and needs, and allow strategies to be developed to address any challenges. This ongoing, careful analysis will enable an agency to refine or adjust processes and practice, on a continual basis, in ways that enhance both implementation and effectiveness.

Scale-up: The purpose of this step is to determine when an intervention is ready for expansion and to plan and implement this expansion with necessary supports in place. Leadership and the implementation team will decide, based on information and data gathered, when the intervention is ready for expansion, and how it should be expanded. A realistic process is needed that outlines the steps to ensure that sufficient capacity has been developed to support the intervention in each new site. Decision-makers should ask questions such as the following:

  • What should the pace of the expansion look like?
  • How will training and technical assistance be provided to each site?
  • Are systems and resources in place to support expansion to the next site?
  • Is communication in place to prepare sites for implementation, and are communication and peer support available between sites?

Sustaining System Change

Full implementation of a new initiative can take from 2 to 4 years. Critical to the process overall is for the agency to ensure sustainability. Planning for long-term sustainability must begin during the strategic planning stage and continue throughout the process. Agency leadership should ensure that funding streams remain available, that staff, external stakeholders, and consumers continue to be involved, that goals are being appropriately worked toward, that all sites are maintaining fidelity to the intervention’s design, and that there is progress toward meeting goals.

Articulating the connections between new behaviors and improved outcomes can be a powerful tool in assuring staff and partners of progress. For example, if moves of children in care are diminishing due to more frequent and better quality worker/child/caregiver visits, as anticipated, support and enthusiasm from staff and partners involved in the innovation will be bolstered.

The innovation, in all its various stages, will need to be fully integrated into the State’s systems. This includes ongoing training, regulations, policies and procedures, and, most importantly, the agency-wide continuous quality improvement (CQI) system. The new practice should be incorporated into the case record review process and data system. Bi-directional stakeholder communication, or feedback loops, should then continue through the CQI process, as should ongoing analysis, assessments, and improvements. The emphasis of CQI on data, expedient diffusion of best practices, and ongoing, cyclical improvement can then continue to guide and strengthen the implementation of the agency’s various initiatives.