The Review Week

Once you have been selected to attend a review, JBS will work with you to make your travel arrangements. You will also receive, 2 to 3 weeks before your scheduled review, all the information you need to prepare to go on site. This information, which will include such material as the Statewide Assessment, the Preliminary Assessment, and relevant State policies, will also be available for you to access at any time on the CFSR Information Portal.

Arriving On Site

In most cases, you will arrive at the review location on the Sunday before the review. You will check into your hotel that evening. The following morning, at around 7:30, you will typically meet the rest of the review team in the hotel lobby and travel together to the review site. There, at the Monday morning team meeting, you will meet your Local Site Leaders, receive your cases, and talk through any logistical issues.

Monday Morning Team Meeting

Once you arrive at your local site, you will participate in the Monday morning team meeting, led by the NRT Local Site Leader. During this meeting, the review team will be introduced, reviewers will meet their review partners, and cases will be assigned. Reviewers will also receive information about the case interviews that were scheduled ahead of time by the State. If there is relevant information about the review specific to that site or that State, it will also be shared here.

Logistical issues will also be dealt with at the Monday morning team meeting. These will include information such as the location of each review pair's workspace, which Local Site Leaders are responsible for handling Quality Assurance, how lunch and dinner will be handled, the time of each evening's nightly debriefing, and so on. At the end of the team meeting, each Site Leader and review pair will receive a tablet PC and supporting equipment and passwords, and the review week will be officially underway.

Day-to-Day Responsibilities

Monday through Wednesday of the review week are when the bulk of the case record reviews, case-related interviews, quality assurance activities, and stakeholder interviews take place. These are long work days, typically beginning at 8 a.m. and often going until late in the evening. Everybody on the review team is expected to participate in all scheduled or assigned activities and to remain on site throughout the work day, unless it is necessary to leave for an interview. Everybody is also expected to participate in each evening’s nightly debriefing.

By Thursday morning, all of the site's OSRIs and SIGs should be completed. The NRT Local Site Leader will have uploaded these records along with the completed Summary of Findings Form to the central server, and together with the other Site Leaders they will begin to prepare for the local site exit conference. Following this conference, reviewers are free to leave the review site and return home. Site Leaders will travel to the site of the statewide exit conference, which is held Friday morning.

Nightly Debriefings

On Monday, Tuesday, and Wednesday evening of the review week, the NRT Local Site Leader will convene a nightly debriefing meeting for the entire local site. The purpose of the nightly debriefing is to go over the day’s activities and to provide a forum that allows each review pair to present a brief, 10-minute summary of the case they completed that day. This summary helps the review team begin to identify themes and trends across the site and also helps Local Site Leaders ensure consistency in outcome ratings among reviewers.

In preparing these case summaries, reviewers should keep in mind that, in addition to having a strict 10-minute time limit, the summary itself should emphasize outcomes, not items. To assist themselves in preparing this summary, reviewers should use the automated Nightly Debriefing Report. When this report is first opened, the automated application will have already filled in some of the general information. However, reviewers will need to complete the remaining information, including details about the case history and ratings. When you are working on summarizing the case ratings, remember that you are synthesizing information from the 23 item ratings to explain the 7 outcome ratings. For example, when considering Safety Outcome 1, this statement:

Safety Outcome 1 was rated as Substantially Achieved because, during the period under review, all accepted reports were responded to in a timely manner (as documented through item 1) and there was no repeat maltreatment (as documented through item 2)

is better than this statement:

The outcome was rated as Substantially Achieved because both items 1 and 2 were rated as strengths.

The second example does not provide enough information about the ratings, while the first provides important details about the specific factors that contributed toward Outcome 1 being rated as Substantially Achieved.

Remember that, for reviewers, the point of the nightly debriefing is to present relevant information about completed cases as concisely as possible and ensure that outcome ratings are consistent across the local site. The purpose of the debriefings is not to educate other team members about all the details of a case or to critique the State’s policies or practices. Rather, you should focus on your findings regarding the actions taken by the State during the period under review. 

In addition to the case summaries, Local Site Leaders who have participated in stakeholder interviews will use the nightly debriefings to briefly summarize the interviews and address the systemic factors that were explored. Again, this allows the review team to begin to identify systemic themes. The entire team will also use each night's meeting to identify problems or concerns regarding schedules, logistical arrangements, instruments, the automation, or any other issues that may have come up over the day.

Thursday Morning Debriefing

By Thursday morning, all case record reviews and stakeholder interviews must be finalized and the instruments transferred to the NRT Local Site Leader's computer. The NRT Local Site Leader will use information from a variety of reports to complete the Summary of Findings Form and will ensure that this and all of the site's records are uploaded to the central server.

Afterward, the NRT Local Site Leader will convene a meeting of the entire local site review team. Like the nightly debriefings, this meeting is intended to provide the entire team a chance to review that site’s findings, focusing specifically on themes and trends that have emerged across cases and interviews. However, unlike the nightly debriefings, which focus only on outcome summaries for each case completed that day, the Thursday morning debriefing involves an item-by-item review of the entire site’s findings. As each item is discussed, the review pairs and Site Leaders will be invited to share relevant information of interest from either the case record review or stakeholder interview.

The Thursday Morning Debriefing will generally last until lunchtime. It is followed by the Thursday Local Site Exit Conference.

Thursday Local Site Exit Conference

Early on Thursday afternoon, following the Thursday morning debriefing, the NRT Local Site Leader will convene the local review team for a local site exit conference. During this conference, the NRT Local Site Leader will share with the local site a preliminary report on the issues and trends identified for that site. This preliminary report will include a description of the strengths and areas needing improvement identified at the local site.

In addition to the local site's review team, attendants at this exit conference may include caseworkers, supervisors, local administrators, agency staff, and other stakeholders. Following the local site exit conference, the reviewers are dismissed to return home. Site Leaders travel to the location designated for the statewide exit conference on Friday.

Friday Statewide Exit Conference

After the local site exit conference on Thursday, reviewers are dismissed to return home while Site Leaders travel to the location designated for the statewide debriefing and exit conference. These events begin on Friday morning, when the remaining review team comes together for the final statewide debriefing facilitated by the NRT Team Leader. By this time, all the finalized instruments and reports from the local sites will have been uploaded to the central server, and the preliminary data and findings from the three review sites will have been reviewed and discussed by the NRT Local Site Leaders for inclusion in the statewide exit conference presentation, which normally takes place in the afternoon.

At the exit conference, the NRT Team Leader delivers a presentation on the review findings for the State, using a PowerPoint presentation. There will be discussion at the conference of issues that may require resolution during the Final Report development process and a brief overview of the next steps in the review process. These steps include the preparation and submission to the State of the Final Report and the development and monitoring of a Program Improvement Plan.

Once the statewide exit conference concludes, the review week comes to an official close.