The Onsite Review Instrument

The Onsite Review Instrument (OSRI) is the tool that reviewers use to collect information during their case record review. Its structure is organized into a Face Sheet, which is used to document general case information, and three sections that correspond to the outcome domains of safety, permanency, and child and family well-being. Each outcome domain is further divided into individual outcomes (for example, Safety Outcome 1 and Safety Outcome 2), which are themselves divided into individual items that relate to the outcome.

The OSRI is used to rate both in-home and foster care cases. For in-home cases, the safety and well-being sections are completed for all the children in the family, and the permanency section is not applicable. For foster care cases, the safety section is completed for all the children in the family, but the permanency and well-being sections are completed only for the target child. In both instances, though, reviewers must distinguish between events that took place over the life of the case and events that took place during the period under review.

Each review pair completes one OSRI per case assigned, assessing and rating items based on information and standards provided in the instrument instructions. They draw equally from two information sources to complete the instrument: documentation from the case record itself, and case-related interviews with children, parents, foster parents, caseworkers, service providers, and other professionals knowledgeable about the case. As the reviewers complete the instrument, item and outcome ratings are assigned and rating documentation must be provided to support those ratings.

It is essential that reviewers become thoroughly familiar with the entire OSRI before arriving on site for the review week. While there are some differences between the layout of the paper instrument and the automated OSRI that will be used on site, learning the paper instrument will provide the foundation of knowledge that reviewers require to work efficiently while on site.

Structure of the OSRI

The hard copy of the OSRI contains a few elements that do not appear in the automated instrument. These elements include General Instructions at the beginning of the instrument and some slightly different formatting in a few of the questions. For the most part, though, the automated instrument and hard copy are identical.

The first part of the OSRI is the Face Sheet, which lists basic information about the case being reviewed, such as the target child’s name, the names of other children involved in the case, the reason for agency involvement, key dates of service, and individuals interviewed during the case review. These individuals are not identified by name on the Face Sheet, but rather by their relationship to the case.

Following the Face Sheet is the main body of the instrument, which is divided into three main sections that are organized by outcomes. These sections, or outcome domains, form the basis of the CFSRs: safety, permanency, and child and family well-being. For each outcome, reviewers collect information on a number of items related to that outcome. The instrument has a total of 7 outcomes and 23 corresponding items organized in the following manner:

  • There are two outcomes and four items under the safety section
  • There are two outcomes and twelve items under the permanency section
  • There are three outcomes and seven items under the child and family well-being section

Each item is identified by an item number and the area that the item assesses. Each item also has a Purpose of Assessment, which clearly identifies the purpose of the information being collected for the item. Each item’s Definitions and Instructions are incorporated into the individual questions. These instructions are very detailed and specific, and are intended to help clarify complicated issues and assist reviewers in making correct decisions regarding answers to each question.

In addition to the outcomes and item questions, the OSRI also includes Rating Documentation questions that must be completed by the reviewers before the case can be considered complete. These rating documentation questions, which include a Main Reason statement and various follow-up questions, are used to provide an explanation and sources of information that justify each item's and outcome's rating.

The Face Sheet

The Face Sheet is the first part of the OSRI. It is used to document general information about the case, such as the case type, the names of the children, the target child, the date the case was opened, and so on. It must be completed regardless of whether the case is a foster care or in-home services case.

The Face Sheet is one of the only parts of the instrument where full proper names can be used. Items 1 and 12 also have questions that require the input of a child's name, but in both of those cases only the child’s first name should be used. No surnames should ever appear anywhere in the instrument except on the Face Sheet, and for the remainder of the instrument (excepting Items 1 and 12) all proper names should be replaced with titles. Examples include "biological mother," "target child," "caseworker," "adoption agency," and so on.

Note that, unlike the items that make up the bulk of the instrument, there is no Rating Documentation attached to the Face Sheet. The Face Sheet itself is not a rated item, and as such should not really be considered as an “item” in the instrument at all.

Note also that there are a few differences between the paper and automated versions of the Face Sheet. Questions A through E, for example, exist only on the paper version. The chart in the electronic version of Question F has more columns than the paper copy does, and there are also follow-up questions in the electronic version (Questions K1, L1, M1, and M2) that do not exist in the paper version.

 

The Outcomes

The OSRI is divided into three sections that are themselves organized around the three outcome domains that form the basis of the CFSRs: Safety, Permanency, and Child and Family Well-Being. Each outcome domain is divided into specific outcomes: Safety Outcomes 1 and 2; Permanency Outcomes 1 and 2; and Well-Being Outcomes 1, 2, and 3. For each outcome, reviewers collect information on a number of “items” related to that outcome. Each item focuses on a specific Purpose of Assessment, and is further subdivided into individual questions.

Once all of an item’s questions are answered, the application will automatically rate the item as either a Strength, an Area Needing Improvement, or Not Applicable. When an item has been rated, the reviewers must complete its Rating Documentation questions. These questions provide evidence and contextual support for the item rating. When all of an outcome’s items have been rated and its Rating Documentation completed, the outcome itself receives a rating. Once all items and outcomes have been rated, and all supporting documentation is included, the instrument is complete.

Note that while reviewers will use the OSRI to review both foster care and in-home services cases, they will complete the Permanency section only if the case under review is a foster care case. For children in foster care, reviewers should consider both Safety Outcomes (items 1 through 4) for all children in the family, but complete the Permanency Outcomes (items 5 through 16) and the Well-Being Outcomes (items 17 through 23) only as they apply to the specific child whose case is under review.

For children receiving in-home services, reviewers should apply the Safety and Well-Being Outcomes to all the children in the family who are both residing with and included in services to the family.

Safety

The safety domain is divided into two separate outcomes: Safety Outcome 1 and Safety Outcome 2. Safety Outcome 1 is "Children are, first and foremost, protected from abuse and neglect." It is composed of item 1 and item 2.

Safety Outcome 2 is "Children are safely maintained in their homes whenever possible and appropriate." It consists of item 3 and item 4.

In general, the Safety domain is concerned with the following questions:

  • Did the agency respond quickly to reports of child abuse and neglect and take immediate steps to protect the children in the home?
  • Once involved, did the agency make sure that children were not abused or neglected again?
  • Did the agency provide services to make sure that children don’t enter or re-enter foster care?

Item 1

Timeliness of Initiating Investigations of Reports of Child Maltreatment

Purpose of Assessment: To determine whether responses to all accepted child maltreatment reports received during the period under review were initiated within the timeframes established by agency policies or State statute, and face-to-face contact with the child was made within those timeframes.

Item 1 investigates the agency’s timeliness of initiating investigations of reports of child maltreatment. We use it to determine whether all accepted reports during the Period Under Review:

  • were initiated within timeframes established by agency policy or State statute, and
  • involved face-to-face contact with the child during those timeframes.

An important tip to keep in mind for item 1 is:

  • This is the only item where the state’s policies are the criteria used to assess the entire item.
     

Item 2

Repeat Maltreatment

Purpose of Assessment: To determine if any child in the family experienced repeat maltreatment within a 6-month period.

Item 2 investigates repeat maltreatment. it is used it to determine whether there were two or more substantiated maltreatment reports within a single 6-month period. At least one of the reports must have occurred during the PUR and involve similar circumstances.

So, if there is at least one substantiated or indicated report during the PUR, you look to see if there are any other reports received and substantiated within 6 months of that first report. Please note that the second report could have occurred prior to the PUR. This is one of the few items where you are asked to look outside the PUR.
 

Item 3

Services to Family to Protect Child(ren) in the Home and Prevent Removal or Re-Entry into Foster Care

Purpose of Assessment: To determine whether, during the period under review, concerted efforts were made to provide services to the family to prevent the child(ren)’s entry into foster care or re-entry after a reunification.

Item 3 looks specifically at whether or not the child welfare system made concerted efforts to provide services to:

  • prevent the removal of a child from his or her home, or
  • prevent a re-entry into foster care following reunification

Some important tips to keep in mind for this item are:

  • it focuses on services
  • you must consider the entire PUR, even if children came in or out of care

This item also examines whether looks at if there was an emergency removal, and if so, whether it was necessary to ensure safety
 

Item 4

Risk Assessment and Safety Management

Purpose of Assessment: To determine whether, during the period under review, the agency made concerted efforts to assess and address the risk and safety concerns related to the children in their own homes or while in foster care.

In item 4, we are determining whether all risk and safety issues have been identified and addressed for a child, regardless of whether that child is at home or in foster care.

We are assessing the child welfare system’s efforts to both:

  • assess safety and risk both initially and during the Period Under Review, and
  • address identified safety and risk needs for the child initially and during the Period Under Review.

Keep in mind this distinction between risk and safety:

  • Risk is the likelihood that a child will be maltreated in the future.
  • Safety refers to imminent danger to the child.

An important tip to keep in mind is that assessments and plans need not be formal. For example, even if your State uses a specific document to assess risk and safety, but such a document is not in the case file, yet, case notes and interviews may provide evidence that informal assessments were made regarding risk and safety. This evidence may include child observations, discussions with the child and caregivers, and observations of surroundings. These informal assessments are acceptable provided you and your partner determine that they have adequately identified and addressed risk and safety issues.

Note, though, that you must not only decide on the adequacy of the assessments, but also how effectively risk and safety concerns were managed during the PUR. In other words, were services effective in mitigating risk and safety issues?

Permanency

The permanency domain is divided into two separate outcomes: Permanency Outcome 1 and Permanency Outcome 2. Together, they contain 12 performance items.

Permanency Outcome 1, "Children have permanency and stability in their living situations," consists of item 5, item 6, item 7, item 8, item 9, and item 10. For Items 7 through 10, please remember this very important point: Item 7 assesses the appropriateness and timeliness of the goal or goals, while Items 8, 9 and 10 assess achievement of said goals.

Permanency Outcome 2, "The continuity of family relationships and connections is preserved for children," consists of item 11, item 12, item 13, item 14, item 15, and item 16.

In general, the permanency domain is concerned with the following questions:

  • Did the agency make good decisions to return a child to parents and provide services to prevent re-entry?
  • Is the child in a stable placement now, and how many placement changes did the child experience? If appropriate, was the child placed in the same foster home as his or her siblings? Was relative placement explored, and did it happen?
  • Were a permanency goal and all subsequent goals established in a timely manner, and were the goals appropriate?
  • Did the agency make concerted efforts to achieve the goal?
  • Was the child placed close enough for parents to have ongoing contacts? Did the agency make sure that visits occurred frequently enough?

Item 5

Foster Care Re-Entries

Purpose of Assessment: To assess whether children who entered foster care during the Period Under Review were re-entering within 12 months of a prior foster care episode.

Item 5 looks at re-entries into foster care, so the main question is whether a child who entered care during the PUR is entering again within 12 months of discharge from a prior foster care episode.
If there was a re-entry, you’re asked to determine if the agency made concerted efforts to prevent this. If the child entered care prior to the PUR and remained in care during the PUR, then this item is always NA.

Like repeat maltreatment, if the child entered foster care during the PUR, you look 12 months before and after that entry and see if there is a second entry into care. This is another instance where you have to look outside of the PUR.

Item 6

Stability of Foster Care Placement

Purpose of Assessment: To determine if the child in foster care is in a stable placement at the time of the onsite review and that any changes in placement that occurred during the Period Under Review were in the best interest of the child and consistent with achieving the child’s permanency goals.

Item 6 addresses two issues. First, it explores whether there have been any changes in placement during the PUR, as well as the reasons for those changes. Second, it examines whether the current placement is stable.

For this item, you need to remember that moves up to higher levels of care because a child’s mental/behavioral needs increase are not necessarily seen as moves that further the child’s case goals; rather, in this situation, you must explore why the child’s mental/behavioral health needs are increasing, whether or not the child is being placed in appropriate settings, the commitment of caregivers, and so on.

Some important tips to keep in mind for item 6 include the following:

  • Stability and instability are defined around the likelihood of an unplanned disruption in the foreseeable future. A foster home placement is, by design, not permanent, and a move from a foster home may or may not be considered as contributing to the achievement of the child’s case goals. It depends on the circumstances. For example, if a child is moved to be placed with a sibling or into a relative home, you would probably consider that as being in the child’s best interest and furthering case goals. However, if the foster parents requested that the child be moved because they couldn't cope with the child's behavior, then you probably would not consider the move as furthering the child’s case goals.
  • If a youth in care is held in detention in a locked facility, this does not count as a placement setting.
     

Item 7

Permanency Goal for Child

Purpose of Assessment: To determine whether appropriate permanency goals were established for the child in a timely manner.

For item 7, we need to determine:

  • what the permanency goal is,
  • whether the goal is appropriate, and,
  • whether it was established in a timely manner.

We also look at the ASFA requirement for filing for TPR. How you answer this item will determine how you complete items 8, 9, and 10. For example, if you identified concurrent goals of reunification with parents and adoption, you would complete items 8 and 9 but not item 10.

An important tip to remember for item 7 is:

  • This item does not consider whether or not the goal was achieved. Achievement is measured in items 8, 9, and 10.

Item 8

Reunification, Guardianship, or Permanent Placement with Relatives

Purpose of Assessment: To determine whether concerted efforts were made, or are being made, during the Period Under Review, to achieve reunification, guardianship, or permanent placement with relatives in a timely manner.

Item 8 determines whether the goal of reunification, guardianship, or permanent placement with relatives was achieved in a timely manner. We look at what kind of concerted efforts were made to achieve the goal, how any barriers were addressed, etc. Generally, for a reunification, we’re looking at a 12 month time period to achieve reunification. However, note that this is a general time frame, and sometimes, depending on the case circumstances, a goal of reunification, guardianship, or permanent placement with relatives should or could have occurred sooner. The instrument instructions will guide you in determining this.

Item 9

Adoption

Purpose of Assessment: To determine whether, during the Period Under Review, concerted efforts were made, or are being made, to achieve a finalized adoption in a timely manner.

Similarly to item 8, item 9 is used to determine whether the established goal of adoption was achieved in a timely manner. For item 9, we’re looking at a time frame of achievement of a legally finalized adoption within 24 months of the child coming into care.
 

Item 10

Other Planned Permanent Living Arrangement

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to ensure:

  • That the child is adequately prepared to make the transition from foster care to independent living
  • That the child, even though remaining in foster care, is in a “permanent” living arrangement
  • That the child is in a long-term care facility and will remain in that facility until transition to an adult care facility

Item 10 is used to determine what efforts are being made to achieve permanency for children with this particular permanency goal.

This goal may not be specified in the written case record using the specific term OPPLA, as some states use different terminology, like Independent Living, Emancipation, etc. This goal refers to a situation in which the State maintains care and custody responsibilities for the child, but places the child in a setting in which the child is expected to remain until adulthood, such as with foster parents who have made a commitment to care for the child permanently or with relatives who have made the same commitment.

In this item, you’ll be looking for formal or informal agreements around the goal of “other planned permanent living arrangement.” If a State has a policy that a signed agreement or court order is necessary to validate a planned permanent living arrangement for a child, then the presence or absence of that agreement would be noted in the instrument. If no signed agreement is required by the State, then reviewers would need to validate the living arrangement through information in the case file and/or through interviews.

There are two important tips to keep in mind for item 10:

  • This item also assesses whether or not the child has been provided with appropriate independent living services.
  • This is the second—and last—instance of where a State’s policy would be considered in the item’s rating.

Item 11

Proximity of Foster Care Placement

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to ensure that the child’s foster care placement was close enough to the parent(s) to facilitate face-to-face contact between the child and the parent(s) while the child was in foster care.

Item 11 is used to assess whether the location of the child’s foster care placement makes it possible for his or her parents to visit.

Some tips to keep in mind for item 11 include:

  • Generally, a travel distance of less than 1 hour is considered close enough to facilitate face-to-face contact.
  • If the parents live separately from each other, this item is assessed using the residence of the parent with whom it’s most likely that the child will be reunified.

Item 12

Placement with Siblings

Purpose of Assessment: To determine if, during the Period Under Review, the agency made concerted efforts to ensure that siblings in foster care are placed together unless a separation was necessary to meet the needs of one of the siblings.

Item 12 asks whether all of the siblings in foster care were placed in the same home. If they were not, it explores the reason.

Keep in mind these tips for item 12:

  • A lack of resources is not justification for separating siblings, unless there is a large group of five or more children.
  • One valid reason for separating siblings in foster care would be to place them with different paternal relatives.  
  • Siblings may be separated to provide specialized services, but concerted efforts must be made to reunite them once those services are no longer needed.
  • This item looks specifically at siblings placed in foster care. Siblings who are not in foster care are addressed in a later item.
     

Item 13

Visiting With Parents and Siblings in Foster Care

Purpose of Assessment: To determine if, during the Period Under Review, the agency made concerted efforts to ensure that visitation between a child in foster care and his or her mother, father, and siblings is of sufficient frequency and quality to promote continuity in the child’s relationship with these family members.

Item 13 assesses the frequency and quality of visitation between the child in foster care and his or her parents and siblings placed separately in foster care.

A tip to keep in mind for item 13 is:

  • It refers only to siblings placed separately in care from the target child during the PUR.
  • If visitation did not occur, then the quality of that visitation must be recorded as NA.

Item 14

Preserving Connections

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to maintain the child’s connections to his or her neighborhood, community, faith, language, extended family, tribe, school, and friends.

Item 14 looks at the efforts that were made by the agency to maintain and reinforce the personal connections that are important to the child when he/she came into care, such as neighborhood, school, friends, faith, etc. This item also assesses compliance with ICWA requirements.

Connections to parents or siblings separated in care should not be included in this item; that’s assessed in other items. However, this is where you would address connections with siblings not in care, such as an adult brother or sister.

An important tip for item 14:

  • If a child has been in care for a considerable length of time, connections maintained or not maintained to previous foster parents or foster siblings would be assessed in item 17A. Item 14 focuses only on connections the child had prior to coming into care.
  • Question 14B, which collects information about Tribal membership, does not affect the item’s rating.

Item 15

Relative Placement

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to place the child with relatives when appropriate.

Item 15 asks whether the child was placed with maternal or paternal relatives. If he or she was not, it explores the reason.

Here are some important tips to keep in mind for item 15:

  • Assessments of relatives can be both formal and informal.
  • This item, like most, explores placement or efforts during the PUR.
  • Even if a State has a formal assessment, an informal assessment is adequate to rate item 15 as a Strength.

A difficult question to answer for this item is: at what point is it okay for the agency to stop looking for relatives altogether? This, of course, will depend on the circumstances of the case, and will be something you will need to discuss with your peer reviewers and Team Leaders. Many times, it is appropriate to periodically attempt to locate or reassess relatives, even if they were initially determined to not be appropriate.

Item 16

Relationship of Child in Care with Parents

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to promote, support, and maintain positive relationships between the child in foster care and his or her parents or primary caregivers through activities other than visitation.

Item 16 is used to determine what else the agency did, other than facilitate visitation, to help the child in foster care maintain positive relationships with his or her parents or primary caregivers. In other words, did the agency include the parents in the child’s medical appointments, school activities, and special events?

Keep in mind the following tip for item 16:

  • Even though the instrument instructions are clear, it’s fairly common for reviewers to include parent/child visitation in this item, even though that has already been assessed in item 13. Remember that item 16 refers to activities other than visitation.
     

Well-Being

The well-being domain is divided into three separate outcomes: Well-Being Outcome 1, Well-Being Outcome 2, and Well-Being Outcome 3. Together, they encompass seven performance items.

Well-Being Outcome 1 is "Families have enhanced capacity to provide for their children's needs." it consists of item 17, item 18, item 19, and item 20. Item 17 is itself divided up into four separate sub-sections: 17A, 17B, 17C, and 17 overall.

Well-Being Outcome 2 is "Children receive appropriate services to meet their educational needs." It is the only outcome in the instrument that has only one item, item 21

Well-Being Outcome 3 is "Children receive adequate services to meet their physical and mental health needs." It consists of item 22 and item 23.

In general, the Well-Being domain is concerned with the following questions:

  • Did the agency do a thorough assessment of the needs of the child, family, and foster family, and provide the services necessary to ensure the child’s well-being?
  • Did the agency make sure that the child’s physical, educational, and mental health needs were met?
  • Were the child and the family really involved in case planning?
  • Did the caseworker meet often enough with the child, parents, and foster family to ensure that the child was safe and that everyone was working toward the goal?

In-Home Cases and Items 21-23

It is important to understand that item 21 (Well-Being Outcome 2), and 22 and 23 (Well-Being Outcome 3) are not necessarily applicable for in-home cases. The instrument gives specific instructions for when these items will be applicable for in-home situations; the items are applicable if educational, physical health, or mental health issues were one of the reasons the case came to the agency’s attention, or if these issues presented while the case was open and the caseworker would reasonably be expected to be involved. So let the instrument be your guide in determining if these three items are applicable for in-home cases.

Records for Items 21-23

Even if State policy requires that educational, medical, dental, or mental health records be kept in the case file, if you can determine through your case-related interviews or the case record that the child’s needs were met, the actual records are not required in order to rate these items as Strengths.
 

Item 17

Needs and Services of Child, Parents, and Foster Parents

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to assess the needs of children, parents, and foster parents at the child’s entry into foster care and on an ongoing basis to identify the services necessary to achieve case goals and address the issues relevant to the family, and whether the agency provided the appropriate services to address the identified needs.

Item 17 is divided into three sections. Section 17A addresses needs assessment and services to children, 17B addresses needs assessment and services to parents, and 17C addresses needs assessment and services to foster parents. For each section, we need to determine whether initial and ongoing needs assessments were conducted and whether appropriate services were provided to meet the identified needs. You’ll also need to look at what the agency did to engage the various parties in services, and if services were effective in meeting the need.

The final section, 17 overall, is where you provide a summary of the three other sections.

Two tips to keep in mind while completing item 17 are:

  • Section 17A shouldn’t include any information related to educational, physical, or mental health needs or services, as those areas are assessed in later Well-Being items. Examples of needs and services that should be addressed include socialization activities, preparation for adoption, services to enhance self-esteem, and engagement with a mentor as a role model.
  • Your overall item rating Main Reason statement should not contain information that was not provided in either 17A, 17B or 17C. Section 17 overall is a short summary of those three sections.

Item 18

Child and Family Involvement in Case Planning

Purpose of Assessment: To determine whether, during the Period Under Review, efforts were made to involve parents and children in the case planning process on an ongoing basis.

Item 18 assesses whether parents and children were really and actively involved in case planning. If they were not, it explores the reasons. In other words, were children and families involved in identifying strengths and needs, services, goals, and so on, and were they involved in assessing progress toward meeting case goals?

One tip to keep in mind for item 18 is:

  • This item does not assess foster parent’s involvement in case planning.

Item 19

Caseworker Visits With Child

Purpose of Assessment: To determine whether, during the Period Under Review, the frequency and quality of visits between caseworkers and the child(ren) in the case are sufficient to ensure the safety, permanency, and well-being of the child and promote achievement of case goals.

Item 19 addresses whether the caseworker’s visits with the child were of sufficient frequency and quality to ensure the safety, permanency, and well-being of the child and promote achievement of case goals. Keep in mind that frequency is assessed by the standards provided in the instrument instructions, not by the policy of the state being reviewed.

Item 20

Caseworker Visits With Parents

Purpose of Assessment: To determine whether, during the Period Under Review, the frequency and quality of visits between caseworkers and the mothers and fathers of the children are sufficient to ensure the safety, permanency, and well-being of the children and promote achievement of case goals.

Item 20 addresses whether the caseworker’s visits with the parents were of sufficient frequency and quality to address the child’s needs and to achieve the goals of the case. Again, frequency is assessed by the standards in the instrument, not by policies of the State.

Item 21

Educational Needs of the Child

Purpose of Assessment: To assess whether, during the Period Under Review, the agency made efforts to assess children’s educational needs at the initial contact with the child and on an ongoing basis, and whether identified needs were appropriately addressed in case planning and case management activities.

Item 21 is used to determine whether the agency identified the educational needs of the child, both initially and on an ongoing basis, and tried to arrange for services to address the identified needs.

Note that the focus of this item is on the agency's response, not the educational services themselves. This is the one instance in which the reviews are not holding the State child welfare agency as accountable for the delivery of services to children. The reason that States are provided a little more leeway with regard to this item is because most educational systems are operated at the local level, with a separate board that oversees policy, practice, and budgetary matters. While States are expected to continue to forge relationships with educational systems, the Federal Government recognizes the different degrees of leverage that child welfare agencies will have in dealing with these systems.

A tip to keep in mind for item 21 is:

  • Unless this item is rated NA, the chart needs to be completed showing educational needs, and services provided or not provided.

Item 22

Physical Health of the Child

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to address the physical health needs of the child, including dental health needs.

Item 22 addresses the physical health needs of the child. To complete it, we need to determine whether the agency assessed the child’s health care needs both initially and on an ongoing basis, and then addressed those needs appropriately and effectively. As in item 21, unless the item is rated NA, the chart needs to be completed showing needs and services provided or not provided.

Item 23

Mental/Behavioral Health of the Child

Purpose of Assessment: To determine whether, during the Period Under Review, the agency made concerted efforts to address the mental/behavioral health needs of the child.

Item 23 addresses the mental/behavioral health needs of the child. We use it to determine whether the agency assessed the child’s mental-behavioral health needs both initially and on an ongoing basis, and then addressed those needs appropriately and effectively. As in items 21 and 22, unless the item is rated NA, the chart needs to be completed showing needs and services provided or not provided.

Period Under Review

When completing the Onsite Review Instrument (OSRI), it is essential to distinguish between events that took place over the life of the case, and events that took place during the Period Under Review, or PUR. Unless specifically indicated otherwise, all items in the OSRI pertain to the PUR. When it is necessary to look outside that period, the instructions in the OSRI will tell you to do so.

While the start date of the PUR will differ from one review to another, the end date of the PUR will be the Friday of the review week. You will be told the exact dates of the PUR onsite, and it will also be provided on your tablet.

Ratings

There are two types of ratings in the OSRIitem ratings and outcome ratings. The item ratings feed into the outcome ratings, so that once reviewers have completed all of an outcome's items, that outcome receives its own distinct rating. Once all seven outcomes and 23 corresponding items in the instrument have received a rating, and rating documentation has been completed, the instrument is considered finished and should be finalized.

Item Ratings

Each item in the OSRI, with the exception of the Face Sheet, receives a rating once its questions are addressed. There are three possible ratings:

  • Strength
  • Area Needing Improvement (ANI)
  • Not Applicable (NA).

During the first round of reviews, reviewers were responsible for deciding which rating each item should receive. Now, with the automated system, item ratings are calculated automatically by the system based on the answers to each item question. If each of the questions under each item is answered appropriately and correctly, the rating that the automated system assigns to the item will also be appropriate and correct. Therefore, it is the responsibility of the reviewer to answer the questions correctly, so that the automated system can assign correct ratings for the items.

Note that if you answer all of the questions appropriately, but don’t agree with the system's automatic rating, there is a mechanism that allows site leaders to override the rating. This is a very rare occurrence, though. In most cases where there is a discrepancy between an item's automatically generated rating and the rating you expect it to have, the discrepancy is due to an incomplete or inacurate answer on one or more of the item questions. Review your answers carefully for errors!

Outcome Ratings

The automated application automatically calculates and rates the outcomes once all that outcome's items have been rated. There are four possible outcome ratings:

  • Substantially Achieved: The required number of applicable items are rated as strengths.
  • Partially Achieved: Some applicable items are rated as strengths, but the number does not meet the level required for the outcome to be rated as substantially achieved.
  • Not Achieved: None of the applicable items is rated as a strength.
  • Not Applicable: None of the items is applicable.

To rate an outcome as substantially achieved, the following criteria must be met:

  • Safety Outcome 1 and Safety Outcome 2: All applicable items must be rated as strengths. Items rated not applicable are disregarded.
  • Permanency Outcome 1: Item 7 and the corresponding item  (8, 9, or 10) rated for the case must be rated as strengths. If the State is using concurrent planning and the reviewer rated two corresponding items, they must both be strengths. No more than one of either items 5 or 6 (if applicable) may be rated as an area needing improvement. Items rated not applicable are disregarded. 
  • Permanency Outcome 2: No more than one of the applicable items for this outcome is rated as an area needing improvement. Items rated not applicable are disregarded.
  • Well-Being Outcome 1: Item 17 must be rated as a strength. No more than one of the remaining applicable items may be rated as an area needing improvement. Items rated not applicable are disregarded.
  • Well-Being Outcome 2: Item 21 is rated as a strength.
  • Well-Being Outcome 3: All applicable items are rated as strengths. Items rated not applicable are disregarded.

Note that, while item ratings are included in the automated application, outcome ratings are not listed in the normal view. To review outcome ratings, use the Preliminary Case Summary Report.

Rating Documentation

Once an item's questions have been answered and its rating of Strength, Area Needing Improvement, or NA assigned, reviewers must completed the Reason for Rating and Documentation section. This part of the OSRI provides space to write a general statement justifying and clarifying the item's calculated rating. This general statement, referred to as a Main Reason statement, is followed by a number of specific follow-up questions designed to help further clarify the rationale for the assigned rating.

Note that, even if all 23 of the OSRI's items and all seven outcomes have been rated, the instrument is not considered completed and ready for finalization until all of its rating documentation has also been completed. For information on completing rating documentation in the automated instrument, see Module 6.4.3: Item Ratings and Rating Documentation.

Main Reason Statement

When composing your Main Reason statement, you should provide strong and clear justification for the item's rating. This information should be concise and clearly presented, should support the rating, and should not conflict with any of the information you have entered for other items. It's important to remember that the site leaders who conduct quality assurance (QA) on your completed instruments will neither have completed a case record review of the files nor have participated in your case-related interviews. Therefore, they will use the information you provide in your Main Reason statements to determine that each item's questions were answered appropriately and that the ratings are therefore correct.

Before you begin writing your Main Reason statement, though, you should take the time to review the item's follow-up questions. Your goal, wherever possible, should be to address most of the issues raised in the follow-up questions in the Main Reason statement itself. The follow-up questions can also serve as a guide for the order in which you should present information in your Main Reason statement, which in turn can help keep the statement as clear and concise as possible.

You must begin the Main Reason statement of every rated item with the phrase, “Item [number] was rated as [rating] because...” and then complete the sentence with a concise summary of why the item received the rating that it did. This summary sentence is extremely important and will not only help you and your review partner to crystallize why you’re rating the item as you are, but will make your justification information much clearer to the site leader who conducts QA on your instrument.

Following this summary sentence should be your explanatory information. This should be succinct and on-topic, but also thorough. Keep the following in mind as you compose your answer:

  • Avoid Abbreviations. The only abbreviations you should use when completing the instrument are PUR, for “period under review”; ANI for “Area Needing Improvement”; and NA for “Not Applicable.”
  • Do Not Use Names. Names are not to be used anywhere in the body of the instrument or rating documentation. The only exception to this rule is the Face Sheet, which can include names, and certain specific places in the instrument where first names are necessary for distinguishing individuals in the case (such as the chart in item 12).
  • Do Not Cut and Paste. Even though the automated application allows for cutting and pasting across items, you should avoid doing so. Each item in the instrument should be answered separately and stand on its own to ensure that its specific purpose of assessment is being addressed.
  • Explain Concerted Efforts. Several items require that you address whether concerted efforts were made by the agency. In such cases, you must detail those efforts clearly and not just state that “concerted efforts were made.” Likewise, if concerted efforts were not made, you should describe the lack of efforts clearly and note what should have taken place in terms of agency efforts.

In addition to the summary sentence and explanation that justifies the item's calculated rating, reviewers also must note the source or sources of the information they used to address each item. These sources should be listed at the end of the explanation. For example, reviewers who used the case record and an interview with the target child to answer and rate one item might conclude their Main Reason statement with:

Sources: case record, interview with target child

If you believe that an item's rating should have been different from what it is, first double-check your answers to each item question. An inaccurate response to one of the item questions is the most typical reason why the rating that is calculated may differ from what you would expect. If, after reviewing your answers, you still think the rating is incorrect, you should consult with one of the local site leaders. If necessary and appropriate, he or she can conduct a manual override of the rating and change it to what it should be. 

Follow-Up Questions

Every item in the OSRI must include a Main Reason statement with its rating documentation. This Main Reason statement is followed by additional exploratory, or follow-up, questions that are intended to help reviewers more thoroughly explore the rationale behind the item's rating. In most cases, these follow-up questions can and should be answered in the Main Reason statement itself. However, you must still enter text for each exploratory question. Otherwise, the automated application will register that you have omitted an answer and your instrument will be considered incomplete.

In these cases, your answers to follow-up questions can be as simple as “See Main Reason” or “NA,” depending on what is appropriate. Remember that you should only use "NA" if the follow-up question is actually not applicable; if you answered it in the Main Reason statement, that means it did apply to the case. If you say you answered it in the Main Reason statement, though, be sure to double-check the Main Reason and verify that the answer is actually there.

Some follow-up questions require that you complete a chart. You must complete these charts on their own; they cannot be answered as part of the Main Reason statement.

Case Record Review

Reviewers are assigned their cases at the review week's Monday morning team meeting. Upon receiving these, each review pair should make plans for beginning work on each case as quickly as possible. Each review pair is generally expected to finalize one case per day, which means that it must then go through the quality assurance process. In any event, all cases at the review site must be completed before the Thursday local site exit conference.

Some tips for working through each case are as follows:

  • Confirm that the case is applicable by checking it against the case elimination guidelines provided in the OSRI Quality Assurance Guide. It is critical that you and your partner ensure that each case you review belongs in the onsite review sample before you get too far in the case review. If it appears that the case does not belong with the sample, you should meet with your NRT Local Site Leader to discuss the possibility of eliminating it from the review.
  • Review the case history to determine how the child became involved with the agency.
  • Identify the key dates within the case, being sure to focus on the period under review except where otherwise instructed by the OSRI.
  • Note areas of the instrument in which information is incomplete, missing, or requires corroboration that will need to be collected through case-related interviews.

 

Case-Related Interviews

An important part of the case review process involves review pairs conducting case-related interviews with key individuals who are involved in the case. These interviews are not conducted as "customer satisfaction surveys," but rather serve as an opportunity to confirm case record documentation or collect information that might be missing from the record. 

One of the early lessons many child welfare workers learned in relation to maintaining case files is this one: “If it isn’t written down in the case file, it didn’t happen.” In the CFSR process, though, that motto should actually be: “If it isn’t written down in the case record, it still might have happened.” It becomes the reviewers’ responsibility to ask the right questions of persons important to the case to determine whether or not it really did happen.

Thus, interview information “weighs” just as heavily as information obtained from the case file documentation. Sometimes, information obtained during an interview may conflict with the documentation contained within the case record or obtained from another interview. In these cases, you and your partner have a responsibility to pursue the issue across multiple interviews until you can determine the most accurate response to the relevant questions. The case-related interviews are critical to gathering all the information needed to correctly complete the OSRI.

Key Individuals

The following key individuals related to a case will always be interviewed unless they are unavailable or completely unwilling to participate:

  • The child, assuming he or she is school age.
  • The child's parent(s).
  • The child's foster parent(s), pre-adoptive parent(s), or other caregiver(s), such as a relative caregiver or group home houseparent (if the child is in foster care).
  • The family's caseworker. If the caseworker has left the agency or is no longer available for interview, it may be necessary to schedule interviews with the supervisor who was responsible for the caseworker assigned to the family.
  • Other professionals knowledgeable about the case. When numerous service providers are involved with a child or family, it may be necessary to schedule interviews only with those most recently involved, those most knowledgeable about the family, or those who provide the primary services the family is receiving. More than one service provider may be interviewed.

Conducting the Interview

While there is no set agenda or checklist to use during a case-related interview, there are general tips you and your review partner should follow to ensure that each interview is as productive and informative as possible. These tips can be divided into three categories: pre-interview, interview, and post-interview.

Pre-Interview

How do you get the most out of the interviews you conduct? Here’s what you should be sure to do before the interview even takes place:

1. Complete case record review quickly but thoroughly. Review the case record quickly but carefully before the interviews, noting the areas in which information is incomplete or missing or areas in which the information should be confirmed by another party.

2. Recognize that time may not be on your side. Ideally, you would have an hour and a half to two hours to review the case file before your first interview; however, this isn’t always the case, depending on the availability of participants.

3. Become very familiar with item questions. Review the questions for each item, noting especially those sections of the instrument for which you did not identify sufficient information during the case record review. This is just another reason that it’s essential for you to be completely familiar with the OSRI so that you’ll know—even in a limited amount of time—which questions to ask which participants. Remember that the OSRI does provide some guidance on where you might find information for each item. This includes where in the case file to look for information as well as appropriate interviewees for each item.

4. Prepare interview questions specific to items and interviewees. Prepare a list of questions that are specific to the items you are rating and the role of the person you are interviewing. This will help you get the most out of the responses and more easily complete the instrument using those responses. There may be some questions that you’ll always want to ask certain parties, particularly to confirm or corroborate other information.

For instance, it’s advisable to always ask birth parents about item 17, needs assessment and service provision, and item 18, Involvement in Case Planning, as well as item 20, caseworker visits. You may also want to ask the child and caseworker—and perhaps the foster parent—about item 14, maintenance of the child’s connections, because many times this information is difficult to find or is missing from the case file. You’ll need to give some thought, either before the review or as it begins, to these “core” questions that you’ll want to ask different parties, particularly in terms of corroboration of information. Thinking through and jotting down “core questions” to be asked before the review begins will help you be more efficient in your interviews, ensure that all needed information is gathered, and more accurately assess and fully justify your item ratings.

Experienced reviewers often have “standard” questions they ask certain parties, like the birth parent or caseworker, to ensure that all relevant information is gathered. If you haven’t already developed some standard questions to ask specific interviewees, we encourage you to think through the items and come up with your own list of questions ahead of time so that you’ll be certain to cover all the important issues.

Interview

There are also several points you should keep in mind when the interview actually begins:

1. Introduce yourself and the interview process. Let interviewees know the approximate amount of time that the interview might take. You may find that you normally spend about 30 to 45 minutes in your interviews, although the interview with the caseworker will likely take longer. Let the participant know in advance that you will need to take notes while he or she is talking. You should not tape-record any interviews.

2. Provide an overview of the review process. Provide individuals with a brief overview of the purpose of the review process and the interview. Explain that the Federal and State governments are looking at how well the State is helping children and families achieve positive outcomes. Let parents or foster parents know that you are interested in learning about their experiences because it will help to determine how the State can better support children and families.

3. Reassure participants of confidentiality. Emphasize that the comments of particular individuals will not be identified by name in any report. Reinforce participants’ confidence in confidentiality by not revealing the comments of other persons interviewed, particularly those involved with the family. Stressing confidentiality is particularly important when interviewing children, parents, or foster parents. Note, however, that if concerns arise regarding the safety of the child, such concerns become subject to mandatory reporting laws. In addition, situations that you believe put the child at risk, such as individuals of whom the agency was not aware living in the home with the child or caregivers allowing a child in foster care to have visits with a non-custodial parent without the knowledge of the State, must be reported to the agency.

4. Explain your neutrality. Another important concept for your interviewees to understand is that you are a neutral reviewer with no ability to affect the case that you are reviewing. This is especially important when you are interviewing birth parents, who may see you as someone who can intervene on their behalf in a case plan or a case’s goals. You’ll need to be very clear that your role is not to specifically help or advocate for them, but to help the State know how to better meet the needs of families in the future. While you should acknowledge complaints raised by interviewees, you should not commit to checking on their situation or to getting back in touch with them.

5. Be flexible in your interview style and approach. Also, as you know, your interviewees may cross the spectrum from child to grandparent to therapist. You’ll need to be very flexible in your interview styles to accommodate the particular parties that you’re interviewing. At the same time, remain focused on what you need from each interview so that you obtain critical information while still using your limited time as efficiently as possible.

6. Get caseworker contact information. We advise you to get a phone number for the caseworker during the interview, and to ask if you may call him or her if further information is needed. It’s been the experience of many reviewers that they need to contact the caseworker again after the initial interview to ask for clarification or obtain further information, particularly if the caseworker is one of their earlier interviewees.

Post-Interview

Once the interview has concluded, you and your partner should:

1. Immediately report child safety concerns. If you hear information in an interview or observe something while interviewing that raises concern about risk or the safety of a child, immediately report that concern to your Local Site Leader (unless it is an emergency that requires you to immediately call 911). The Local Site Leader will work with you and the child welfare agency to address the issue. 

Note that you should always strive to ensure that children are not upset by these interviews, and normally, they aren’t. However, in the event that a child appears upset after an interview, be sure to immediately tell a Local Site Leader so that the State can respond to the situation by providing support to the child.

2. Record the interview results. Immediately after the interview, you should record your interview notes more completely into the appropriate sections of the OSRI. Note that you should not tape record interviews.

3. Schedule additional interviews as needed. You may discover that additional interviews beyond those scheduled by the Local Site Coordinator are needed in order for you and your review partner to complete a thorough case record review. If this happens, you should immediately consult with your Local Site Leader about the possibility of scheduling a new interview. Depending on where you are in the review week and with your case load, this may or may not be possible.

Finalizing the Instrument

Once reviewers have finished their case record review, interviewed everyone involved with the case, answered all of the questions in the automated instrument, and ensured that every item is rated and documented, they must perform Preliminary Quality Assurance (QA) on the entire instrument. This Preliminary QA is intended to ensure that all of the answers and ratings are accurate and complete.

Once the Preliminary QA is finished and reviewers have verified the completeness of the instrument, they are ready to data transfer the record to a Local Site Leader's tablet for First-Level QA. Through this process, a Local Site Leader reviews the instrument and adds stickies to any items that seem problematic. Reviewers must then address these problem items before the record moves on to Second-Level QA.

Once First- and Second-Level QA are complete, the record is ready for the additional QA steps of Local Site Finalization and Data Validation. In addition, the NRT Local Site Leader will collect all of the local site's cases to a single tablet in order to prepare the preliminary report that will serve as the foundation for the local site exit conference that takes place on Thursday. At some point prior to leaving the local site, he or she will also ensure that all of the site's records have been uploaded to the central server. The cases' outcome ratings will then become a part of both the the Friday statewide exit conference and the Final Report that serves as the foundation for the State's Program Improvement Plan.