The Automated Onsite Review Instrument

The Automated Onsite Review Instrument is the electronic version of the OSRI. It has the same overall structure as the paper instrument but differs in a few specific areas. It does not, for example, re-create the "General Instructions" page that begins the paper instrument, and in a few instances the automated application sub-divides item questions differently from those that are listed in the paper instrument. A few of the charts, most notably Chart F on the Face Sheet, are also formatted differently in the automated instrument from those in the paper version.

By and large, though, the structure of the automated instrument exactly follows that of the paper instrument. This includes listing each item's Purpose of Assessment and question instructions as well as its Rating Documentation and follow-up questions

One notable difference between the automated instrument and its paper version is how Outcome Ratings are handled. In the paper instrument, each Outcome Rating has a separate page that must be manually answered by the reviewer. In the automated instrument, Outcome Ratings are generated automatically once the Outcome's various items have all been rated, and there is no separate Outcome Rating page. To view an Outcome Rating, you must use the Preliminary Case Summary Report

Working With Cases

There are three ways for review pairs to work with OSRI cases while onsite. They can add a new case, edit existing cases, or delete existing cases. All three of these options are accessed through the OSRI Menu in the Menu Bar

Add New Case

To create a new OSRI case, select Add New Case from the OSRI Menu. The Add New Case window will appear. Next, complete the following steps:

1) Enter the case's name in the Case Name area. For foster care cases, the case name will be the full name of the target child. For in-home services cases, the case name should be the family surname.

See Module 6.3: Inputting Data for more information on how to enter text.

2) In the Case Type box, select whether the case is foster care or in-home services by clicking the appropriate choice.

3) Click the arrow beside the Date Reviewed box to display the pop-up calendar control. The current date will be highlighted in red. Select a review date for the case by clicking the appropriate date.

4) To finish, click the Save and Start button. You may also click Cancel to close the Add New Case window without saving.

Once you click the Save and Start button, the Face Sheet for the new case will open. To exit the Face Sheet, click Overview to return to the Overview Screen. The new case will be displayed in the Record Summary Grid with a QA Status of Record Created.

Delete Existing Case

It is possible to delete a case from the database. However, deletions are permanent and cannot be undone. You should always be sure to backup the database before you delete any case or cases from it.

To delete cases already saved to the system, use the following steps:

1) Select Delete Existing Case from the OSRI Menu to open the Deletion Screen.

2) On the Deletion Screen, click the arrow beside the Selected Record Menu to open a list of cases available for deletion. Click the case you wish to delete, then click the Delete Selected Record button.

3) A Confirm Delete window will open asking if you wish to backup the database before deleting. If you click Yes, you will be asked to select a destination for the backup file. If you click No, the deletion will continue.

4) A Delete Record Final Warning window will open reminding you that record deletions are permanent and irreversible. Click OK to proceed with the deletion or Cancel to change your mind.

5) Once you click OK, a new window will open confirming that the record has been deleted. Click OK to continue.

6) Click Overview to return to the Overview Screen. The case will no longer be displayed in the Record Summary Grid.

Remember that you cannot undo a deletion. Once you click OK, that case is wiped from the system and can be restored only by restoring a previous version of the database.

Edit Cases

There are two ways to open a case for editing:

1) Click the Edit button in the Action column of the Record Summary Grid. This will open that case’s Face Sheet.

2) From any screen, select Edit Cases from the OSRI Menu. This will open the Face Sheet for the first case listed in the Record Summary Grid.

Once you have opened a case for editing, you can switch to another case without returning to the Overview Screen by using the Selected Record Menu, which is located near the top of the screen. Click the down arrow in the field labeled Selected Record to open this menu to display a list of all your current cases. Select a case from this list to open its Face Sheet.

If, after creating a case, you need to edit its name, case type, or date reviewed, click the Edit button located beside the Selected Record Menu. This will reopen the Add New Case window for that case, from which you can enter changes. Click the Save and Start button to close the window and return to that case’s Face Sheet. Your changes will be saved to the system.

OSRI Screen Layout

All screens in the OSRI use the same basic layout. At the very top of each OSRI screen is the Title Bar. Below it is the Menu Bar. Below the Menu Bar is a display window that shows the current reviewer or Review Team. To the right of it is the Selected Record Field, which displays the name of the currently selected case. The drop-down arrow opens a list of other cases that you can edit. To the right of the Selected Record Field is the Edit button.

Below the Selected Record field and Edit button appear the green Document Navigation Bars. If you are viewing the Face Sheet, there will only be one bar labeled Face Sheet. Everywhere else in the OSRI, three bars will display: one for the current section, one for the current outcome, and one for the current performance item. You can use the Document Navigation Bars to navigate through the instrument using either Arrow Navigation or Directory Navigation.

Below the Document Navigation Bars is the item’s Purpose of Assessment. Below that is the Save Bar and the Question Area. At the bottom of the OSRI screen are two Navigator Bars: the Unanswered Questions Navigator and the Unresolved Comment Navigator. Both can be turned on and off through the OSRI Menu and allow you to quickly access unfinished sections of the OSRI.

Save Bar

Below the Purpose of Assessment is the Save Bar, which displays buttons for saving data as well as the item’s Calculated Rating. There are generally three save options available: Save>Rating Documentation, Save>Next Item, and Save

Save>Rating Documentation: This saves your answers and opens the item’s Rating Documentation Screen. Note that on the Face Sheet, this button does not appear because the Face Sheet is an unrated item.

Save>Next Item: This saves your answers and moves you forward one screen to the next item in the OSRI.

Save: This saves all of your answers. The application remains on the current screen.

Note that none of your answers to OSRI questions are saved to the system until you use one of the three save options available on the Save Bar. If you attempt to exit the current screen without saving, the application will prompt you with a pop-up window to first save your work.

Question Area

Below the Save Bar is the main Question Area. Because each item in the OSRI has more than one question, there are multiple Question Areas for each item. To scroll through the various questions, use the scroll bar located on the right-hand side.

Each question is identified with a designation (normally a letter or number) that corresponds to a designation used in the Questions Overview Bar (see below). To the right of the question’s designation is a colored oval that shows that question’s status. A question’s status may be Saved, Unsaved, Locked, or Blank.

Saved questions (light blue) have been answered and saved to the system using one of the three Save buttons.

Unsaved questions (yellow) have been answered or edited but are not saved.

Locked questions (dark gray) have been locked by the system and cannot be edited. These are usually questions that, due to answers already input, are either no longer applicable to the case or were automatically answered by the application.

Blank questions (tan) have not been answered.

The color used with each question’s status also corresponds to the color that appears in the Questions Overview Bar.

Below the question’s designation and status is the question itself and space in which to answer it. See Module 6.3: Inputting Data for more information on answering questions. At the bottom of each question are any special instructions/definitions relevant to it. You should read these carefully before answering the question.

Questions Overview Bar

To the left of the Questions Area is the Questions Overview Bar, which is a vertical bar made up of boxes that correspond to each of the item’s questions. Each box’s color will match the color of that question’s status. Click inside a box to jump to its matching question. If you float the cursor over a box, a pop-up window will appear showing its question’s designation, status, and answer. Note, however, that the answer will not appear until you have saved your work.

Item Ratings and Rating Documentation

Each OSRI item is rated automatically by the application once all of its questions are answered and its Calculated Rating is shown on the Save Bar. The exception to this is the Face Sheet, which is unrated. There are four possible ratings for each item:

  • Incomplete (questions remain unanswered)
  • NA (Not Applicable)
  • Strength
  • ANI (Area Needing Improvement)

If an item has been overridden, a double asterisk (**) will appear beside the rating. If you float the cursor over the rating, you will be able to view the override.

Once an item has been rated, the reviewer must complete that item’s Rating Documentation. This documentation is entered on the Rating Documentation Screen, which is accessed by clicking the Save>Rating Documentation button on the Save Bar.

The Rating Documentation Screen is laid out in the same format as every other OSRI screen, with two exceptions: the Purpose of Assessment is replaced by the Reason for Rating and the Save>Rating Documentation button is replaced by the Save>Item Questions button. Clicking this button returns you to the regular OSRI screen.

Answering Rating Documentation Questions

The first question on each Rating Documentation Screen is referred to as the Main Reason Statement, and it is where the review pair must provide the main justification for the item’s Calculated Rating. There are certain formatting and content issues that you must keep in mind when composing your Main Reason Statement; see Module 3.3.1: Writing the Main Reason Statement for more information. The same is true for the follow-up questions that come after the Main Reason Statement.

Remember that when you are completing the automated instrument, every question must have an answer. If you answer one of the follow-up questions in the Main Reason Statement, you must enter text in the follow-up question to indicate this. For example, you might type "See Main Reason" or "Answered in Main Reason." If a follow-up question is Not Applicable to the case, then you must type the letters "NA" as an answer. You cannot leave any questions blank.

Also, note that there is no need to complete the Rating Documentation immediately. A review pair may choose to finish the Rating Documentation after they answer all of the OSRI’s questions, or they may complete each item’s Rating Documentation as they complete each item. Work through the instrument in the manner you find most efficient.

Overriding Item Ratings

There are times when a review pair may disagree with the rating that the automated application assigns to an item or outcome. In these cases, the system does allow for any item’s Rating to be overridden and changed. Note that review pairs can only suggest that a Rating be overridden; only Site Leaders can actually perform an override.

If a review pair believes that an item’s rating should be changed, they should indicate this in the Main Reason question on that item’s Rating Documentation Screen. Local Site Leaders will review these answers during First-Level QA and decide whether overriding the item’s rating is justified.

If an override is justifiable, then the Site Leader can use the Override button. The Override button can only be accessed through the Screen View. To override an item’s Rating, use the following steps:

  1. Navigate to the appropriate item. Click the Override button, which is located in the middle of the Save Bar.
  2. The Item Rating Override Window will open. The field at the top of the window displays the item’s current Rating. Beneath that is the Select Override Rating drop-down menu. Use this menu to select the new item rating.
  3. Enter the reason for the item rating override in the text area of the Item Rating Override Window. When you are finished, click the Save and Close button.
  4. The item’s original Calculated Rating will still display in the Save Bar. However, two asterisks (**) will display beside it to indicate that it has been overridden, and if you float the cursor over the rating a pop-up window will display both the new rating and the reason for the override.

You can further edit an item’s rating or the reasons for overriding it by clicking the Override button to reopen the Item Rating Override Window. Repeat the steps described above to make any changes.

Note that you can undo an item rating override by clicking the Undo Existing Override button, which is located beside the Save and Close button. Clicking this button will open a pop-up window asking you to confirm your selection. Click the Yes button to proceed with the undo, or the No button to cancel.

Once an override has been undone, the Item Rating Override Window will close. The item’s Calculated Rating will be returned to what it had been originally, and the asterisks marking the override will no longer be present.

OSRI Navigation

Within an individual item, the Questions Overview Bar, which is located in the Question Area, allows you to quickly jump from question to question without having to scroll up and down. Similarly, the application also features tools that enable you to quickly move from item to item. These tools include the Document Navigation Bars and the Navigator Bars.

Document Navigation Bars

The Document Navigation Bars allow for the use of Arrow Navigation to move through the instrument item by item, or Directory Navigation to select an item from a directory tree view of the entire instrument. See the links below for more information about these methods of navigation.

Navigator Bars

There are two Navigator Bars located at the bottom of the screen: the Unanswered Questions Navigator, which lets you quickly access any unanswered question in the entire instrument, and the Unresolved Comment Navigator, which enables review pairs to quickly jump to any stickies that have been placed in the instrument as a result of the Quality Assurance review.

Arrow Navigation

The green arrows located beside the Document Navigation Bars allow you to move through the various OSRI sections, outcomes, and items in single steps. Clicking a right-facing arrow will move you forward one step, and clicking a left-facing arrow will move you backward one step. If there are no steps remaining in either direction, the corresponding arrow will be grayed out.

Note that the green arrows correspond to the part of the instrument that the navigation bars identify. For example, if you are on Section I, Safety Outcome 1, Performance Item 1, then the top bar identifies the section, the middle bar identifies the outcome, and the bottom bar identifies the item. If you click the middle arrow, you will move forward one outcome, to Safety Outcome 2 (which is still in Section I but begins with Item 3). The green arrows will gray out when you can no longer move forward or backward through that section or outcome.

Directory Navigation

Directory Navigation provides a fast way to move through the OSRI in nonsequential order. To use Directory Navigation, click anywhere inside any of the Document Navigation Bars. The bars will be replaced by a directory tree that displays the entire OSRI.

The item you are currently viewing will be highlighted in blue. Scroll through the directory by using the scroll bar on the right side and click on the section, outcome, or item that you wish to access. The OSRI will jump to that screen.

Note that the directory will close once the new screen opens and will be replaced by the normal Document Navigation Bars. You can also restore the Document Navigation Bars anytime by clicking within the white space of the directory itself. To re-open Directory Navigation, click the green Navigation Bars.

Unanswered Questions Navigator

The Unanswered Questions Navigator allows you to quickly jump to any single question in the automated instrument that has not yet been answered. It is an effective tool to use toward the end of your case review as a way of verifying that you have remembered to answer every question in the instrument. It is also an important part of Preliminary QA.

The Unanswered Questions Navigator is located at the bottom of the screen. To use it, click the down arrow located on the far right. A list will open that shows all of the questions that have not yet been answered for that case. Each question is identified in brackets as being part of the Face Sheet (FS), OSRI, or Rating Documentation (Rating) as well as by its number and letter designation. To jump to one of the questions, simply click it in the list. The application will immediately jump to its screen.

Note that the Unanswered Questions Navigator works best when you have a single case open for editing. While you can use the Unanswered Questions Navigator from the Overview Screen,  the list that displays there will show unanswered questions from every case on your tablet instead of the one case you are probably focusing on. When you have a case open, the Unanswered Questions Navigator will list only the unanswered questions from that case.

Note also that the Unanswered Questions Navigator can be turned off and on through the OSRI Menu.  The default setting is ON. To turn it off, select the Show Unanswered Questions Navigator option from the drop-down list. The orange highlighting around the option will disappear. You can turn on the Uanswered Questions Navigator at any time by re-selecting it from the menu.

Unresolved Comment Navigator

The Unresolved Comment Navigator displays unresolved stickies that have been placed within a case record by a site leader during the Quality Assurance process. It displays at the bottom of the screen, alongside the Unanswered Questions Navigator. It can be turned off and on by selecting Show Comment Navigator from the OSRI Menu. Orange highlighting around the menu's icon means that the Unresolved Comment Navigator is active.

To use the Unresolved Comment Navigator, click the down arrow on the right side of the bar to open a list of all item questions and rating documentation that currently have an unresolved sticky. Scroll through the list and click on the sticky you wish to view, and the application will jump to it. You can then respond to the sticky normally.

Note that the Unresolved Comment Navigator works best if the case you are working on is open. If the case is open, the stickies that display in the bar will only be for that case. If you open the Unresolved Comment Navigator on the Overview Screen,  it will list all unresolved comments for all the cases on your tablet.

Also note that the Unresolved Comment Navigator does not track or identify the comments that you have viewed. They remain on the list until your site leader has resolved them, which only happens after the case is transferred back to his or her tablet. For this reason, you should work through the stickies added to the comment bar methodically, in a sensible order (top to bottom or bottom to top) so that you do not lose track of where you are.

Completing the OSRI

Before a review pair submits a completed OSRI to a site leader to begin First-Level QA, its QA Status must first be changed to Ready for QA Review.

Note that only cases that have been entirely completed and have undergone Preliminary QA should have their status changed to Ready for QA Review. At this point, you are ready to transfer the case record to the site leader who is handling First-Level QA.