Quality Assurance

Quality Assurance, or QA, is a critical component of the CFSR process. The QA process is designed to ensure that, as much as possible, all cases completed during the review week consist of clean, complete data that accurately answers all questions, and that all item and outcome ratings are accurate and supported with well-reasoned Rating Documentation.  

The QA Review process as a whole is divided into seven parts:

  1. Preliminary QA
  2. First-Level QA
  3. Second-Level QA
  4. Local Site Finalization
  5. Data Validation
  6. Third-Level QA
  7. Data Change Management

There are numerous tools and functions built into the automated application to assist and streamline the QA process. These include QA tools that can be used by both reviewers and site leaders, data transfer functions, and the ability to post electronic comments, or "stickies."

QA Tools

The Quality Assurance (QA) process is an integral part of the onsite review. Review pairs must conduct an initial QA on all cases that they complete. This initial QA, referred to as Preliminary QA, helps ensure that basic, easily corrected errors (such as missing information) are caught and corrected before they take up time in the later stages of the QA process. During later stages of QA, such as First-Level and Second-Level QA, site leaders become involved in the process and work with review pairs to correct any errors or omissions in case records. 

There are a number of QA tools built into the application that are designed to streamline the QA process for both review pairs and site leaders. While many of these tools involve advanced QA functions used in First- and Second-Level QA, such as data transfers and stickies, there are also numerous resources that review pairs and site leaders can use to assist in reviewing the instrument, especially during a Preliminary QA. Two of these resources, the Completeness column and the Unanswered Questions Navigator, are essential for review pairs in determining that an OSRI record is complete and ready for QA.

Another important resource are the application's many built-in reports. Especially useful for Preliminary QA are the Completed Case Report, the Preliminary Case Summary Report, and the Case QA Rating Summary Report.

During First- and Second-Level QA, the Unresolved Comment Navigator is a critical tool for review pairs to locate and respond to the stickies that site leaders add to a case record.

 

Stickies

"Stickies" are electronic notes, or comments, that can be attached by a site leader to an item's question in an OSRI record. They are intended to draw attention to places where corrections, revisions, or additional input might be required by the review pair, and are an integral part of both First-Level and Second-Level QA.

To add stickies, the site leader must first transfer the OSRI record to his or her tablet. At this point, the record becomes active on the site leader's tablet, and locked on the review pair's tablet. While it is locked, the review pair cannot edit the record in any way. They can, however, read it and access reports on it. The site leader should conduct QA as quickly as possible, add any stickies that are necessary, and then transfer the case back to the reviewer pair's tablet. The review pair will then use the Unresolved Comment Navigator to locate the stickies and respond to it as appropriate.

When the case is again transferred to the site leader's tablet, the site leader can resolve the sticky (if the issue has been adequately addressed) or add to the comment and return the case for further clarification by the review pair. This process can continue until the issue is resolved, although it is likely that site leaders will ultimately choose to discuss persistent issues face-to-face rather than continue to exchange sticky notes back and forth with a review pair.

Adding Stickies

After a Local Site Leader transfers an OSRI record from a review pair's tablet, he or she will be able to open it for QA review by using the QA button in the Action Column of the Record Summary Grid. The case will open as a Completed Case Report, which will allow the site leader to scroll through the entire instrument in one window from start to finish. From this report, the site leader can add stickies as necessary to any single question. Each sticky will then be accessible by the review pair when the case is transferred back to their tablet.

To add a sticky to rating documentation or item questions, use the following steps:

1) Click the Add link, which is located beside the question identification.

2) The Conversation Form Window will open. The case name and the question to which you are commenting display at the top. Click the Add New Message button to add your sticky

3) The Add/Edit Message Window will open. Type your comment in the text field. When you are finished, click the Save button to close the window.

4) The Conversation Form Window will now display your comment below the question. As the first comment in the conversation, it will be highlighted in yellow. Your name will display beside it, and an Edit and Delete button will appear on the right-hand side. Clicking the Edit button will reopen the Add/Edit Message Window, where you can edit your answer. If you click Delete, a confirmation window will open asking you to verify your selection. If you click OK at this window, your comment will be permanently deleted from the Conversation Form Window.

5) Click Close to exit the Conversation Form Window.

Note that the presence of stickies will be noted on the Completed Case Report by the use of orange highlighting. Clicking on the View/Add link will reopen the Conversation Form Window and allow further editing or deletion of the sticky. Note, however, that once a sticky has been responded to by reviewers, it can no longer be deleted. It can only be resolved.

There is no limit to how many stickies each OSRI record can hold. There is also no limit to how many times stickies can be sent back and forth between site leaders and reviewers. However, if it becomes obvious that a case is overloaded with stickies, or if one particular sticky appears to keep getting passed back and forth without resolution, site leaders are probably well-advised to seek out a face-to-face conference with the review pair, rather than continue an unproductive exchange.

Note also that there may be times when you want to review a case as the review pair sees it, using the Screen View instead of the Case Summary report. One reason why this may be true is if you want to review the instrument's instructions for a particular question and you do not have a copy of the paper instrument handy. Since the instructions are not included in the Case Summary Report, you would need to open the case in a screen view.  For information on how to do this, see Module 6.8.2.2: Adding Stickies in the Screen View.

Adding Stickies in the Screen View

There may be situations where you need to view a record’s individual item screens as a reviewer sees them. In these cases, the Screen View should be used.

To access Screen View, select Edit Cases from the OSRI Menu. This will open the Edit Cases Screen. Use the Review Team drop-down menu to select the correct reviewer, then the Selected Record drop-down menu to select the case.

The case will open automatically to its Face Sheet. The layout and navigational functions are the same as for reviewers, although the Save Bar no longer features any save buttons. Instead, it displays the Rating Documentation button, the item’s Calculated Rating, the Override button, and the Next Item button.

Use the normal navigational controls to move through an OSRI record in the Screen View. To add a sticky to rating documentation or item questions, click the Add button, which is located in the upper right-hand corner of the question’s Identification Area. The Conversation Form Window will open. Adding comments here works the same way as in the Report View.

When you close this window, the Add button will be replaced by the View button, which indicates that comments have been started. Clicking View will reopen the Conversation Form Window.

Note that while the Screen View does allow site leaders to view a case as reviewers see it, the case is locked for editing. It is possible, however, to use the Unlock a Case function to make emergency edits to case information.

Responding to Stickies

Each sticky added to an OSRI case by a site leader must be read and responded to by the review pair. The response will most likely involve edits to question answers, but must also include a written comment back to the site leader. These comments serve as an indication to the site leader that the sticky was properly addressed.

To open a sticky, click the View button located above the question. The Conversation Form Window will open. The name of the commenting site leader will display beside his or her comment, which will be highlighted in yellow. Read the comment, then click the Close button and make any necessary changes to the question. When you are finished with your edits, reopen the sticky and use the following steps to post a response to the site leader's comment:

1) Click the Add New Message button to open the Add/Edit Message Window

2) The Add/Edit Message Window will open. Type your comment in the text field. When you are finished, click the Save button to close the window.

3) The Conversation Form Window will now display your response below the Local Site Leader’s comment. Your name will display beside it, and an Edit and Delete button will appear on the right-hand side. Clicking the Edit button will reopen the Add/Edit Message Window, where you can edit your answer. If you click Delete, a confirmation window will open asking you to verify your selection. If you click OK at this window, your comment will be permanently deleted from the Conversation Form Window.

4) Click Close to exit the Conversation Form Window.

Use the Unresolved Comment Navigator to continue working through each sticky added by the Local Site Leader. When you are finished, return to the Overview Screen and change that case’s QA Status back to Ready for QA Review to indicate that it is once again ready for download. The site leader will transfer the case to his or her tablet and read through your comments and edits.

Resolving Stickies

Once the review pair has addressed the issues raised in the stickies that a site leader added to an OSRI, the site leader must transfer the case back to his or her tablet and determine whether or not the stickies were adequately addressed. To do this, open the case in its Report View by using the QA button in the Action Column of the Overview Screen. Existing stickies will appear with orange highlighting. Click the View button to reopen the Conversation Form Window. A message from the review pair will display below your comment indicating that they addressed the issue raised by your sticky. You must decide if their response is adequate or if the question still requires more input.

If the response is adequate, check the small box labeled Resolved at the bottom of the Conversation Form Window. A notification that the conversation has been resolved will appear and the sticky’s orange highlighting will display with a different hue. Also, the View button will be replaced by a Re-Open/View button.

If additional input is required, add a new comment for the reviewer by clicking the Add New Message button to bring up the Add/Edit Comment Window. Type your comment in the text box and click the Save button when you are finished adding your new comment. Click the Close button to close the Conversation Form Window. You must now transfer the case to the review pair so that they can view the sticky and make the necessary corrections.

After all of the stickies are resolved, the case will be ready to move on to either Second-Level QA or Local Site Finalization.