CRAD Development

Background:
In the Implementation Plan for DNFSB Recommendation 2004-1, DOE committed to the development of an Oversight Manual with Criteria Review and Approach Documents (CRADS) for many areas, including Startup and Restart of Nuclear Facilities.

What is a CRAD?
CRADs are the documents used in the IP to establish the depth of the Readiness Review (RR) and provide guidance to the RR team. As such, the quality of these documents significantly impacts the overall quality of the RR. The CRADs form the basis for evaluating the Core Requirements (CRs), which are derived from the 17 CRs listed in DOE O 425.1D. Each CR is evaluated based on the established criteria. The criteria should be specific and as objective as possible.

The graded approach is applied to the scope of the RR through the CRADs. The areas that are significant to the startup or shutdown should be assessed to a greater extent than other areas. For example, if during a maintenance shutdown, a system was modified or a new system was added, the training, procedures, documentation, and safety basis for the new system should be reviewed comprehensively. Another system in that same facility that did not undergo modification would receive a less comprehensive review that could consist of a sampling of the training and procedures associated with the system.

In general, a CRAD consists of the following elements:

1. Performance Objective: Identification of the expectation(s) or requirement(s) to be verified, which reflect the complete scope of the readiness review as specified by the plan of action.
2. Criteria: The specifics by which the objective will be measured, including regulatory and/or site-specific requirements.
3. Review Approach: A statement of the documents, interviews, and observations which will be used to obtain objective evidence in order to determine if a criterion is met or not.

Creating the Performance Objective
The Objective in each CRAD includes all, or portions, of one or more CRs. To write the objectives, use the following as a guide:

• Begin with Requirements;
• Use performance goals;
• Use performance expectations; and
• Be global to functional areas, allowing criteria to address specifics.
   

Example:
OBJECTIVE: Line Management has established and implemented a Fire Protection Program to ensure adequate protection of operations and activities.

Writing Criteria
The criteria in the CRAD are developed so that the criteria reflect the Objective. This ensures that all CRs are addressed regardless of the approach used in developing the criteria. The criteria that address the CR or portion of a CR included within the Objective should follow, and be related clearly to, the requirements. It is important to remember that the sum of the criteria must provide adequate reasoning that the objective has been met. Each criterion is a statement of the specific actions or attributes the team members use to make a judgment as to the readiness of the site, facility, or process to operate in a specific area.

The criteria should be specific statements that are:

• Measurable (i.e., the program will have ten working elements); and
• Assessable (i.e., documented proof of meeting the working elements).
   

Criterion statements may also be:

• Operational performance parameters relating to measurements, or forces for example:
  
  • Mass – quantities of special nuclear material in glovebox;
  • Time – seconds required for emergency power;
  • Airflow – the air will circulate 9 times per minute;
• Derived from guides, standards, and good practices which are associated with requirements.
   

Examples:

1. TSR commitments associated with the fire suppression system including design, construction, operability, maintenance, and surveillances are accurately and effectively implemented.
2. Issues identified during previous reviews (e.g., CDNS Biennial Reviews, HSS reviews, self-assessments, etc.) have been appropriately resolved and adequate corrective actions have been completed, or a clear path to completion is indicated.

Review Approach
The Review Approach section of the CRAD describes the documents to be reviewed, the personnel to be interviewed, and the shift evolutions, including tours and walkdowns, to be observed that enable the team to reach a conclusion as to whether the criteria have been met. The final portion of the CRAD should include any references (i.e., DOE Orders, mandatory standards, or site-specific requirements) against which the preceding criteria are to be assessed.

It is important to research and determine how documents are communicated. There are two ways to determine if documents are being followed or understood.

• Interviews, and
• Observations
   

The review approach must link back to each of the criteria and must be tailored to the organization, facility, staff, and the procedures being assessed. If the review approach is conducted correctly, then your documented review can be used in your final write up.

As a part of the interviews, assessors should develop lines of inquiry (LOIs). LOIs are a set of questions directly related to the criteria, and the review approach, which are used to determine if an objective or criterion is met or not. Questions are tailored for various levels of management and workers. Each LOI should be clear and not be open-ended so that answers from each interview can be compared.

Examples:

1. Does the contractor perform periodic assessments of the Fire Protection Program?
2. What mechanisms does Line Management use oversee the contractors’ Fire Protection Program?

For additional information on and/or examples of CRADs see DOE-STD-3006.

Visit the Training link for CRAD development training.

For further information or comments on the ORR web site, please contact James Heffner.