Lessons Learned from SQA Assessments - 9/23/2004

• Software Requirement Document (SRD) and Software Design Document (SDD) are essential for developing quality software and life cycle maintenance.
  
  The information for SRDs and SDDs are typically extracted from system design documents that provide the process system design and operation details. System design documents generally do not address software application, its functionality and performance requirements that are essential for the software design and development. Success of a software development project relies heavily on how well the software requirements are defined. In the absence of SRD and SDD, the software developer must rely on good communication with the system design engineer and understanding of the system design document.
  
  Observations: It is evident from site assessments that the majority of software projects did not have SRDs and SDDs. The sites using the SRDs and SDDs were found to have clear understanding of what was needed to develop and maintain the quality of the software. The sites without the benefit of the SRDs and SDDs appeared to be relying heavily on the available experts on the projects and also on the process system engineers to ensure that the software developed or procured would meet the project needs. This is particularly important for the software used for the process system controls.



• Software procurement specifications should specify details of software requirements, not just catalog data.
  
  Software procurement should be integrated with the SQA program with clear requirements and responsibilities for planning and executing procurement of software related items and services. Appropriate interface with Engineering and IT departments should be established and proceduralized. Proper evaluation and qualification of suppliers in accordance with NQA-1 standard and follow up surveys and re-evaluation are crucial to SQA. Absence of technical requirements in the procurement specification could contribute to poor quality products, or products with limitations. Now-a-days vendors provide many options, and without appropriate specifications selection of the options could be limited. The procurement specification should also specify quality and documentation requirements commensurate with software applications.
  
  Observations: The sites procuring programmable logic controllers for the process systems only specified the vendors' catalog model information for procu.5ement specifications without any supporting documentation for the suitability and applicability of the technical requirements.
  
  
• Formal procedures for software problem reporting and corrective actions for software errors and failures need to be maintained and implemented rigorously.


Organizational responsibilities for software problem reporting and corrective action for errors and failures should be clearly identified and implemented through well documented procedures. Completion of corrective action should be documented and reviewed periodically. Without such practices recurrence of the problems may not be prevented and may not help to learn any lessons from the errors. Contractual specifications should require software vendors to notify DOE contractors of newly-found errors in the codes.

Observations: Many sites resolve their software errors and corrective action process at a project level and maintain informal coordination with vendors or other effected entities.


• Software quality assurance (SQA) program and procedures should be implemented rigorously.


SQA program encompasses all the procedures and requirements that are essential to ensure quality of the software product and its life cycle maintenance. Clear, appropriate and well documented program and procedures coupled with qualified and trained personnel and a self-assessment program is the foundation for establishing and maintaining the software quality.

Observations: Site assessments revealed inconsistencies in the requirements contained in the SQA program and procedures and their implementation. Many sites rely on individual expertise and their personal effort and put less importance on corporate program.


• Appropriate qualifications and training on software use is essential for proper use of safety software


Software developers and users should have requisite qualification, and be trained in SQA procedures and requirements. Software developers and users should have appropriate basic underpinning of the technology used in the software and should be knowledgeable in software quality assurance, verification and validation, configuration management. and error reporting and corrective action. The qualification and training requirements need to be documented in SAQ procedures and an approved user/developer list maintained. Through personnel training the SQA culture needs to be developed for a successful SQA program.

Observations: SQA assessments indicate that very sophisticated and complex software are being used sometimes without appropriate training.


• Appropriate software control and configuration management is essential for safe use of the software.


Safety software should be controlled at all times both in terms of its version, distribution, residence and access. An inventory of safety software should be maintained. Configuration management is needed to control utility or calculational type software, such as spreadsheet (Excel) or Mathcad.

Observations: Lack of proper control had resulted in multiple versions being available at the same time and even some with known errors. Assessments have noted deficiencies with configuration control in terms of software version and documentation.