Validation Activities Related to the SaaS Application

Validation Activities Related to the SaaS Application

Overview

Software-as-a-Service (SaaS) model, which has been defined as the situation wherein a company uses a vendor’s software application from devices through a web browser or program interface. The client does not manage or control the underlying cloud infrastructure; including the network, servers, operating systems, storage, or even individual application capabilities; with the possible exception of application configuration settings.

SaaS and the Life Science Industry

In the life science industry, there may be a number of SaaS products throughout the organization, including customer relationship management systems, human resource information systems, laboratory information management systems, learning management systems, quality management systems, and other areas. For this reason, given that validation of a SaaS application is a shared responsibility between the client and the SaaS vendor at the deployment phase and as the software is periodically upgraded, IT and validation teams must consider new validation approaches to SaaS applications.
With a SaaS application, the vendor’s engineering and IT personnel are responsible for designing and maintaining the internal data center, computer hardware, software, security, disaster recovery, and so on. In this paper, we outline the typical steps performed by the company’s quality assurance (QA), IT, and validation teams as well as the expectations of the SaaS vendor. The shift of certain steps to the vendor makes the vendor audit a critical activity for the client’s validation, auditing, IT, and QA team, so we will share best practices related to that audit as well.

FDA Validation Overview

According to the US Food and Drug Administration, computer system validation is the formalized documented process for testing computer software and systems as required by the Code of Federal Regulations (21 CFR 11.10.a).
To be compliant and reduce the risk of an FDA Warning Letter, life science companies must validate all of their software, databases, spreadsheets, and computer systems and develop the appropriate documentation for all phases of the software development lifecycle (SDLC).
FDA validation guidance and the SDLC describe user site software validation in terms of installation qualification (IQ), in which documentation verifies that the system is installed according to written specifications; operational qualification (OQ), in which documentation verifies that the system operates throughout all operating ranges; and performance qualification (PQ), in which tests must span the underlying current good manufacturing practice (cGMP) business process.
In a similar way, EU Annex 11 also focuses on the product lifecycle and “user requirements” related to each system. However, Annex 11 places more emphasis on “people” and management accountability than FDA regulations—specifically, the individuals with responsibility for the business process and system maintenance.

Validation Activities for SaaS Applications

To address these regulatory obligations, life science companies typically perform a number of activities: creating the solution design document (SDD); developing the user and functional requirements; developing test scripts; and conducting IQ, OQ, and PQ testing, among others. This effort has the potential to dry up the IT and validation teams’ bandwidth, adding many months to the software implementation or enhancement project. In some cases, a company may not have the qualified resources to accomplish the many validation tasks.
One of the appeals of the SaaS application is that a company can shift some of the validation effort to the SaaS vendor. This enables the company’s validation team to initially focus on an audit of the vendor’s data center along with their QA and validation methodology to ensure these activities are performed at the same standard as would be performed by the client’s own QA and validation teams. Typically, the three days or “less/more” spent auditing the SaaS vendor can dramatically reduce the time spent validating the system.
Audit results may be incorporated into a risk assessment to leverage vendor-supplied documentation. In many cases, following an audit of the data center and SDLC methodology, a client’s audit and validation team will then develop the core validation documents:
• Validation Plan: Describes the internal activities that are of part of the overall validation approach to be conducted by the client.
• User Requirements Specification: Provides details on the functionality that the clients demand from the SaaS application. These required items are often categorized as “critical,” “mandatory,” or “nice to have” and are the basis for the validation/testing effort.
• User Acceptance Test Scripts: Clients should have their own test scripts that either augment vendor-owned QA test scripts or unique scripts that demonstrate that the company has tested specific usage of the system.
• Validation of Configurations and Customizations: Clients often test any unique software configurations or customized programming being developed by the vendor for the company.
• Traceability Matrix: Used for mapping user requirements to vendor documentation, UAT test scripts, and internal procedure documents.
• System Governance Procedures: May include system administration and change control for maintaining a validated state during the systems use and operation.
The following table summarizes the typical responsibilities as described above.

Auditing the SaaS Vendor: Best Practices

Clients often conduct an audit of the SaaS vendor at the vendor’s primary location, so the vendor can demonstrate adherence to quality software engineering and testing principles. The client’s validation teams will typically review QA documentation and project files that were developed as part of the design, development, testing, and implementation of the application.
Although qualifying a vendor by audit is not a new practice, as the SaaS model is more widely implemented, the method in which clients qualify the SaaS vendor must evolve to be focused on the software lifecycle process.
The company should ensure that the vendor’s quality procedures demonstrate effective planning, operation, and control of its processes and documentation. The vendor’s QA team will need to present that a valid development methodology is in place and that thorough testing was used to provide the client with confidence and assurance that the application is fit-for-production use. Furthermore, the vendor’s QA team must demonstrate that it is involved with the following activities:
• SDLC stages
• Business rules, graphical user interface design elements, and interoperability with existing features
• Test traceability and full product testing before any software enhancements are released to a production environment.
For example, below is a list of typical validation documents that SaaS vendors will be asked to present during an audit:

• Business Requirements: Focuses on the needs of the user and describes exactly what the system will do.
• Functional Requirements: Focuses on how the system will do what the user is expecting.
• System Design: Focuses on capturing the system design based on the functional requirements. It depicts screen layout, system functions, and other aspects of the user experience to fulfill the business requirements.

• Requirements Traceability Matrix: Captures the relationship between the business requirements and the system requirements, system design, and test scripts that satisfy them; this document should reflect the latest enhancements made to the platform. The SaaS vendor should ensure that documents are tracked through the use of a traceability matrix that maps the business requirements.
• Test Scripts: Used for new enhancements or custom projects. These scripts should cover basic platform functionality, audit trail functionality, reporting tools, and other high priority areas for the client. In addition, custom testing should be executed for any custom programming performed.
• Test Plan and Validation Plan: May be adopted by the vendor as standard operating procedures because the SaaS application is often multi-tenant.

• Validation Summary Report (VSR): This report summarizes vendor testing activities, discrepancies, and other validation activities.
Other key documentation that SaaS vendors may be asked to present during an audit are:
• Product release documentation
• Project management processes
• Corrective and preventive action processes
• Service delivery processes
• Product support/maintenance (including back-up processes)
• Issue tracking processes
• Internal Audits—Vendors should periodically audit their business processes and controls
• Control—Vendors should show control of system documentation and change management
• Security—Security of buildings, hardware, documents etc.
Key Questions to Ask Vendors Related to Electronic Signatures and Audit Trails
When a life science company evaluates SaaS applications, the company should develop a standard vendor questionnaire, which may include questions such as the ones below in these general categories.
Governance/System Access
• Can access to the system be controlled by qualified personnel who will be responsible for the content of the electronic records that are on the system?
• Does the vendor have a 21 CFR Part 11 assessment in place?
• Does the vendor have a distinct qualification documentation process in place, addressing different level of requirements and design
documentation?
• Does the vendor have a structured change approach in place for infrastructure, applications and interfaces, change process coverage (planning, execution, and documentation of changes), change logs, and differently involved roles?
Audit Trails
• Does the SaaS application generate an electronic audit trail for operator entries and actions that create, modify, or delete electronic records?
• Are date and time stamps applied automatically (as opposed to typed in manually by the user)?
• Is the source and format of the system date and time-defined, controlled, unambiguous, and protected from unauthorized changes?
• Do the printed name, date, time, and signature meaning appear in every printable form of the electronic record?

Electronic Signatures

• Will the electronic signature be unique to one individual and not be reused or reassigned to anyone else?
• Is the signed electronic records information shown both electronically and on printed records for the electronic signature?

Documentation

• Does the validation documentation demonstrate that 21 CFR Part 11 requirements have been met and the controls are functioning correctly?
• Is the architecture (including hardware, software, and documentation) easily available to auditors?
• Does the vendor have processes in place for documents to ensure documentation control and to maintain integrity?
• Does the vendor have a change control processes for system documentation?

Summary

Validation of a SaaS application has the potential to improve its time-to-deployment and reduce the validation effort and overall costs for the life science organization. However, the validation team within the organization must expect the same commitment to validation activities from the SaaS vendor. To summarize, here are the key steps that can ensure a successful SaaS application validation project:

1. Audit/assessment of supplier’s capabilities, quality system, SDLC, customer support, and electronic record-keeping controls
2. Development of the user requirement specifications and validation plan
3. Configuration of supplier system to meet user requirements (configuration specification)
4. Vendor testing and release to customer
5. Personnel training
6. User acceptance testing/traceability matrix
7. Develop system administration, operation, and maintenance SOPs
8. Validation summary report and release of system for production use.