Quality Assurance: Software Quality Standards

Software quality assurance (SQA) operates within a structured landscape of international standards, federal procurement requirements, and industry frameworks that govern how software products are developed, tested, documented, and released. The field intersects regulatory compliance, engineering discipline, and risk management across sectors from defense contracting to commercial healthcare applications. This reference covers the definition and scope of software quality standards, their structural mechanics, classification distinctions, and the tensions that practitioners and organizations must navigate.

• Definition and scope
• Core mechanics or structure
• Causal relationships or drivers
• Classification boundaries
• Tradeoffs and tensions
• Common misconceptions
• Checklist or steps (non-advisory)
• Reference table or matrix

Definition and scope

Software quality standards define the minimum acceptable criteria for the processes, work products, and organizational capabilities involved in software development and maintenance. The ISO/IEC 25010:2011 standard (part of the SQuaRE — Systems and Software Quality Requirements and Evaluation — series) establishes a quality model with 8 top-level characteristics: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability. Each characteristic is further decomposed into sub-characteristics, giving organizations a measurement vocabulary rather than a pass/fail threshold.

Scope extends from the organizational level (how a software development organization manages its quality system) down to the product artifact level (what measurable properties a software deliverable must exhibit). Federal agencies apply scope through procurement instruments: the Department of Defense applies MIL-HDBK-61B for configuration management of software items, and the FDA regulates software as a medical device under 21 CFR Part 11 and its Software as a Medical Device (SaMD) guidance, which directly references IEC 62304 for software lifecycle processes.

The ASQ (American Society for Quality) recognizes the Certified Software Quality Engineer (CSQE) credential as the primary professional qualification in this space, covering standards literacy, auditing, testing, and metrics.

Core mechanics or structure

Software quality standards operate through 3 interlocking mechanisms: process requirements, product requirements, and verification requirements.

Process requirements specify how development activities must be conducted. IEEE Std 730-2014, Software Quality Assurance Processes, published by the IEEE Standards Association, defines the minimum content and activities required in an SQA plan, including configuration management interfaces, problem reporting, and risk management tasks.

Product requirements specify measurable attributes of the software artifact itself. ISO/IEC 25010 provides the quality model; ISO/IEC 25023 provides the corresponding measures. A reliability sub-characteristic such as "fault tolerance" is quantified using defined metrics (e.g., the percentage of test cases where the system continues to operate after injected faults).

Verification requirements specify how conformance is demonstrated. This includes static analysis, code review thresholds, test coverage requirements, and audit trails. The NIST Cybersecurity Framework intersects software quality at the verification layer by requiring organizations to identify, protect, detect, respond to, and recover from incidents — a lifecycle that depends on documented software quality practices.

The CMMI Institute's CMMI for Development (CMMI-DEV) frames maturity across 5 levels. Level 2 ("Managed") requires documented and repeatable processes; Level 3 ("Defined") requires organization-wide standards; Levels 4 and 5 add quantitative management and optimization. Federal contractors in defense and intelligence sectors are frequently required to achieve CMMI Level 3 or higher as a contract prerequisite. More detail on process maturity appears on the CMMI Framework reference page.

Causal relationships or drivers

Software quality standards proliferate in direct response to documented failure modes. The 1994 Ariane 5 rocket failure, caused by a floating-point conversion error in reused software from an earlier rocket model, established that software from a prior system cannot be assumed to meet quality requirements for a new operational context without revalidation — a principle now encoded in IEC 62304 §8.1 on software change management.

Regulatory pressure is the dominant external driver. The FDA's 2023 final guidance on cybersecurity in medical device premarket submissions requires a software bill of materials (SBOM) and a documented secure product development framework (SPDF). Organizations that lack an SQA infrastructure aligned to IEC 62304 cannot generate the required artifacts for FDA 510(k) or PMA submissions.

In the federal civilian sector, the Office of Management and Budget (OMB) Memorandum M-22-18 (September 2022) requires agencies to only use software that conforms to NIST SP 800-218 (Secure Software Development Framework, SSDF). Agencies must attest to this conformance for all software used in federal operations, making SSDF alignment a procurement gate rather than a best practice.

Classification boundaries

Software quality standards divide along 4 primary axes:

1. Domain-specific vs. domain-agnostic: ISO/IEC 25010 applies across all software; IEC 62304 applies exclusively to medical device software; DO-178C applies exclusively to airborne software. Domain-specific standards carry mandatory status within regulated sectors and supersede general standards where they conflict.

2. Process standards vs. product standards: IEEE 730 and CMMI-DEV govern how work is conducted. ISO/IEC 25010 and its companion standards govern what the software must be. An organization can comply with process standards while producing a low-quality product if the product measurement layer is absent.

3. Mandatory vs. voluntary: Standards referenced in federal regulations or FDA guidance carry de facto mandatory status for organizations seeking market access. ISO 9001 (quality management systems) is voluntary except where contractually required, though many procurement contracts mandate ISO 9001 certification or equivalent.

4. Static vs. dynamic quality attributes: Maintainability, portability, and security are largely assessed through static analysis and architectural review. Performance efficiency and reliability require dynamic testing under operational conditions.

For a structured view of how these boundaries interact with regulatory frameworks, the classifications above are foundational to determining which standards apply in a given context.

Tradeoffs and tensions

Coverage vs. velocity: Comprehensive test coverage requirements (e.g., MC/DC — Modified Condition/Decision Coverage — required at DO-178C Level A) impose significant engineering overhead. For avionics software, achieving MC/DC can require test suites that exceed the source code volume by a factor of 3 or more. Agile development methods, which prioritize short delivery cycles, create structural tension with the document-heavy, stage-gated requirements of standards like IEC 62304 Class C or DO-178C Level A.

Standardization vs. innovation: The IEC 62304 standard was first published in 2006. Software architectures based on machine learning models, containerized microservices, and continuous deployment pipelines do not map cleanly to the lifecycle model assumed by IEC 62304 or DO-178C. The FDA's 2021 action plan for AI/ML-based SaMD acknowledges this gap explicitly, noting that predetermined change control plans (PCCPs) are required for adaptive algorithms — a mechanism that the existing standard lifecycle framework did not originally anticipate.

Auditability vs. agility: ISO 9001:2015 §8.5.1 requires documented information as evidence of process execution. In high-velocity software environments, maintaining audit-ready documentation for every sprint cycle creates resource burdens that organizations frequently underestimate until an audit occurs.

Third-party vs. first-party assurance: Standards permit self-declaration of conformance (ISO/IEC 29119 for software testing) in many cases, but regulated sectors (medical devices, aviation) require independent third-party audit. This creates a two-tier market where organizations serving regulated sectors bear significantly higher compliance costs than those in unregulated commercial software markets.

Common misconceptions

Misconception 1: ISO 9001 certification means software is high quality.
ISO 9001:2015 certifies that a quality management system exists and is followed — it does not certify that the software product meets any defined quality level. An ISO 9001-certified organization can produce defective software while remaining in full conformance with the standard, as long as the defects are documented and processed through corrective action procedures.

Misconception 2: Testing is software quality assurance.
Testing is one verification activity within an SQA system. IEEE 730-2014 defines SQA as encompassing planning, configuration management, problem reporting, supplier control, records collection, and training — none of which are testing activities. Equating QA with QC (quality control, which includes testing) conflates two distinct functions.

Misconception 3: Higher CMMI maturity levels guarantee better software.
CMMI maturity levels measure process institutionalization, not product quality outcomes. The CMMI Institute explicitly states that CMMI is a process improvement framework, not a product certification. A Level 5 organization can still produce software with significant defects; the framework predicts process predictability, not defect-free output.

Misconception 4: Open-source software components do not require quality evaluation.
OMB M-22-18 and NIST SP 800-218 apply equally to open-source and proprietary software used in federal systems. The SBOM requirement ensures that component-level quality and vulnerability tracking extends to open-source dependencies regardless of licensing model.

Checklist or steps (non-advisory)

The following sequence reflects the standard phases in establishing an SQA system aligned to IEEE 730-2014 and ISO 9001:2015:

1. Scope definition — Identify the software items, development lifecycle model, and applicable domain standards (e.g., IEC 62304 for medical, DO-178C for aviation).
2. SQA plan development — Draft a Software Quality Assurance Plan (SQAP) per IEEE 730-2014 §6, identifying activities, responsibilities, and resources.
3. Standards and procedures selection — Reference applicable standards (ISO/IEC 25010 quality model, NIST SSDF if federal-facing) and document the selected coding standards, testing standards, and review procedures.
4. Metrics baseline establishment — Define measurable quality objectives using ISO/IEC 25023 measures or equivalent, covering reliability, security, and maintainability characteristics.
5. Configuration management integration — Establish linkage between the SQA system and the configuration management system per IEEE 828-2012 (Standard for Configuration Management in Systems and Software Engineering).
6. Internal audit schedule — Schedule periodic SQA audits per the SQAP. Audit records must be maintained per ISO 9001:2015 §9.2.
7. Nonconformance processing — Establish a problem reporting and corrective action process per IEEE 730-2014 §6.8 and document the resolution of each nonconformance.
8. Supplier/tool qualification — Qualify development tools and third-party software components; document qualification evidence per DO-178C §12 (if applicable) or equivalent process.
9. Records and retention — Archive SQA records per applicable regulatory retention requirements; FDA 21 CFR Part 820 requires design history file records to be retained for a period of 2 years beyond the product's commercial distribution.
10. Management review — Conduct periodic management reviews of SQA effectiveness per ISO 9001:2015 §9.3, with documented outputs including decisions on resource allocation and process changes.

Reference table or matrix

References

• ISO/IEC 25010:2011 — Systems and Software Quality Requirements and Evaluation (SQuaRE)
• IEEE Std 730-2014 — Software Quality Assurance Processes
• IEC 62304:2006+AMD1:2015 — Medical Device Software Lifecycle Processes (FDA reference)
• NIST SP 800-218 — Secure Software Development Framework (SSDF)
• OMB Memorandum M-22-18 — Enhancing the Security of the Software Supply Chain
• FDA — Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions (2023)
• CMMI Institute — CMMI for Development
• ASQ — Certified Software Quality Engineer (CSQE)
• NIST Cybersecurity Framework
• ISO 9001:2015 — Quality Management Systems