9 criteria for the right test management tool in medical device development

Why testing is different for medical devices

ISO 13485 defines the requirements for the quality management system and is effectively a prerequisite in the EU for conformity assessment under the MDR. Software is governed by IEC 62304, with three software safety classes: A (no injury possible), B (non-serious injury) and C (serious injury or death). The higher the class, the stricter the obligations. Unit verification is required from Class B upwards, while detailed software design for each unit is only required for Class C. The class is derived from risk management in accordance with ISO 14971, which therefore determines the entire verification effort. Complementary standards include IEC 62366-1 (usability), IEC 81001-5-1 (cybersecurity) and AAMI TIR45, which describes how agile methods can be reconciled with IEC 62304. IEC 62304 is currently being revised; the second edition replaces the classes with process rigour levels.

Three principles before you choose a tool

Test strategy before tool. The tool maps your process, not the other way round. Before choosing one, you should have clearly defined test levels with responsibilities, risk-based test depth, verification criteria for each requirement and a re-verification strategy in place.

Test management is not test automation. Automation executes tests and generates results. The tool orchestrates and provides evidence: what was tested, against which requirement, in which environment, with what result and with what release. The aim is an evidence layer on top of a largely automated test landscape.

Appropriateness rather than bureaucracy. The standards demand appropriateness, not maximum documentation. Too much documentation is just as much an anti-pattern as too little.

Nine criteria for a suitable test management tool

1. End-to-end traceability
Requirement, risk control measure, test case, result and anomaly are linked bidirectionally and can be evaluated. This lets you see immediately which requirements are untested and which risks are unverified. Isolated solutions alongside requirements management mean manual maintenance and audit risk.

2. Audit-proof records and baselines
Every test case has an immutable audit trail, and every test status is clearly assigned to a software version. Even years after release, it must be possible to reconstruct what was run in which version and with what result. Completed results remain unchangeable; corrections are only made as a new, justified entry.

3. Test levels and risk-based test depth
Unit, integration and system tests can be reported separately, in line with the verification levels of IEC 62304. The depth of testing depends on the risk.

4. Anomaly and change management
A failed test leads directly into the problem resolution process, including safety assessment and re-test tracking. If an approved requirement changes, the tool shows the affected test cases and traceability chains as the basis for impact analysis.

5. Approvals, roles and permissions
Reviews and second-reviewer sign-offs are documented. If your QMS requires a four-eyes principle, the tool enforces it technically. Otherwise, a gap opens up between the documented process and the one actually practised – a standard audit finding. Where the FDA is relevant, electronic signatures in accordance with 21 CFR Part 11 are also required.

6. Result import and test environment as metadata
Results come automatically from the CI pipeline, not by hand. For every run, the DUT version, HIL configuration, compiler and test bench set-up are recorded. This is crucial for reproducibility on hardware. It should also be possible to map variants for hardware derivatives.

7. Suitability for Scrum
Test cases can be created directly from stories in the issue tracking system and handled within the sprint, and the re-test workflow is lean. The more manual maintenance a tool requires, the more likely the team is to work around it – and the traceability no longer holds.

8. Long-term availability
The MDR requires technical documentation to be retained for at least ten years, and fifteen years for implantable devices. The tool needs exportable, stable formats. Vendor lock-in is a real risk.

9. Validated tool
The tool itself must also be validated for its intended use (ISO 13485, 4.1.6; IEC/TR 80002-2), including risk-based revalidation for upgrades. Vendors that supply validation documentation save considerable effort here.

Scrum and embedded: where the details matter

Scrum and IEC 62304 are compatible, but they need to be deliberately interlinked. The release train model has proven effective: several sprints work towards a release, with formal activities such as design reviews, completion of verification and documentation freeze taking place at the release boundaries. The Definition of Done is extended to include regulated activities, such as code review, passed unit tests, error-free static analysis, updated traceability and documentation maintained within the sprint. Typical pitfalls are postponed documentation, changes to verified requirements without impact analysis, and a forgotten maintenance process.

Embedded software brings additional specifics: the test pyramid ranges from host unit tests with mocks through HIL integration tests to system tests on the real device. Hardware-software integration must be explicitly addressed in the verification strategy. Compilers, static analysis and test frameworks must themselves be qualified, and the toolchain is versioned so that builds remain reproducible.

Where AI is already supporting test management today

At present, AI delivers the greatest benefit in the quality analysis of requirements: it detects ambiguities, missing verification criteria and testability issues – in other words, errors that come to light before testing. Also helpful are draft test cases generated from requirements, which the team reviews and adopts, the detection of duplicates, and summaries of test results for release readiness reports. Responsibility for the evidence remains with people.

Conclusion

A good test management tool provides evidence of what was tested against which requirement, in which environment and with what result, and who released it. It is built on a clear process: test strategy first, traceability as the backbone, evidence as extensive as the risk requires and as lean as possible. Automate rather than document.

NewTec supports manufacturers in aligning test strategy, process and tool landscape for medical device software – from classifying the requirements of the standards to audit-ready evidence in everyday Scrum practice.