When is a (new) summative evaluation needed?

How design changes impact usability

A lot of medical device manufacturers often wish to update their devices through the product life cycle. This can be software updates or minor aesthetical or functional modifications to the design. However, regardless of the update to be implemented, all changes must undergo an impact assessment in terms of their impact on the safety and effectiveness of the device. This article will introduce you to assessing the usability impact of your design change and thereby plan for any needed summative evaluation. Let’s look at a few examples:

How should these design changes be handled from a Usability Engineering perspective and which activities should be planned when the device is already on the market?

The focus of Usability Engineering for medical devices is safety and effectiveness. Therefore, the evaluation must assess the impact on these parameters. This focus on safety and effectiveness is not unique for changes to the device user interface but are in focus of all design changes. Many manufacturers choose to have a checklist for assessing impact of all design changes. It is also important to document why something does – or does not – impact e.g. the usability engineering file.

In some cases, a more thorough assessment is needed. Thorough analysis and the involvement of a Usability Engineer is often necessary to assess whether the design changes impact how representative users use the device safely and effectively. Technolution recommends assessing design changes from a Usability Engineering perspective in the following way:

Impact assesment Yes No Comment / Rationale
Does the design change impact the Intended Use?
X
No changes to intended use.
Does the design change impact the Use Specification?
X
No changes to user groups, patient groups, use environment, operating principle or indications for use.
Does the design change impact the Labeling changed?
X
No changes to labeling.
Does the design change impact the Design Verification?
X
Design verification must demonstrate that the design input requirements are still met when implementing the new screen.
Does the design change impact the Design Validation?
X
If the technical performance of the screen is verified according to the design input requirements, not further design validation is necessary as the conclusion of the existing design validation is still valid.
Does the design change impact the User Interface (UI)?
X
No changes to the user interface.
Does the design change impact how the device is used?
X
No changes to how the device is used.
Does the design change impact the Requirement Specification?
X
No changes to the requirements specification.
Does the design change impact the Risk Analysis?
X
No changes to the risk management file.
summative evaluation
Does the UI change involve a safe-related task?

The first question to answer is whether the design change is related to a critical task, i.e. a task if performed incorrectly – or not performed at all – could cause serious harm to the patient or user. If not, it should also be assessed whether this design change introduces a new risk.

Has the existing UI been evaluated?

If the device has never been evaluated in a summative evaluation and it is related to a critical task, it is recommended to perform a summative evaluation. It is assumed that critical tasks have been evaluated for already marketed devices. If the device has already been evaluated, the conclusion of this evaluation may still be valid.

Does the design change impact safety-related use?

To answer this question, the involvement of an experienced usability professional may be needed. If the design change affects the safe and effective use of the device, a summative evaluation shall be performed as described in both IEC 62366-1:2015 and FDA’s guidance on Human Factors Engineering from 2016. If the design change does not influence how intended users perform critical tasks using the device, a thorough assessment must be documented. Such an assessment may use existing usability data from the previously performed summative evaluation.

Evaluating the impact of design changes on how a device is used safely and effectively

Let us look at a couple of possible outcomes of the impact assessment:

We wish you good luck and success in assessing your design changes. Remember to include all aspects in your impact assessment – also Usability Engineering. Please don’t hesitate to contact Technolution for any questions or inquiries.

feel free to contact

Morten Purup Andersen

Senior Development Engineer / HFE Specialist
Morten, writer and expert in summative evaluation
Want to know more?

Insights you might also be interested in: