Agile medical device development is compatible with ISO 13485 and IEC 62304, as long as the agile process is adapted to the regulatory requirements: documentation, reviews and testing belong in every iteration. A manufacturer of cardiology devices describes how this worked, first in a pilot project and then in two large projects.
Experience from a Medical Device Manufacturer
After 25 years of developing medical devices and software for cardiology diagnostics and for the outpatient monitoring of vital signs in high-risk patients, we have built a consistent and complete quality management system for our company. It includes a structured, documented development process for our hardware and software that meets regulatory requirements such as DIN EN ISO 13485 and DIN EN IEC 62304.
As the organization, and research and development in particular, kept growing, this process reached its limits. We had to improve it so we could run projects more efficiently and shorten our development cycles. Inspired by industry best practices, by our own experience, by conferences and by the literature, we decided to investigate and evaluate agile development processes.
Agile Medical Device Development: Choosing the Process
The main difficulties in choosing an agile development process are meeting the regulatory requirements and producing the necessary technical documentation. Traditional development processes were often linked to regulatory requirements and referenced them directly. Most agile processes make no reference to regulatory requirements at all, and certainly not to those for medical device manufacturers.
A strict, by-the-book implementation of one agile process was therefore not an option. Instead, we decided to take methods, ideas and approaches from several agile processes and adapt them to our own development process, with the regulatory requirements in mind. In other words, we selected agile principles and fitted them into our process, on the condition that we kept to the relevant standards.
We did not roll the new process out across the whole organization. We introduced it into our daily work through a pilot project that covered every relevant aspect of a typical development project in our company:
- hardware, software and firmware development
- medium complexity
- a broad range of departments and roles involved
Our main sources for the agile ideas, methods and approaches were:
- Manifesto for Agile Software Development
- Kanban: Successful Evolutionary Change for Your Technology Business
- Agile Testing: A Practical Guide for Testers and Agile Teams
- Implementing Lean Software Development: From Concept to Cash
- Basiswissen Medizinische Software (the German textbook for the Certified Professional for Medical Software)
- The Scrum Guide
Proven Practices and Their Limits
Agile processes offer a wide range of methods and best practices. While we introduced and refined our new process, we identified the practices that have paid off most for us since the switch. At the same time, there are limits, mainly because of the regulatory requirements, and we have to keep them in view at all times.
Planning in Small Iterations
The biggest effect on our project management, both on how we steer a project and on how efficiently the development team works, came from splitting the project into small iterations. For the big picture and overall planning, we still set up a traditional project plan. For the operational work of the development team, though, that plan is broken down into small iterations, or sprints. A sprint lasts two to four weeks, and the planned work has to be finished within it.
Each iteration is planned in a separate planning meeting at the start of the sprint. The product owner (our product manager) explains the highest-priority requirements and discusses them with the development team, which is made up of developers and testers. That way, everyone involved, product owner, developers and testers, shares the same understanding of the requirements. Based on that shared view, the development team decides what it can deliver in the sprint.
Delivering part of the product means more than finishing the development work. The following tasks also have to be completed within the sprint:
- design specification and its review
- unit and component tests
- refactoring of existing parts
- review of source code, schematics, etc.
- integration and system testing
- regression testing of existing parts
This ensures that what comes out at the end of the sprint is not a quick-and-dirty implementation, but an artifact with assured quality and consistent documentation. At the end of the sprint, the development team presents its results and the product owner reviews them, giving feedback directly to the team.
There are plenty of best practices for organizing planning meetings and sprints and for estimating the work of the next sprint. Our experience shows, though, that getting the project to work in small iterations matters far more than the details of planning and estimation. The methods for organizing and managing the iterations can change during the project and should be adapted to what the development team needs.
The big advantage of working in iterations like these is that you can check progress more often, review the development results during the project and keep the development team and the product owner aligned on the requirements.
Communication and Collaboration
Over the previous years of development, team members had focused more and more on “their” own work. For the organization, that led to two big problems:
- Parts of the code, schematics and so on were understood by only one or two developers and could not be maintained by anyone else. When problems came up while those people were ill or on vacation, analyzing them was very difficult.
- Developers sometimes got lost in solving problems that someone else on the team could have solved much faster. That cost a lot of efficiency.
We chose two approaches to avoid these problems: the daily meeting and continuous reviews.
Daily Meeting
In a daily meeting, time-boxed to around 15 minutes, each team member tells the others what they have done since the last meeting, what they will do until the next one and whether anything is blocking their work. With this short meeting, which can be held as a stand-up in the morning, everyone knows what is going on in the project. It reduces the risk that someone gets stuck on a problem and wastes time on it when a colleague could solve it quickly. It is important that the meeting is not seen as a “report to the project manager”, but as a transfer of know-how to the rest of the team.
Continuous Reviews
Part of the definition of done for each sprint is that every artifact (design documents, source code, schematics and so on) is reviewed by at least one other team member within the sprint. Depending on complexity and risk, some documents are reviewed formally, others informally. The result is a continuous quality check of the artifacts and broad knowledge of them across the team. Since we introduced continuous reviews, the quality of our development teams’ artifacts has risen noticeably. Some developers went a step further and started pair programming, with two developers working on one problem at the same workstation.
Tools and Traceability with Polarion ALM
While introducing the new process, we evaluated whether our existing tools could support it. Our main rule was to define a part of the process first, the way it suits our needs, and only then decide which tool could support it. The development tools themselves stayed largely as they were.
To manage our entire development process, we chose an application lifecycle management tool: Polarion ALM. Polarion is very generic, and almost every part of it can be customized. It is built on the Subversion version control system and works with work items (requirements, design, test cases, anomalies, etc.) that can be linked to each other and to other artifacts such as source code. The main benefits for us were:
- Traceability across items: we can show the implementation path, for example from a requirement through the design to the test cases, the defects found and the corresponding source code. That makes a simple impact analysis possible when an item changes. This traceability is also a core part of the regulatory requirements for medical device manufacturers.
- Traceability over time: because Polarion is built on Subversion, you can go back to a set of work items as it was at a specific point in the past. You can, for example, look at the requirements as they were in an old release and show the changes between that release and the current state of development.
- Customization: Polarion lets us adapt the tool to our process requirements. A wide range of templates, APIs and configuration options is available to design a workflow that fits.
Task management for the development teams was another big problem. Different tools were used for tasks, Microsoft Outlook and Polarion among them, but neither was suited to tracking, changing and working through tasks quickly. As a result, the developers simply didn’t use them. After a few iterations we found a truly lightweight tool for managing tasks: a plain whiteboard with sticky notes. Each note is a task with the status open, in progress or done. The notes also reference the relevant items in Polarion, such as defects or requirements. The whiteboard is ideal for the daily meeting too, because it shows the current state of the iteration.
Team Motivation and Mindset
More than in traditional development projects, the team members’ soft skills and the team’s motivation decide whether the project succeeds. Everyone on the team has to engage with the new methods and approaches for the process improvement to work. Our development team’s experience shows that skeptical team members usually had not yet seen the benefit of things like small iterations. The best argument is the demo in the review meeting, iteration after iteration.
Agile methods often include a playful element that motivates the team. A few ideas that inspired us:
- a daily stand-up outside the office building
- motivation cards for team members
- bug-hunting sessions
- planning and estimation poker
- small challenges
But the whole change is not just a process change. It also touches the mindset and culture of the organization. For a development team used to a traditional process, that is not always easy, and it doesn’t happen overnight, especially when developers have worked only in their own domain and interacted little with the team.
How Many Iterations Until Agile Pays Off
Everyone has to understand that the switch takes time and that efficiency may drop at first. In our projects, it took five to seven iterations before the team worked more productively and more efficiently than before.
Regulation and Documentation
One special feature of medical device development is the sheer number of regulations that have to be met. A manufacturer that wants to sell worldwide also has to satisfy many regional standards to be approved in those countries. On top of the product standards there are process standards that require traceable, complete documentation from the first step of the project to the last.
Quality and risk control are another aspect. A medical device has high demands on safety and reliability, which calls for a broad range of quality assurance throughout the project, from reviews through static analysis to dynamic testing.
These requirements for documentation, quality assurance and risk control have to be part of your process in every iteration. When we started defining our new development process, this looked like a showstopper for moving to an agile process. Our experience shows, however, that these regulatory requirements can be met in an agile process as well. In two areas, the agile process even has advantages over the traditional one for us:
- Documentation: under our definition of done, the documentation also has to be finished at the end of every iteration. That means the documentation (requirements, design specification, test cases, etc.) has to be consistent with the development artifacts and reviewed within the sprint. The result is complete, consistent, quality-checked documentation at the end of every iteration.
- Test automation: just like the documentation, test automation grows with every iteration and is adapted continuously.
In both cases, our experience is that within a few iterations it is no longer only the tester who aims to finish the documentation and test tasks within the iteration. The whole team starts working hard on documentation and test automation. In addition, a colleague from quality management joins the agile development team to support these activities.
Conclusion
Moving to an agile development process has worked for us as a medical device manufacturer. The regulatory requirements force you to think through every step of the process change, mainly because agile methods and approaches don’t put the focus on documentation and regulation that the medical field needs. So an adapted agile process is required. The methods, ideas and approaches established in agile development can be used to improve an existing development process. For us, the result is a standards-compliant and more efficient development process, which is now used in two large projects developing medical hardware and software.
Frequently Asked Questions
Why can’t an agile framework simply be adopted as-is in the medical technology sector?
A strict agile framework did not meet the requirements. Traditional development processes were often directly linked to and referenced by regulatory requirements, whereas most agile processes in 2012 made no reference to them, let alone to the requirements for medical device manufacturers. The manufacturer therefore selected individual agile methods, ideas, and approaches and adapted them to its own process, with the requirement to comply with the relevant standards.
How do you implement a process change without immediately overhauling the entire organization?
Through a pilot project. The new process was not rolled out organization-wide but was first tested in a project that covered all relevant aspects of a typical development project: hardware, software, and firmware development; moderate complexity; and a broad spectrum of departments and roles involved. This allowed the methods to be evaluated in day-to-day work before they became standard practice.
What must be completed within a sprint for a deliverable to be truly finished?
Significantly more than just implementation. A sprint lasted 2 to 4 weeks, and its scope included design specifications along with their review, unit and component testing, refactoring of existing components, review of source code and schematics, integration and system testing, as well as regression testing of existing components. As a result, at the end of the sprint, the team didn’t produce a “quick and dirty” artifact, but rather a deliverable with assured quality and consistent documentation.
What helps prevent silos of knowledge when only one or two developers understand the code or schematics?
Daily coordination and continuous reviews. In the daily meeting (limited to about 15 minutes), each team member reports on their progress, next steps, and any roadblocks. It is important that this be understood as a transfer of expertise to the team and not as a report to the project manager. In addition, at least one other team member reviews each artifact in the sprint, formally or informally, depending on its complexity and risk.
Should you choose a tool first, or define the process first?
The process first. The main rule was to define a part of the process to suit your own needs and only then decide which tool could support it. For task management, heavyweight tools failed because the developers didn’t use them. A whiteboard with Post-it notes proved effective there, as it also made the status of the iteration visible.
How quickly does switching to agile work pay off?
Not immediately. It affects not only the process but also an organization’s mindset and culture, especially if developers previously worked only within their own domains. Efficiency may actually decrease at first. In the projects described, it took 5 to 7 iterations before the team was working more productively and efficiently than before. The presentation during the review meeting, iteration by iteration, was most effective in convincing skeptical team members.
How can you prevent documentation and test automation from falling behind in short iterations?
By making both part of the “definition of done.” At the end of each iteration, requirements, design specifications, and test cases had to be consistent with the development artifacts and verified within the sprint. Test automation also grew with each iteration and was continuously adapted. After just a few iterations, the entire team was working on it, not just the tester, supported by a member of the quality management team.
