Keep CI small

A CI pipeline with 6,000 tests in under 10 minutes - sounds utopian, but it can be done. The key lies in intelligent test selection: only the tests that really cover the changed code are run, prioritized according to proximity to the commit, error history and test runtime. The technical implementation required custom integration with PyTest, code coverage mapping via API and a nightly update system - all developed in a small team with limited resources. Software testing can be fast if you have the courage to tailor existing tools to your own needs.

Podcast Episode: Keep CI small

Lars and his team have developed a concept for keeping the CI pipeline at Dolby under 10 minutes despite a constantly growing test suite. They rely on various automation tools that not only save time, but also minimize sources of error. Lars explained in detail how these various tools were integrated into their development process to create a seamless and automated workflow chain.

“In the 10 minutes you have to build all the backends (…) you have Linux, Windows, Arm, 64bit (…) unittest and testers” - Lars Kempe

Lars began his professional career as an electrician, a job he held for eight years, including his apprenticeship. After further training in Dresden, through which he became a state-recognized technician and obtained a technical college entrance qualification, he moved to Nuremberg. There he started his career in the field of mobile communications technology as a system tester at Lucent Technologies. He then successfully completed his bachelor’s degree and returned to the mobile communications sector, where he worked as a tool developer and tester. In 2011, he moved to Dolby, where he started as a tester and currently works as QA Lead in the automotive division.

Highlights der Episode

• CI pipeline stays under 10 minutes through automatic test selection based on source code changes with TeamScale.
• Parameterization in PyTest quickly turns a test case into 50-100 cases - granular code coverage becomes essential.
• Nightly coverage updates create a maximum one-day gap - manual updates in between are sufficient for rapid development.
• Time-based testing budget (5 minutes) prevents pipeline overflow better than fixed test count limits.
• Small teams with multiple roles integrate tools faster - QA Lead programs the PyTest plugin integration himself.

How Dolby optimizes its CI pipeline: Insights into speed and efficiency

In this episode, I talk to Lars, QA Lead at Dolby, about the challenges and solutions of CI pipeline optimization for maximum efficiency. Find out how Dolby builds a fast and effective pipeline and what role test automation plays in this.

The desire for speed

Today I was able to talk to Lars Kempe, the QA Lead at Dolby, about a topic that is close to the hearts of many in software development: How do you keep the Continuous Integration (CI) pipeline lean and efficient? We are all familiar with the Dolby brand - be it through surround systems or movie music. But there’s more to the iconic logo than just outstanding audio quality; there’s also a sophisticated strategy for optimizing your CI pipeline.

The challenge: speed versus quality

Lars began by talking about the challenges he faced. One of the main goals at Dolby has always been to ensure fast CI. But how do you achieve this without compromising the quality of the software? Lars explained that they had started to develop a new project from scratch with the goal of fast CI. With a mixture of experience from previous projects and the introduction of the Team Scale tool, they were able to successfully implement this approach.

The key to success: test automation

One of the key strategies to accelerate the CI pipeline at Dolby was to fully automate their testing. The new project started with a few hundred tests, but thanks to Pytest they were able to quickly generate a high number of test cases. This allowed them to design their pipeline efficiently while ensuring comprehensive coverage. Parameterization played a central role in this - numerous test cases could be generated from a single test function.

Integration and optimization: a continuous process

The integration of Team-Skill into their pipeline was not without its hurdles. Lars shared with us the difficulties and effort required to ensure a seamless integration. Adapting to their specific environment required development work and creative solutions. However, through close collaboration with team skill support and persistent dedication, they eventually managed to create a pipeline that is both fast and accurate.

Focus on efficiency: time as a decisive factor

One of the most interesting aspects of Dolby’s approach is the focus on time as the main criterion for efficiency. By setting a maximum run time for tests within the CI - in their case 10 minutes - they were able to create a process that was both quick to provide feedback and thorough. This rigorous time constraint forced the team to constantly work on the efficiency of their tests and ensure that only the most relevant tests were performed.

Future plans: continuing on the path to perfection

To conclude our conversation, Lars spoke about Dolby’s future vision for their CI pipeline. Despite the success they have already achieved, Lars sees room for improvement. Particularly in the area of merge requests, they are striving to optimize the process even further. The goal remains the same: to maintain a fast, efficient pipeline that enables the team to bring high-quality software to market quickly.