Dynamic analysis of embedded systems means observing software while it runs on real hardware under real-time conditions, because simulations cannot fully capture the physical and timing properties of embedded systems. The gold standard for this is embedded trace: a hardware interface in the microcontroller that streams the control flow to the outside without affecting the application. For the first time, this makes it possible to measure code coverage directly in integration testing.
Key Takeaways
- Embedded trace reads out a microcontroller’s control flow through dedicated hardware interfaces without changing the application’s execution time, something classic instrumentation fundamentally cannot do.
- Measuring code coverage in integration testing on real hardware is now technically possible. It solves the core problem that coverage could previously only be measured for unit tests in isolation, on the PC or on the target.
- Instrumented code inflates memory requirements so much that safety-critical embedded systems simply have no room left for it, because on average every fifth instruction is a decision that needs its own counter.
- Embedded testers need both electrical engineering and computer science skills, because a bit won’t be set if the circuit behind it isn’t properly closed or opened.
- Safety standards such as DO-178C already accept coverage evidence from integration testing, yet in practice most teams still meet it at unit level, even though that costs more and tells you less.
What Makes Embedded Testing Harder Than Testing Web Applications
Embedded testing differs from classic software testing mainly in three ways: scarce resources, real-time behavior and a tight coupling with the electronics. That is why dynamic analysis on the real hardware matters so much here, and why anyone testing embedded systems needs to know the basics of electrical engineering.
Memory is small, space is tight and the timing requirements are strict. A bit only gets set in memory if the circuit behind it is right. If you can’t read a circuit diagram and don’t know Ohm’s law, you can’t test meaningfully at this level.
Then there’s the nature of the application. An embedded system runs in real time. If you ignore that real-time behavior in testing, you may end up testing something other than the system that will actually run later.
The profession reflects this. Many embedded developers come from electrical engineering rather than pure computer science. Martin Heininger studied electrical engineering himself. Both worlds have to come together, and you have to want exactly that from the start.
Why the Hardware Is the Ground Truth
In safety-critical embedded work, the hardware is the only reliable test environment. A simulation isn’t enough for acceptance, because its correctness is almost impossible to prove.
You can certainly test on the PC, especially at unit level and under certain conditions. But the closer you get to system and acceptance testing, the less you can avoid the real electronics.
The reason is the question every safety case raises: can you prove that the simulation reflects reality? In practice, you rarely can. So you test on the physical system.
On top of that, hardware has become more complex. There used to be one CPU with external memory, clear models and predictable execution times. A piece of software needed a certain number of CPU cycles and was done at a defined point in time.
In modern multicore systems with caches, processes push each other out of memory. Determinism is gone, or can only be restored with a lot of effort. For many modern SoCs, accurate models simply don’t exist anymore. What’s left is trying things out on the real system, combined with the ability to look inside it.
Where Embedded Testing Differs from IT Testing
Three areas clearly set embedded testing apart from classic IT testing: volume testing, code coverage requirements and observability.
Volume testing with huge databases and real personal data doesn’t exist in that form in the embedded world. What counts as load testing there is closer to a stress test on a bus such as CAN or Ethernet, with comparatively little data. An IT tester would barely notice those data volumes.
With code coverage, it’s the other way around. In IT, 100 percent coverage of every line or even every condition is neither common nor sensible, given the sheer size of the systems. In a safety-critical environment, the opposite applies: a defect in the code causes a real problem. So everything has to be tested.
Security follows a different pattern, too. Here IT is ahead, because it has more experience with the relevant tests and protection concepts. In the embedded world, the topic is only just getting started. A common strategy: if there is an interface to the outside, the larger system on the other side has to protect the small embedded system.
Why Instrumentation Hits Its Limits in Integration Testing
Instrumentation works very well for isolated unit tests, but it becomes a problem in system and integration testing. It changes the very system you want to test.
Here’s how it works: roughly every fifth instruction is a decision that has to be recorded. Behind each decision sits a counter that is read, incremented and written back on every pass. That bloats the code considerably.
This has two unpleasant consequences. First, the instrumented code often no longer fits into the limited memory. Second, the execution time changes. A read-modify-write in the cache is fast, but in external memory it can easily take 100 times longer.
In a system with guaranteed response times or interrupt reactions, that tips the behavior over. In the end, you are testing the instrumented system, which has only a loose connection to the actual application.
In aviation, this is so critical that the instrumentation sometimes stays in the release build, so that exactly the tested code gets shipped. Other domains test once with and once without instrumentation and hope that both variants behave the same.
Embedded Trace: Dynamic Analysis without Disturbing the System
Embedded trace gives you a view into the running system without the application noticing. A hardware unit attached to the CPU broadcasts what the CPU is doing, without changing the application at all.
This non-intrusive observation avoids the core problems of instrumentation. No bloated code, no distorted timing. Alexander Weiss calls it the gold standard of observation, though with clear prerequisites.
“There’s hardware on the CPU that broadcasts what the CPU is doing, without the application noticing.”
(Alexander Weiss)
The most important prerequisite is physical. Almost every modern microcontroller has a trace interface, but it also has to be accessible on the board. In early hardware revisions, the trace signals are often picked up from various spots on the board, until a later redesign finally includes the right connector.
The practical advice: put it in the requirements specification early that the trace port must be provided. It isn’t needed in production, but it is needed for debugging and testing, and that is exactly where it will otherwise be missing.
How Trace Data Is Evaluated
Trace data can’t be evaluated in software in real time. It needs dedicated hardware. Even the fastest microcontroller produces more data than a pure software decoder could process live.
Several protocols are relevant for getting the data out. Parallel output over many pins is possible, but awkward, because gigabit bandwidths need a lot of pins. High-speed serial interfaces such as Aurora push out several gigabits per lane, optionally over multiple lanes.
A third option uses a system interface such as PCI Express. That is elegant and widely available, but it has a catch: as soon as the trace unit takes bandwidth on PCI Express, it becomes intrusive again. A dedicated serial trace interface remains the cleanest technical solution, but it isn’t always available.
The evaluation is done by an FPGA, a large, freely programmable chip. It takes in the individual trace fragments massively in parallel and decodes the protocol. For code coverage, it holds a separate hardware counter for each instruction address. After the measurement, the counters add up to how often each point in the control flow was executed.
The trace protocol is heavily compressed. It doesn’t output the actual jump address, only whether a direct branch was taken or not. Without the application binary, the data stream can’t be interpreted, which also eases security concerns. If you want to lock down the trace interface on top of that, you can disable it permanently with a fuse.
Code Coverage in Integration Testing Instead of Unit Testing
Embedded trace makes possible what long wasn’t: measuring code coverage on the real target during system and integration testing. Until now, instrumentation effectively limited coverage to unit testing.
The original idea behind coverage measurement comes from aviation, around 1992. Requirements and tests share a weakness: you never know exactly when they are complete. Code coverage, by contrast, is mathematics based on the source code. At 100 percent, the statement is unambiguous.
Coverage was meant as a tool for finding missing requirements and missing tests by checking coverage against both. At unit level, that didn’t work, because there are no requirements at that level. Measuring in integration testing closes exactly that gap.
At this level, you can in principle even measure during real operation: in theory during a test flight, in practice in the lab on a single control unit. The hardware tells the truth, and the code that actually ran becomes visible.
Martin sees the main benefit in functional safety. Formal unit testing in safety projects is very expensive today, with large test teams, and often adds little value.
Integration Tests First, Unit Tests Only with Justification
It would make sense to give integration testing priority and add unit tests only where coverage can’t be reached any other way. The justification should come before the additional unit test, not after.
Aviation standards such as DO-178C leave both routes open today. You can achieve the required coverage through integration tests or through unit tests. That freedom also reflected what was technically possible at the time.
The distinction matters: this is not an argument against unit testing, and not a return to big-bang integration. Everything a developer sensibly does in unit testing should stay, and it belongs back in the developer’s hands. The criticism is aimed at the excessive formalism that ties up a lot of money in safety.
Standards always reflect the state of the art at the time they were written. Software moves fast, so a standard can already lag behind when it comes out. For a technology like embedded trace to get its chance, standards need to leave room for new methods.


