Artificial intelligence (AI) and software testing are two important topics in today’s software and system development. Using them together or on top of each other offers the opportunity for enormous synergies.
Although artificial intelligence has been the subject of research for decades, it has been on a very media-effective triumphal march in recent years. All tasks seem solvable, all human intelligence unnecessary, the possible consequences controllable. There are many convincing demonstrations. One example is AI-controlled computer players who beat the world’s best GO player Lee Sedol in the form of Alpha GO. Incidentally, GO is much more complex than chess and the game sequences are therefore much more difficult to predict. Other AI applications can accurately recognize the content of images. This opens up a wide range of applications, from the early diagnosis of dangerous diseases to the monitoring of public spaces. But is it that simple?
Hypes
Of course, the results presented seem very convincing. But is the path that the AI has chosen to achieve the result always really intuitive? Studies have shown, for example, that some images were not classified as horse images based on the actual horses depicted. Instead, they were classified based on the piece of forest also present in many horse pictures. Others based on the signature of the photographer (who often takes pictures of horses). In this way, the miracle of AI was disenchanted by some prematurely lauded examples. Memories of the horse “Kluger Hans” came to mind, who could only count for appearances.
Failures
In addition, failures such as the accident involving an UBER vehicle were also exploited by the media, so that autonomous vehicles were soon considered a danger. It is then easy to overlook the fact that in Brandenburg alone, an average of 2 to 3 people are killed in road accidents every week. A less than perfect AI could certainly offer advantages here. But there are other issues behind this. Accordingly, this technology is sometimes exaggeratedly praised and sometimes condemned before the connections are clear.
So I see an exaggeration here, both for better and for worse. Despite all the hype, AI has a lot of potential, even in safety-critical applications. The prerequisite for this is, of course, that this technology can be well secured.
A number of questions can be asked in this context. In the following, I will touch on a few of them on various sub-topics and thus provide an introduction. As I said, science has been dealing with more in-depth questions on this topic for decades.
Evaluation of the AI
First of all, statistics play a major role here and are used for the internal evaluation of situations, images, etc. The confusion matrix can be used to evaluate prediction and reality for binary classifiers: What is predicted correctly? Where and how is the AI wrong?
There are various means of evaluating these results. For example, the harmonic mean of accuracy and sensitivity, also known as the F1 score. In any case, it is clear that the significance attributed to the result varies depending on the domain. For example, the incorrect diagnosis of the existence of a tumor (which does not actually exist) is not so bad. However, the non-detection of a (actually existing) tumor is very important for the patient’s life expectancy.
Test-Know-how
In addition, the experienced tester naturally has another question: Which of the quality assurance tools he has known for many years can be used here?
- Do white-box test procedures even make sense or are they more like the still controversial tests to determine human intelligence?
- Does it make sense to divide the test into different test levels, as we know it from the V-model? For complex systems that hide one or more AI-based algorithms inside, this makes perfect sense. Does it also make sense for machine learning with a large number of intermediate layers? This leads in the direction of explainability of the implementation.
- What do we actually pay attention to during the test? Is it just a matter of the algorithm producing better results than its predecessor or do we subdivide things more precisely? Functional tests and non-functional tests? What about IT security? Even minimal changes to the design of traffic signs can have an impact: if an autonomous vehicle interprets the “30” on a km/h sign as “80” and wants to drive through the city at a corresponding speed. The effects of inconsistent situations such as the stop sign on the highway can be just as disastrous.
- The question also arises as to when the self-learning system is actually allowed to learn? Permanently in use? If so, a commuter’s self-driving vehicle could very soon be trained for the peculiarities of the daily route. The rest will be “forgotten”. Or should the AI only be allowed to learn during servicing or development? What are the restrictions depending on the application domain?
AI for the test
On the other hand, we testers are of course tempted by another idea, namely to use the unlimited possibilities of artificial intelligence for software testing itself. There are also interesting developments in this area. One area of application that stands out is performance testing. Here, the AI can detect anomalies in system behavior and system load depending on the input data. These observations could be used to bring the system closer and closer to the load limit or beyond.
Finding similarities and commonalities can be used in many other fields. For error messages, test specifications, test object log files, the generation of test data based on data format descriptions or test sequences based on code analysis. Another exciting topic is the use of an AI as a test oracle. This raises another question: can an AI that serves as a test oracle also be used as a system to be tested? And can it do it even better than the original? The question of limits also arises: what decisions can and should we leave to an AI? Some people are reminded of the trolley problem. This is already unsolvable for humans, or at least difficult to justify in most cases. If a fatal accident is unavoidable and you can still influence the outcome, who gets to live and who has to die?
These and other thoughts are an introduction to this highly interesting topic. It is economically very interesting and will have many exciting years ahead of it.
Frequently Asked Questions
Why don’t impressive demonstrations of AI systems prove that they can reliably recognize objects?
Because the path to the result is often different than expected. Studies have shown that images were classified as pictures of horses not because of the horses depicted, but because of a patch of forest in the background or the signature of the photographer, who frequently photographed horses. This is reminiscent of the horse “Kluger Hans,” who could only count for show.
Does a fatal accident involving an autonomous vehicle argue against the use of AI in safety-critical areas?
Not necessarily. The 2021 article pointed out that in Brandenburg alone, an average of two to three people died each week in traffic accidents. Even imperfect AI can offer advantages in such situations. The technology is alternately hyped and condemned before the full picture is clear. A prerequisite for its use remains that it can be adequately safeguarded.
Why is a single metric insufficient to assess the quality of an AI classifier?
Because the significance of an error depends on the domain. The confusion matrix allows predictions and reality to be compared for binary classifiers, while the F1 score represents the harmonic mean of accuracy and sensitivity. In medicine, diagnosing a tumor that does not exist is far less serious than overlooking one that actually exists.
Can the test levels of the V-model be applied to systems that incorporate AI components?
For complex systems that incorporate one or more AI-based algorithms, dividing them into test levels makes perfect sense. It remains unclear whether this also applies to machine learning with a large number of hidden layers. This question leads directly to the issue of the implementation’s explainability. It is also unclear whether white-box methods are effective here at all.
What role does IT security play in testing AI-supported systems?
A significant one, because even minimal manipulations of the environment can skew the results. If an autonomous vehicle interprets the “30” on a speed limit sign as “80,” it will drive through the city at a correspondingly high speed. Similarly critical are inconsistent situations, such as a stop sign on the highway. Security aspects are therefore just as important as functional testing and non-functional testing.
What risks arise when an AI continuously learns during operation?
The system may become overly specialized. A commuter’s self-driving vehicle would soon be trained on the specifics of their daily route, while forgetting everything else. One alternative is to limit learning to service or development. The appropriate level of restriction depends on the application domain.
How can artificial intelligence itself be used in software testing?
One obvious area is performance testing: Depending on the input data, the AI detects anomalies in system behavior and system load and can gradually push the system to its load limit and beyond. Other applications arise from identifying similarities—for example, in error messages, test specifications, log files—as well as in the generation of test data and test procedures.
Can AI serve as a test oracle?
The idea is appealing, but it raises an uncomfortable question: If an AI is good enough to determine the expected result, it could potentially serve as the system under test itself—perhaps even better than the original. This also raises the fundamental question of which decisions we are willing to leave to an AI in the first place.
