When classic testing tools fail on legacy interfaces, new approaches are needed. A practical case shows this clearly: Deutsche Bahn's mobile checkout required two weeks of manual acceptance testing because UI elements did not offer any IDs. Instead of selectors, a visual approach was used that combines screenshots, OCR, image recognition and language models. Test cases are created in TypeScript or as no-code flows, which shortens the distance between specialist testing and automation. Initial figures are promising: 60 out of 270 cases run in around three hours. Parallelization should make 210 plus possible in 60 to 90 minutes.
In this episode, I talk to Jonas Menesklou and Umar Usman Khan about AI-supported testing for legacy applications. Starting point: the mobile cash register of Deutsche Bahn. Manual acceptance testing took two weeks, standard tools failed due to missing element IDs. AskUI makes it visual: screenshot, OCR, image recognition and LLM instead of classic selectors. Tests are described in TypeScript or generated from test cases using no-code. Initial results: 60 out of 270 cases run in around three hours.
"You don't have to tell it what this login button looks like, but the model has learned beforehand that a login button looks like a login button and then finds the position where it is via the overall screenshot" - Umar Usman Khan, Jonas Menesklou
Umar Khan is Lead Quality Engineer at Deutsche Bahn Fernverkehr AG.He is responsible for quality assurance and test automation in various digitalization projects. Since 2018, he has been working on bringing more standards and automation to projects and also modernizing legacy applications with the help of AI tools.He finds modern quality assurance and test automation particularly fascinating. For him, the goal is a stable "green" pipeline after automatic test execution and automatic reporting of the results - and that's exactly what makes him happy.
Jonas Menesklou is co-founder and CEO of AskUI, an innovative tech startup specializing in AI-powered automation solutions. With a background in software engineering from the Karlsruhe Institute of Technology, his vision is to develop a new generation of automation solutions. AskUI is used by companies worldwide and employs people all over Germany.
Every large company has them: legacy systems. They are old, often run stably, but become increasingly difficult to maintain and test over the years. It becomes particularly critical when new requirements arise, such as automation. Deutsche Bahn is facing precisely this challenge. Its long-distance employees use mobile POS systems that were not developed in-house from the ground up. As Umar Usman Khan explains, it is an Android-based application with Windows and .NET 6. Element IDs are missing, standard tools such as Appium and Selenium are ineffective. Two to three weeks of manual testing with three people are the result - time that is lacking in fast-paced everyday life.
But giving up is not an option. The solution: AI-supported testing with AskUI. Jonas Menesklou describes how the approach works in a fundamentally different way to conventional automation: instead of searching for element IDs, a model analyzes screenshots of the operating system. A cloud inference with image recognition, OCR and LLMs recognizes buttons and fields purely visually - completely independently of the code underneath. The description in the test case is sufficient: "Click on the green login button at the top left." The AI finds the element as a human would. No markers, no recordings, no references. The AI understands the UI using natural language and visual features.
The team creates the test cases in TypeScript or other modern frameworks. The integration is flexible, as Umar Usman Khan and Jonas Menesklou report. Less tech-savvy users have the option of formulating test steps as CSVs or using a no-code tool. The model converts natural language, as stored in test management systems, directly into executable tests. Simply upload the test case, execute it and the AI does the rest.
Test cases that used to take two weeks now sometimes run in just a few hours. For example, 60 out of 270 complex acceptance tests have been fully automated and can be executed in around three hours instead of days. With full automation, the team aims to be able to cover over 200 test cases in just one and a half hours. The resource savings are enormous.
The new solution also comes with hurdles. Jonas Menesklou and Umar Usman Khan openly report on the current limits. The AI models have to be hosted specifically and image processing takes time, especially compared to classic tools such as Playwright. The startup is constantly adapting the models, bringing caching and compression into play to improve speed. Maintenance effort for UI changes remains, but is lower than with code-based solutions thanks to automatic image capture and training support. Weaknesses such as blurred screenshots can be rectified by manual retraining - the AI learns as it goes.
Another point: peripheral tests, such as the print function or credit card scanner, are not yet fully automated. But the path is clearly recognizable. Further test areas are being developed step by step.
The example of Deutsche Bahn shows: Even old systems can be tested efficiently and flexibly using innovative methods. AI enables completely new automation options where traditional approaches fail. Those who are willing to experiment with new tools and are open to partnership save time and resources and improve quality where it was previously difficult to achieve.
The journey is not over yet, but the initial successes speak for themselves. And as Richie says at the end of the podcast episode: "Next year, we'll see how things have progressed. The journey remains exciting.