“There’s budget again. It just has to say AI on the label.”
(Richard Seidl)
Test data management is having a revival. Driven by AI, of course, and everything people hope to get from it: better, more precise test data, easy generation, effortless management across system boundaries. The ideas are bubbling over about what will be possible … or would be … or, well, maybe … hm.
Before AI can help with test data, you need a radical cure for your data and a data architecture that fits.
The Challenges of Test Data Management
Software development has a handful of classic headaches. We have more or less solved some of them, such as releases (pipelines) and test environments (cloud). Test data is a different story. Especially in application landscapes with many different systems and data stores, test data initiatives tend to spiral out of control fast. No surprise, because the challenges come from every direction:
- Data, data, data: there is simply a lot of it. Countless tables and fields, plus thousands, sometimes millions, of records. All of them linked every which way, with some import/export files sitting in directories on top.
- Compatibility: every system has its own schema and its own way of organizing data, and they don’t necessarily fit together. A clean data architecture across system boundaries is hard. Different ownership adds another layer of complexity.
- Synthetic vs. real test data: synthetic test data helps me enormously with my structured test cases (boundary values, equivalence partitions, etc.), but it simply isn’t reality.
- And as soon as you move to real data with all its quirks, errors, and so on, data protection is knocking at the door: anonymization and pseudonymization eat up a lot of energy and time again.
- Want to top it off? Then your test data also has to cover historization and time travel. Yeah. Jackpot.
AI Will Sort Out Our Test Data, Right?
But hey, no problem at all. Just throw every rule, requirement, and so on into an AI, and then we generate our test data across all systems, almost in real time. A dream. Except I’m pretty sure it won’t be that easy. There are some nice approaches out there for generating and managing test data. What I observe, though, is that a lot of it treats symptoms. I’d rather put two other questions on the table.
First the Radical Cure, Then the Data Architecture
What data do I really need? (And of that: which do I really, really need?) Just because we can store everything doesn’t mean we have to. Adding a field to a table is so easy. The effects can be dramatic. So: just leave things out and delete every structure nobody needs. A radical cure for (test) data!
Do I have a data architecture that fits? In cross-system architectures I see lots of interfaces and dependencies, but hardly ever an overall picture of the data stores, the data flows, and where which data belongs, so that nothing gets stored redundantly or in circles. That’s when it starts to make sense.
And once the data is in halfway decent shape, then I’ll think about doing something with AI 😉
