Quo vadis

Michael Mlynarski, how will software testing change through AI?

July 2023

Michael is a managing director who likes to advise in projects himself. His heart beats especially for agile software development and testing. He founded QualityMinds and some product startups. In recent years he has worked with a strong focus on machine learning and autonomous driving.

Information about the person as of summer 2023.

Where do we stand today with testing using AI?

Relatively at the beginning. Even in software development itself we are not far along yet. With GitHub Copilot, some approaches to predictive maintenance of cloud applications and the like, we have only just “started running”. Many vendors claim to be “AI-based” (especially in the test automation community), but we are still far from the big leap. For the few solutions on the market, we also cannot really answer the question “Who tests, and how, the AI with whose help we test?”. Even though I have presented it relatively negatively, the direction is the right one! Perhaps ChatGPT (also not a novelty, but it has still moved the market) will bring more innovation and speed to the market for AI-based testing.

How will AI change software testing in the next five years?

In my opinion, AI is one of the few technology trends that carries the potential for rapid change (to avoid using the word digitalization). I also do not believe that we will now experience a big “AI winter”, but at most a slight slowdown.

What will AI bring us in testing over the next five years? I hope for significant improvement of approaches to test automation (instead of manually implementing hundreds of test cases). Many of the heuristics and patterns in test design could be learned in order to do exploratory testing more efficiently. In test data management (including generation) we also urgently need AI support, since things will fundamentally move more and more toward (test) data. Last but not least, we can benefit greatly from planning algorithms in test planning, especially if we correlated the usual artifacts such as test logs, requirements, bug tickets and code commit logs with each other.

Which skills do testers need for that?

A difficult topic, which we discussed in an episode of the QualityHeroes podcast. In theory we would have to distinguish between skills needed to a) test AI-based systems and b) skills for AI-based testing. For the first, it is necessary to have a much better understanding of data structures and of how machine learning algorithms work (for example linear regression, neural networks, deep learning, etc.). The “shift left” principle gains even more importance here, since data analysis before training neural networks saves a lot of time and resources and thus money. Operating AI-based test tools, in turn, requires not only skills in using and parameterizing such tools but also in knowing the possible limits of the approaches. For example, when assessing test coverage for AI-based test automation, it is worthwhile to use supervised learning approaches to test more efficiently instead of generating and executing tests for everything. That requires test experience, test methodology and knowledge of how the AI works.

Will the role of testing still exist in 10 years?

That is a very generic question. I believe everyone who needs testing in the form we know today, for whatever reasons, will continue to do so. Ten years ago (and in the decade before that) we thought that thanks to test automation we would no longer need manual test execution. Ten years ago many thought cross-functional teams would make tester roles unnecessary, and so on. I very much hope that in ten years we will have less and less “dumb testing” (mindless execution of the same test cases that should have been automated long ago) and will instead use real creativity, closeness to development and business, and AI-based testing approaches. Then QA/testing is fun again and can often do more and faster than is currently the case in many projects.

More posts

View all →