Autonomous vs Automation testing

To perform any kind of effective testing activity it is required to have testing context awareness, healthy running environments, good sample data and a clear understanding of the business rules. The aim for a good RoI on testing is then to achieve Autonomous testing instead of Automation testing.

In general, automation testing is wildly and wrongly misunderstood as an evolution of manual testing. The confusion comes from trying to replace the mechanical activity of a test engineer for some technology that can emulate human actions. The set of actions performed by a test engineer are just the topmost thin layer of required tasks to do effective testing. Therefore, testing has not much to do with automation.

“We become what we behold. We shape our tools and then our tools shape us."

Automation is a Technique, Not Testing by JEFF NYMAN

The situation is especially difficult when the solution under test is legacy and not designed to be testable. Under such kind of solutions, the test engineer has to compensate for all the technical debt, hacks, bad code, lack of knowledge of technical and business rules, etc…. All this work is not automatable, at least not until we have good Artificial Intelligence A.I specialised in testing. All those tasks require thinking, analysing, a huge dose of patience and social intelligence instead of A.I.

Therefore the actual question is “How the H.. do we automate in the real world of software development?” I mean, actual testing, not just running scripts. Actual and useful testing activity that can quickly feedback development teams of actual bugs, design flaws, architecture deviations, broken API consumer contracts, and more.

The answer proposed is simple. We need autonomous testing instead of automation testing. We need testing technology to help test engineer do their validation and verification job faster and easier. The test engineers need more autonomy to perform their work. Test cases need to be more autonomous. This means that the test environments test data, test scripts and logging reports should be well packaged to be executed at any given time. All knowledge about how to prepare a test needs to be collected and prepared with a set of top quality tools. Hence, the work of a Software Engineer in Test is to implement all those tools and technologies to support the work of a test engineer and the development team by extension.

In summary, our goal in the testing industry is to provide more and better autonomous testing technology instead of trying to do more and more automated scripts. The QA coverage needs to be defined in terms of what level of autonomy do we have for our test cases instead of how many test cases do we have.

In the same way that we, as an industry, understood the benefits of DEV-OPS, we should aim for TEST-OPS, which means testing operations technology. When a test engineer is asked to validate or verify some bug, the perfect answer should be:

“We have a test for this situation. Here it is, run it and get your feedback.”

This means that the duty of a test engineer is no longer to execute tests cases. This role should evolve into a full-time job of preparing autonomous test cases. As a result, we have a better and more efficient layer of test that the stakeholders or the development team are able to execute faster to answer the right questions at the right time.

Posted by Marc Andreu

I hope it helps,