AI and ML in Testing Automation
AI and ML in Testing Automation
AI, even though not that new either as a concept or as a technology, has gained a lot of steam over the last few years. Accordingly, there is a race to put the “AI” and “ML” tag on anything and everything.
Yours truly is no different as such. It is just that being a bit old-school, I tend to ask: “What business problem does it solve for me or my organization?”
It seems that Enterprise Search, a Universe in itself, is one such area where ML has a not-so-straight forward applicability.Unlike Google or Facebook, a search solution within the Enterprise or even within customers of an Enterprise doesn’t have the luxury of a million clicks in an hour, not even a million clicks in a day. Therefore, the challenge becomes how to get good data upon which your model could be trained.
The solution has to be as unique as the problem. A comprehensive Enterprise Search solution has to be a medley of which ML is one part. A bit of ML: supervised, possibly unsupervised as well, simple logistic regression, NLP among other things, each of which can be added or proportioned depending upon requirements.
I park this topic of ML on Enterprise Search for some other day. It’s been brought in only for comparison purposes. Today, I want to talk about something I feel more strongly about regarding the Applicability of ML: Test Automation.
Consider this scenario: A conventional software needs to be tested in a CI/CD environment where code is changing quite frequently. Here is the classic “precision testing” challenge: “Which test cases do I need to run for a specific code change?” Often, we run the entire Selenium test suit and that is a problem because it is cumbersome, redundant, time consuming etc.
My theory: With AI/ML Automation Testing, the fixed boundary of limited input and results that come with a typical Software Program is a HUGE advantage.
Due to the fact that our inputs and results are bounded in a more well-defined manner than say, a “Search” requirement, a ML model that can cater to precision testing requirements in a way that a conventional Selenium based test suite cannot provide. ML is likely to bear faster and more precise results in this case than in a “Enterprise Search” solution.
Through canary testing, a small set of users can test the new code like they would in the real world. ML can compare the results from these users with those from running the current production application. As a next step, it is not too difficult to determine the discrepancies between the observations from the two set of users.
In another scenario, ML Crawler Bots can test any given UI application and greatly reduce, if not entirely eliminate the need of Record and Play tools.
ML test cases are self generated and self-maintained since they are based upon self-learning and discovering. This is in sharp contrast to Selenium based automation test cases which break easily if say an input parameter name is changed.
AI, specifically supervised Machine Learning seems like a very promising candidate as the next generation automation in Software Testing.
IMHO, those who adapt to ML in Testing Automation, shall reap long term benefits.