Testing the customer experience with Eggplant

Building a good customer experience should be an important goal for any user-facing app (which is to say, the vast majority of mobile and web apps): better experiences mean happier customers, and happier customers are more likely to stay customers, make additional purchases, tell their friends about you (in a positive light), and so on. The reverse is also true: a bad experience creates frustrated, unhappy customers that won’t remain your customers for long (or, at best, will do so begrudgingly). This has pretty much always been the case, but it has become a particularly acute issue in recent years as the number of apps available has exploded – a lot of them hastily and shoddily built – and consequently customers are both more aware of what a good experience feels like and less tolerant of bad experiences when they happen. This is compounded by the fact that we, as a society, are increasingly used to immediacy (and increasingly intolerant of delays), thanks in large part to the convenience and availability offered by smartphones and the internet. In short, customer expectations are higher than ever, and aren’t going to go down any time soon.

In fact, when looked at in aggregate over your customer base, even seemingly miniscule negative experiences can have significant costs. Over 15 years ago now, Amazon published a rather infamous study, in which the company declared that every 100ms of latency cost them 1% in sales. This was not a one-off: similar results have surfaced in the years since, with the overwhelming message that poor performance loses customers. Moreover, this principle seems to apply basically regardless of industry – finance, banking, retail, telco, whatever, because all of these industries have users, and apps catering to those users.

If you take these ideas to heart, the natural next question to ask is how to create a good customer experience. This question is complex – certainly too complex to fully address here – but an undeniably good start is to ask whether the experiences you are providing are of a sufficient quality to satisfy your customers, and how any potential changes to your apps will affect the customer experience (either positively, negatively, or neutrally). In order to answer these questions, you will need customer experience testing, covering both functional (i.e., does it work right) and non-functional (i.e., does it work well) tests. Vice versa, there’s a compelling argument that ensuring a good customer experience should be the chief goal of your testing efforts, even if they might also accomplish other goals (such as accelerating time to value, or managing platform complexity) at the same time.

Eggplant, now a Keysight product having been acquired in 2020, is a holistic test automation platform that can combine its long-standing and well-regarded application testing with Keysight’s more hardware-based testing and measurement systems. Accordingly, it is one of the few (perhaps the only) testing platforms capable of truly end-to-end testing, starting with the physical hardware and ending with the logical software.  Moreover, Eggplant has long been a proponent for prioritising the customer experience, and it could even be said that its raison d’etre is to enable you to deliver truly exceptional experiences to your customers, accomplished in large part via its user experience testing.

The greatest strength of Eggplant on this front is its extensive use of AI to drive intelligent and automated model-based testing. The way Eggplant puts it, tests are executed “through the eyes of a user” as if by a “synthetic human” that will recognise and interact with elements of the system under test (images, buttons, text and so on) as a human would, with corresponding expectations in regards to the user experience. In other words, it behaves similarly to a real user. This provides as accurate an assessment as possible as to how the system will behave under contact with its actual users.

More specifically, Eggplant uses real-time image analysis to identify and track visual objects over time in order to determine how (and if) the user interface changes over the course of a test, using deep learning to detect and classify interface elements. In addition, user experience and other non-functional tests (such as performance and load) are layered on top of and embedded into functional tests to ensure they are performed at the same time, with the same frequency, and are not treated as the second-class citizens of the testing world.

And that’s really the crux of the issue: unlike many testing tools, Eggplant actively positions the customer experience front and centre, as the foremost goal of the testing process. It should therefore come as no surprise that, in terms of testing the customer experience, Eggplant is exceedingly hard to beat.