As Zuora continues to Reinvent, implementing new backend architectures that allow us to move from a monolith to microservices, we’re also introducing changes that revamp our product’s UI. This is outlined in Wei Liu’s recent blog post.
Our Platform UI Team is actively working towards enhancing the user experience (UX), and have been rolling out granular changes including pixel adjustments, new icons, and word modifications that can introduce subtle or sometimes significant shifts in UI layout, icon disparity, and typography.
Due to the expansiveness of the changes being rolled out, verifying the correctness of these changes can not be covered by manual testing alone; verification needs to be augmented with UI automation that has the ability to refer to a source of truth every time there are UI changes. The goal being to decrease the amount of manual testing hours after each deployment and, ultimately, the need for manual testing, with the hopes of catching UI regressions more quickly and effectively.
Quick Intro to Ghost Inspector
Ghost Inspector is an automation testing tool designed to cover full front-end UI testing. It comes with a variety of useful features that help QA Engineers cover almost every use case, including:
Test run scheduling
Multi-browser support for test suites
Multi-geo support for tests
Chrome plugin to quickly record new test steps
AWS Console integration
Test export capabilities: Selenium JSON or HTML formats
Ability to add assertions while recording test steps
How we use Ghost Inspector
There are a few Zuora product teams that use Ghost Inspector today:
Zuora’s Analytics releases are very heavy in new features and UI enhancements. This makes it difficult to keep up with every change that is introduced - be it via new code or library upgrades. Ghost Inspector’s pixel-by-pixel comparison is extremely useful in supporting large feature deltas between releases. Also, daily scheduled test runs drop the team a message via slack/email whenever a test case fails.
Figure 1 - Zuora Insights UI
Zuora for Salesforce
The Zuora for Salesforce team is currently evaluating Ghost Inspector as well. The primary use cases are to test the interaction between Salesforce and Zuora. In particular, Ghost Inspector enables the team to do two things:
Bridge the gap that exists between our current manual tests and our existing automated tests
Quickly spin up new modular test cases. This enables the team to leverage the full breadth of features that Ghost Inspector provides
Here is what test steps look like in Ghost Inspector:
Figure 2 - Ghost Inspector Test Cases
Note that attributes (e.g., #id_username) can be used to locate elements in the product UI.
How Ghost Inspector Stacks Up
Ghost Inspector has been compared to a standard Selenium and webdriver approach. After comparing Ghost Inspector to a standard Selenium and webdriver approach, we discovered that Ghost Inspector surpasses Selenium and can benefit developers in ease of use, test readability, and debugging capability.
Based on our experience, we’ve identified the following pros and cons of using Ghost Inspector:
Good variety of UI actions such as drag and drop, mouse over, keypress etc.
Unable to loop through multiple UI elements
Ability to select timeout for each test
Recording doesn’t always recognize elements
Modularize test steps
XPaths are not optimized
Make assertions while recording
Relies more on XPath rather than CSS
Set global variables
Tests can fail due to selectors not having a unique ID or attribute identifier, as well as the recorder recording unreliable XPaths
Low monthly cost approx. $500/month
Use spreadsheet to input test cases
Easy to ramp up on software
Flexibility to run test suite in sandbox, production, and staging environments
No need to download code locally to be up and running on suite
The Analytics team will use Ghost Inspector more broadly while enhancing the test cases to reduce brittleness and increase maintainability.
The Zuora for Salesforce team is still trialling Ghost Inspector and will make a decision about how Ghost Inspector could fit into their long-term QA strategy.