Our automation team recently had the experience of working with one of Australia’s fastest growing FinTech companies.  

All of these factors were important especially when launching something new into the marketplace.

The ‘Agile’ delivery approach was a given however certain aspects needed a more traditional systematic approach to ensure the delivery road map was not compromised.  Fortunately, this included a rigorous QA and automation process while ensuring things were documented along the way.  This highlighted the companies mature approach to making sure things were right and ready for launch.

The automation team started with the development delivery ensuring that when software was checked in then it was automatically tested using the framework and tools we put in place.  This allowed developers to have almost instant feedback on their build where they could either rectify issues identified or if it passed then leave it as part of the main code base.  This alone had significant time savings and took the reliance off waiting for manual QA to occur, especially when time was not on their side!

Once the development automation was completed aspects could be extended to then be used for the formal integration testing, regression and audit testing which allowed the company to benefit from reuse and reduce the cost of automation.  This was then further used for acceptance testing, environment data population, health checks and getting the results from the front-end input and comparing the calculations were correct.

 

 

The technical delivery included:

• Analysis of the technical and business environment to determine the best automation approach
• Implementation of the automation framework including detailed documentation of the architecture
• Ensure the automation would be compatible with Salesforce and their technical architecture of the platform
• Creation of the automation including detailed documentation to allow for client-side team members to run and update it
• Running and recording the results from the automation
• Automated logging of defects, results as well as automated reports sent out to key stakeholders

 

The types of technology and tools sets we used in the automation were:

• Selenium WebdriverIO  
• AWS
• Node.js
• NPM  
• Babel  
• JavaScript  
• Jasmine  
• Allure reporting – as per WDIO
• JSforce – Salesforce API library
• Axios  
• Allure CLI  
• Twilio API  

What were the benefits?

Although there is always an upfront investment, the team did not have to wait for weeks until they could use the automation.  Once the tools were in place then the working automation was released daily so it could run and used by the teams.  This aligned to a continuous integration model, speeding up the feedback on quality for the development and business teams while minimising manual effort. 

The summary of manual versus automated effort below outlines the true benefits for the team.  This was not about simply reducing costs and cutting down staff, it was about time to market for a product release, getting ahead of the competition and doing it in a smart and effective way.  It also allowed key subject matter business and technical experts to focus on their primary roles within the organisation rather than constantly being put on less complex QA activity.  Their involvement in the QA process could be more focused and tailored around scope, validating a complex business rules and doing a final manual cross check of results from key transactions.

With the upfront investment it allowed for it to be reused and built upon for future releases and product enhancements meaning it was not thrown away or shelved at the end which unfortunately is where some automation can end up if not put in place correctly.

Summary of core tests and timings:

	Total tests including variations of a test	Manual Efforts – 1 person	Automated regression turnaround time
Functional Automation	479	10 – 12 Days	30 mins based on the apps responsive time
Business process automation  for external Auditing	This testing included forward dating the calculations engine and system processing for up to 25 years as well as validation of logs and calculations.  Data snapshots for audit purposes were also taken.	35 – 40 Days	22 – 24 Hrs

How often were the tests used?

Environment	Frequency
DEV	15 – 20 times per day. Essentially on every commit from each developer.
User Acceptance Test	4 – 5 times a week

As with any automation, it is never going to solve 100% of the manual challenge but if put in place correctly, gradually built up and used often, a company will wonder why things were done manually for so long.