The situation
A recent enterprise client has a large portfolio supporting over 2 million customers. To maximise business growth, they needed a single, consolidated view of their products and services across the organisation.
To achieve this, Enterprise Architecture embarked on an ambitious transformation of their mobile application, centering around a new application design, a complete re-architecture of the backend platforms, and a reimagining of how data is managed across all systems.
This initiative required a coordinated approach to realise the benefits of the transformation throughout the whole enterprise.
The challenge
The many different portfolio departments operated in isolation, each having their own culture, accountability, and processes. This fragmented way of working led to a lack of strategic alignment, resulting in crippling performance bottlenecks being exposed too late in the development process.
Performance testing activities were handled by a separate team operating outside the project’s scope. This exacerbated the performance issues by creating lengthy delays, inconsistent results, and quality concerns.
We were engaged to help fix the performance problems through design, implementation, and execution of a modern performance testing framework. They worked with the delivery teams to uplift their capabilities, developing a culture of ownership for their performance testing activities.
The process
We engaged business stakeholders in conversations about customer journeys. By setting target baselines for each journey, and all underlying platforms, we were able to determine the metrics for success.
Leaning on Gherkin syntax (Given-When-Then) to describe the performance test parameters, we built a modern automation framework for continuous performance testing against these baselines. This framework enabled the organisation to continuously validate performance, giving delivery teams the capability to drive better customer experiences through continuous improvement.
The framework enabled rapid identification of where bottlenecks were occurring, allowing for issues to be stack ranked to ensure effort was concentrated on the biggest customer impacts.
The results
- Increased team capabilities – Teams are able to take ownership of their performance testing needs and drive better decision-making based on accurate data.
- Reduced performance testing cycle from weeks down to days – Issues are identified early in the development process, making them cheaper and easier to address.
- Test creation reduced by 85% – Simplicity in design and execution enabled creation of new performance tests to be done in an hour, rather than a day.
- Satisfied customers – by focussing on the customer experience and driving a cultural shift towards team accountability, customer complaints dramatically reduced, and more user activity observed on the platform.
Final Thoughts
We introduced customer journeys into conversations with Enterprise Architecture and business stakeholders, and together we were able to define what success looked like through target baselines for each journey and all underlying platforms.
Building an automated framework to continually test against these baselines gave the organisation a vehicle to rapidly validate performance improvements, and drive an even greater customer experience through continuous improvement.
This innovative approach gave the business an accurate representation of how long customers were waiting for their platform to respond across their whole experience. Teams could then prioritise on greatest customer impact, leading to a new app that delivered a pleasant customer experience, easing stakeholder concerns.
Through visualisations and automation, the business now has an automated performance test framework to measure customer impact, make data-driven calls, and rapidly validate new features and ideas.
