Flomatika is a SaaS platform that originated from within Cprime Elabor8’s need to provide reporting on agile delivery metrics to customers. We used AWS to test and learn how to bring this product to market and created a scalable, secure, and extensible platform for Flomatika. AWS enabled us to gradually evolve this platform as Flomatika scaled to address the customer needs and the market demand.
Help our customers get better insights on realising roadmaps
At Cprime Elabor8, we have a passion for helping to solve our customers’ problems. As an Agile Transformation Consultancy, many of our customers had challenges with getting actionable insights on how they were delivering against product roadmaps. Flomatika was born out of the need for addressing this.
Flomatika is a Value Stream Management platform, focused on helping mid-market and enterprise customers pursue faster and more predictable value delivery to realise their roadmaps. Providing near real-time, end-to-end visibility into product delivery flow, Flomatika helps our customers to understand how the delivery ecosystem is performing to effectively govern and steer at product/project, program and portfolio levels.
Flomatika provides actionable insights and surfaces hidden constraints which hinder the ability to improve speed, increase quality and optimise value. This level of clarity and insight is a powerful catalyst for change, giving you the power to intervene with confidence and course-correct where required.
Prove it is worth doing and take the next steps to scale, without breaking the bank
When we got going, it all started with a collection of spreadsheets for building reports for our customers on Agile delivery for understanding flow metrics. We kept creating this same set of reports from scratch over and over again on multiple engagements, which our customers loved. We always looked for a series of inputs on where to get data, building in a layer to aggregate and interpret data in a way that is relevant and meaningful to the customer. This pattern was codified which Cprime Elabor8 decided to automate by building a SaaS product to measure flow called Flomatika, that was spun off as a separate company that became a client of Cprime Elabor8 for the tech build.
Our first step was to build some low fidelity prototypes to test the idea with clients, which received an emphatic “Yes, please” response from all new potential customers. So, with this emphatic positive response, we skipped the next step of high-fidelity prototypes and our next step in the process was to figure out how to build a SaaS platform. We did what everyone else did, we started building. Our first prototype was built fast and provided us with a way to validate the idea with the market. We shared it with potential customers as a pilot to get input into how they liked the user interface, how the existing functionality helped, and what else they were looking for. It gave us a lot of insight into additional features we had to build.
However, going from the initial prototype to a minimum viable product was a challenge. While we used some quick and dirty methods to build the initial prototype, for the minimum viable product we wanted a future-proof underlying platform to add additional features with limited refactoring. We wanted to architect it better to enable us to evolve easily, which led us down a microservices architecture path. The next step of the challenge was picking the right AWS technology to fulfil the desired functionality to match performance characteristics.
Creating an evolutionary foundation, identifying performance characteristics, and matching to suitable technology
Our original prototype was built using React (hosted on S3) for the front-end, AWS Lambda for the API implementation, DynamoDB for the database with SNS for notifications. For the minimum viable product (MVP), we identified and created the core components of the platform as microservices. For this, we introduced CloudFront, Aurora (Postgres), VPC, SQS, and Redshift as part of the MVP platform.
We had some user-facing components which needed fast user responses for updating and storing unstructured data. We had some components which had more structured data storage requirements for detailed metrics, for real-time analytical processing to aggregate and serve as a single source of truth. So, one of the trade-offs we had to make was between DynamoDB vs Aurora (Postgres) which we had to pick for each component.
We introduced CloudFront as the Content Delivery Network (CDN) for fast caching to enable high-speed, low-latency performance, security, and developer ease-of-use. We introduced API Gateway making it easy for developers to create, publish, maintain, monitor, and secure APIs at scale. This also gave us the ability to de-couple the implementation of the API from the interface for the API facade. So, when we had a new version of the implementation, we simply had to re-point and use a new version of the API.
When we wanted to secure the cloud components of the SaaS platform from the interface modules we had to build for integrating with each customer’s systems, we introduced Virtual Private Cloud (VPC) to create a partitioned network. This gave our customers the confidence that their internal systems were accessed securely. When we wanted to introduce distributed queuing between the microservices components, we introduced Simple Queue Service (SQS). This enabled the SaaS platform to scale without bottlenecks for message handling and processing.
When we wanted to aggregate data across multiple microservices, to get end-to-end visibility of processes, we introduced Redshift as the data warehouse for the SaaS platform. This allowed us to keep the microservices simple and build more complex aggregation and analytics into the data warehouse.
We created a continuous integration and continuous deployment (CI/CD) framework, which included multiple solutions for different parts of the platform.
- Terraform was used for Infrastructure as Code (IaC) to bring up the infrastructure, separate from the build and deployment pipelines. It provisioned Databases, VPC, DNS, and S3 buckets for each new environment.
- For the front-end, we used Gitlab CI/CD pipelines to build and deploy the website continuously. Our front-end was React-based and we used SonarQube to test for code quality and code security.
- The serverless framework enabled us to build and deploy back-end as linearly scalable microservices and create runtime infrastructure for API Gateway, Lambda functions, SNS, and SQS. This allowed us to deploy these components together whenever we had to do a change to this back-end.
- For the monitoring, alerts, debugging, and logging, we set up CloudWatch manually.
How this helped Flomatika to move with confidence, scale for the demand, and experiment on new features
Incremental spend
By taking this approach, not only was Flomatika able to build the platform incrementally, but it was also able to create a scalable platform that was future-proof by allowing it to evolve easily. They were able to prioritise components of the platform that were urgently needed for new users they were planning to onboard as soon as there was a demand for them. So, Flomatika knew exactly what to prioritise on for the build, when the customers willing to pay for these features were in the sales pipeline.
Cost-effective
With the serverless architecture-based microservices, Flomatika was able to scale as the demand increased by using AWS Lambda. Due to their serverless nature, this allowed them to have fine-grained and linear scalability of cost, to match the revenue growth. This is in stark contrast to going down a container-based architecture path, which would have a coarse-grained scalability model.
Secure and robust
The system was fast and performant because we had enabled caching through the CDN and picked the correct type of data store to match the functionality. We built the customer’s system integration modules in a self-serviceable way for customers to do it themselves, giving them the confidence they were provisioned inside a secure private network.
Developer friendly delivery pipeline
The delivery pipeline enabled Flomatika to have a cost-effective deployment with great visibility into the deployment process. This pipeline was also extensible, allowing them to add a mobile front-end with a few simple changes to the Gitlab CI/CD pipelines. Most importantly, the collaboration between developers with pull requests in Gitlab, review comments, and approving pull requests for integration allowed the teams to work from anywhere, but still be in touch with how the functionality on the platform was evolving.
