My journey to agility
After years of experience as a Business Analyst and Project Manager working on large-scale waterfall projects, discovering agile methodologies and principles was quite an eye-opener for me. In fact, I almost felt as though I had stumbled across a whole new world which was just waiting to be unearthed.
After researching Lean, Scrum, XP and Kanban, I got to thinking – why don’t more traditional projects adopt some of the agile techniques in order to improve the way projects are delivered? In this post, I’ll explore some of the things I would now do differently in a waterfall project, since being exposed to the world of agile.
I learned that the definition of insanity is doing the same thing and expecting a different result
There have been numerous times when I felt I had lost faith in big projects. Times when the projects were so immense that team members didn’t understand how their daily tasks contributed to the overall solution.
Times when all our functional requirements had to be completed prior to development and testing, but there were so many requirements that priorities got lost. Times when stakeholders would provide feedback only to have Solution Architects disagree. The list goes on.
All too often these traditional projects encountered the same challenges and fell victim to the same mistakes, without enough being done to address the underlying issues. Why didn’t these projects embrace alternative methods?
I believe that knowledge is a powerful thing
I’ve found myself asking this question a lot since learning about ‘new’ agile techniques. I expect it’s quite difficult for some organisations to adopt agile due to the significant internal culture-shift required, but in retrospect, I think perhaps it’s not just that – the Business Analysts on my projects simply didn’t know any better. So, let’s take a look at what we could have improved.
Towards agility: Smarter release planning
In my previous projects, project scope was defined by Senior Management and the Project Manager, with assistance from Subject Matter Experts. The project schedule was set and the remainder of the project was then managed using timelines working towards a ‘big bang’ release.
It’s called a ‘big bang’ because after months (or even years) of project work, everything is released in one hit and all users of the old system are migrated to the new one. The risk with this approach of course, is that the demonstration of working software occurs late in the project and the customer may not end up with the outcome they were expecting.
Seeing the agile manifesto in action changed my view of the world
The agile manifesto states that: “The highest priority is to satisfy the customer through early and continuous delivery of valuable software.” But even if your project culture doesn’t support continuous delivery, there are still some things you can do.
For example, functionality can be delivered in smaller, more manageable batches which can then be promoted more frequently into production-like environments. After all, it makes sense to deconstruct large risks into a series of smaller ones.
This approach was something we tested in order to push my last project in the right direction. We sliced the release into ‘drops’ (as opposed to ‘sprints’ used in Scrum), and named them as such because they were so heavy with code, it felt like we were dropping things every time we moved it around.
The split-releases mitigated the risks of the ‘big bang’ approach, and features were prioritised and assigned to a drop, with the aim to deliver meaningful (and working) features into a production-like environment.
Although the drops went for longer and contained many more features than you would find in a typical agile sprint, the project was able to successfully demonstrate progress via working software, instead of deliverable sign-off percentage.
So what would I do differently next time
So, what would I do differently if I had another go? Certainly I would define the Minimum Viable Product (MVP) to focus planning and delivery on the core features, as not all features had the same priority and not all delivered the same client value.
The functionality was split across drops like Lego blocks just because it fit, and release planning was more like matching implementation size with resource capacity, rather than focusing on matching delivery order with client priorities.
So despite the release being split into drops, it still felt as though resource capacity was the key driver of the release plan.
I now realise how inefficient it was to deliver on the first drop, only to have it sit idle for a year waiting for the ‘big bang’ release into production. An MVP would have focused on delivering the core feature set into production, followed by the lower priority items.
Towards agility: Reflect and improve
Segregating this project also created an opportunity to reflect at the end of each drop and cite improvements for the next one, as post-implementation reviews (similar to retrospectives) were conducted iteratively rather than as a one-off at the end of the project.
This approach promoted a system of continuous improvement; allowing stakeholders and project team members to celebrate milestones and draw attention to areas in need of improvement.
This opportunity, however, was limited to one per drop, whereas delivering in even smaller batches would allow for more frequent post-implementation reviews and provide further incentive to optimise processes.
Towards agility: Boost elicitation techniques
Having lead large groups of Business Analysts, Functional Analysts and Systems Analysts in the past I have observed that elicitation is much easier said than done. Many analysts struggle to effectively facilitate requirements workshops and end up firing close-ended questions to unengaged stakeholders while jotting down notes in their personal notebooks.
In fact, 60 – 80% of project failures can be attributed directly to poor requirements gathering, analysis and management.
The Business Analysis Body of Knowledge (BABOK) identifies brainstorming, document analysis, focus groups, interviews and surveys, prototyping and requirements workshops as some of the key requirements elicitation techniques. These elicitation techniques are fairly standard across all project methodologies, but there is a whole world of techniques to explore.
Broadening my horizons became my mantra
In my last project Kanban walls were non-existent, and story cards and sharpies didn’t even exist in the stationery cabinet! Reluctance to change (and heavy emphasis on documentation) could possibly be reasons why Business Analysts in traditional waterfall projects don’t investigate new elicitation techniques, but as I say to my anti-pescatarian friends: “Just because you haven’t tried seafood, doesn’t mean it’s no good.”
How many times have you left a meeting feeling positive and accomplished, only to receive an email from a key stakeholder stating: “I did not agree to that!”. I’ve found that using story cards in requirements elicitation workshops can be a particularly effective method for alleviating uncertainty between meeting attendees, especially when requirements are ambiguous.
The requirements are developed as a team and everyone agrees on the decisions as a team. It is a great way to engage your stakeholders, to visualise the requirements and to capture decisions that are being made, as cards can be grouped, sorted and reprioritised without the need to erase everything on the whiteboard.
I learned there was no going back to traditional project management
All meeting attendees will also feel more involved and can immediately see the results of the discussion, minimising the risk of misinterpretation and eliminating the need of a minute-taker, who, potentially may not capture all the relevant details. Since being exposed to these techniques, I simply cannot go back to my old elicitation ways.
Towards agility: Remove inefficiency and waste
With waterfall projects being constrained by timelines and tight schedules, every minute of inefficiency contributes to a delay of the overall project. How often have you heard: “I didn’t know what the critical date was” or “the dates and priorities weren’t clear.”
Project Managers and Lead Business Analysts often resort to over-engineering tracking spreadsheets as a means to assign, track and report on tasks. The maintenance of these spreadsheets—along with defect tracking tools, change management repositories and hours spent on risk/issue logs—eats into time that could otherwise have been spent delivering value to the customer.
I do not miss using spreadsheets and email to track progress and assign work
The first of the seven lean software development principles is ‘Eliminate Waste.’ In a project that I worked on previously, tasks were tracked using gigantic spreadsheets and communication with the team was largely via email.
These methods resulted in delays to team members commencing tasks, and also in the misinterpretation of deadlines and priorities. We wasted precious time in ‘waiting’ as team members missed tasks that were assigned to them and experienced issues with ‘transportation’ (or task-switching) as team members moved onto new tasks without completing their first assignment.
Kanban has changed my life!
Applying agile methodology techniques such as a Kanban board to track work instead of a spreadsheet, and introducing daily stand-up meetings with the team could have been hugely beneficial in this situation where waste had become a real issue. The Kanban board tracks the backlog of tasks/stories, and teams manage them daily from the start of the process until completion.
Utilising a Kanban board improves visibility and builds trust within the team, while eliminating wasteful occurrences. I now know that introducing a Kanban board would have improved efficiency, promoted collaboration and minimised delays. In hindsight, I have to wonder why I didn’t employ this process on my last project!
Towards agility: Embrace change (just a bit more)
We tend to assume that the benefit of correcting a deviation from the plan will exceed the cost of doing so – which places completely unwarranted trust in the original plan. Traditional projects tend to take several months, or even years, to go live and requirements are more than likely to change during that time.
My hamstrings have never been in better shape
In my experience, waterfall projects often make the mistake of focusing so strictly on timelines, schedule, budget and resourcing, that they neglect to prioritise the delivery of customer value. The rationale for certain requirements in the original plan may change over time and not end up delivering as much value as first thought.
And though it would be detrimental to incorporate every changing requirement, my point here is that as customer needs evolve over time, waterfall projects need to adapt and embrace the idea that going live with a relevant and profitable product is far more effective than sticking to an outdated original plan.
Wrapping it up… Downward dog!
In conclusion, while it could be argued that forcing hybrid solutions into traditional environments or making sudden changes to internal culture in order to completely embrace agile has the potential to be counterproductive, I see no reason as to why we shouldn’t incorporate some of the more effective agile techniques and principles to improve the way we ‘do waterfall.’
Employing just a few simple agile practices allows us to mitigate the risks of the ‘big bang’ release, opens up new elicitation techniques, minimises wasteful project activities and places an emphasis on embracing change as part of organisational culture – all innovative and proactive methods of ensuring a successful outcome for the customer.
Further reading:
- Agile Manifesto
- The Business Analyst As Strategist
- International Institute of Business Analysis 2009, A Guide to the Business Analyst’s Body of Knowledge® (BABOK® Guide), Version 2.0, Toronto, Ontario, Canada.
- Reinertsen, Donald G. 2009, The Principles of Product Development Flow, Second Generation Lean Product Development, Celeritas Publishing, Redondo Beach, California, USA.
