Rally – Agile Tool Review

What is Rally?

Recently we kicked off our series of Agile tool reviews with JIRA, and now it is time for the next one: Rally, a hosted software solution created by Rally Software based in Colorado, United States.  With over 100,000 users, this is definitely one of the bigger players in the Agile field.

Rally comes in three versions (more about those later) and for this review we have looked at the Community Edition, which is free to use and allows up to 10 users and 1 project.

Is Rally easy to use?

The interface of Rally looks clean and gives you clear views on your backlog and iterations. When signing in for the first time, you will be asked for your role in the project, and Rally will create a personal dashboard for you with information panels configured to help you in the day to day tasks that come with that role.

The dashboard is a very powerful feature in Rally, allowing you to select and drag panels with all kinds of graphical information about the current iteration and releases. It even allows you to add panels with the content of other websites, or custom build your own  using HTML and Javascript.

Unfortunately, the great feeling of User Experience doesn’t last while navigating around the application.  On first impression, most screens look good and they seem to make sense, until you start to use them, and you quickly find inconsistencies and other quirks. A few examples:

  • It is possible to configure the displayed columns in some screens, but not in others.
  • There is a number of actions you can do on one user story in a list, but once you select a number of stories some of the actions (like delete) are suddenly no longer available
  • Clicking on the name of the story will result in a popup being displayed where you can change a subset of the values. However if you wanted to get the full overview, you would have to press another (obscure) icon that results in a new screen with more options.

All that said, even though it will make the learning curve longer, you will get used to most of those UX quirks and it doesn’t necessarily make Rally a bad product. If Rally would fix some of those issues, we do think that it will make the acceptance of the product in new environments easier.

According to their website Rally supports IE from version 8.0, Firefox from 3.6 and the newest versions of Chrome (from 6.0) and Safari (from 4.0). We only used Chrome during testing and we found no problems while using that browser.

What apps are available?

Next to the flexible dashboard, the Apps are a very powerful feature of Rally.

Rally has provided an App-SDK that allows you (and other app providers) to build free app extensions -using Javascript and HTML- that can be used in the Rally software. At the time of writing the app library has 52 apps available, ranging from digital kanban walls to burn down charts and other visual reports.

It is really hard to select the right app from the tiny screens inside Rally itself. There is too much information and no filter option. The best way to select the app you want to use (or adapt) is to go read the descriptions on the Rallydev website first and then select that app within the tool.

The apps screen itself is also a bit hard to find; it has been hidden in an unobtrusive menu option (see the screenshot).  It allows you to extend the menu’s with one or more apps, the process is awkward but the result is really nice: you have just created a menu item that shows you exactly what you want to see. One more thing, there seems to be a restriction on the Community edition only, allowing you to use only one app at a time (this restriction isn’t listed on the Rally website).

Overall: although setup is awkward, apps are definitely a very interesting feature.

How easy is to configure Rally?

Setting up a project in Rally felt fairly intuitive.  That said, there is no real workflow to help you through setting up a project, so you have to find out by yourself how the software works with Iterations and Releases. Then again, if you are familiar with Agile projects and terminology, this will feel quite natural and setup will be quick.

Adding new users is easy; every user has some default values (like timezones and permissions) that can be changed if needed. The overview of all the users works quite well. In the Community edition there is only one project, so every user just gets added to that project. In the other editions you can add users to projects and give them different permissions.

One of the major shortcomings of Rally is the fact that you cannot change the workflow states. They use “Defined”, “In progress”, “Completed” and “Accepted” and there is no way to change those names or add new states. Obviously, this is no issue if you are fine with those states, but if your teams wants to add a new state such as “Testing” or “In review” you’re out of luck.

What I really liked about Rally

Backlog Prioritisation & Management

The product owner, and in many cases the BA, in an agile environment spends a lot of his time writing new user stories and (re-)prioritising the backlog.

Rally offers an easy way of quickly entering a number of user stories without loosing the overview, by just typing the title and selecting a few attributes directly in the overview screen. This is particularly convenient after (or during) a brainstorm on (new) functionality for the project or product.

When it comes to prioritisation you can either drag and drop your user stories to the desired position on the backlog, or change a setting and manually enter priorities as a number. Dragging the stories to the desired spot in the priority list does feel the most intuitive and flexible, and a nifty little line shows your current position while dragging.

It is also possible to create parent-child relationships between stories, allowing for the creation of “epics” that many Product Owners like to use.

It does take some time to work out how to use the different overviews. It was not clear to us why the creators of Rally chose to have a separate “backlog” and “user stories” overview with very different behaviours and fields.

Iteration Management

There is a dedicated screen in Rally for iteration management. It allows you to drag stories from the backlog to the iterations and back again. When a story is dragged into an iteration, Rally checks the velocity for that iteration (you can set that per iteration) and indicates how many more points can be scoped in.

Most teams we know will use a physical wall/board to maintain progress during the iteration. To maintain synchronisation between distributed teams a digital wall can be very helpful. This is also true for Iteration managers and Product Managers who like to have a digital backup of the current status in case the cleaner accidentally takes down the wall.

Although it is hidden in the massive amount of apps, there is a wall/board that allows team members to drag and drop stories from one stage to another.

Rally Wall..

While using the wall you are confronted with the Rally limitation that I mentioned before: there is no way to change the states. So you will either have to make your physical wall reflect the states/lifecycle that Rally uses, or accept the inconsistency.

Many Iteration Managers like to use burn down (or burn up) charts to see how the iteration is progressing. It is a bit disappointing that Rally does not provide burn down charts out of the box as most agile teams that we know use those charts. However, there are a number of extension apps available that can be used for this purpose.

The ones we tried however were only able to show burn charts for stories with tasks attached. If your team does not break its stories down in tasks, and you want a Rally burndown, you will probably have to wait until someone creates that extension, or do it yourself.

Story composition

As described before, it is easy to quickly create a number of user stories by just entering their titles. Editing the stories themselves works with an easy to use rich text editor that works as expected. It is possible to attach one or more files, with a maximum size of 5 MB, and all types seem to be allowed. We could not find a setting for changing the maximum attachment size, but this might be available in the paid versions.

Edit User Story

Other “objects” can also be attached to the story such as defects, test-cases, tasks and comments. The latter allows people to comment on the story, but there is no way to remove or edit comments that have been placed before.

Breaking down stories into tasks is something some teams like to do during their estimation sessions. It is fairly easy to create those, and in the iteration status overview it is possible to see those tasks, but tasks are not visible on the digital wall app. This seems to be more a shortcoming of the wall app, than the Rally application itself though.

Most Agile testers we know like to keep track of the defects they raise on a story, so that they can retest those defects once the implementation of that story has been redeployed.

Rally’s implementation of defect management is easy to use and allows team members to quickly open and close defects, add attachments (like screenshots) to the defects and get a good overview of the number of current outstanding defects.

Does Rally integrate with other tools?

We haven’t tested this but according to their website Rally integrates with a number of tools and platforms. IDE’s like Eclipse, Bugtracking tools like Bugzilla and even competitor Jira, Source code management systems like Subversion and Git (only local versions, not with Github) and build tools like Jenkins and Ant. All those integrations are only available in the paid versions: Enterprise and Unlimited.

One possibly interesting feature that is only available in the Unlimited version is the integration with Salesforce that would allow customers and other stakeholders to enter bugs or new feature requests through Salesforce directly in Rally.

How does Rally handle release management?

Rally is set up around releases and iterations. Although an iteration can exist by itself, the idea is that you create a release with a beginning and end date, and then add iterations to that release. If you already had iterations set up before you create releases, Rally will add the iterations that fit in the time-frame to that release.

In the community version of Rally, release management is limited to an overview that provides all stories and defects within a release. The paid versions also have integration with Build and Source code systems, but we haven’t tested those.

Reporting

The standard overview screens of Rally are basic but contain the necessary information: they will show you the number of completed stories in an Iteration or Release as a list, not in any graphical way.

This is where the power of Apps extensions comes in. There is an app for  just about every type of report you can think of and with a little tinkering in Javascript and HTML you can create any type of report you want.

Pricing

The Community edition is free and allows one project and up to 10 users.

The Enterprise edition has unlimited users and unlimited projects, and -at the time of writing- costs 35 USD per user per month. The Unlimited edition costs 49 USD per user per month, and adds single sign on (via SAML 2.0 which is supported by Google for example), test management (providing extra fields, screens and overviews for test cases and suits) and integration with a number of enterprise tools such as Salesforce.

What support does Rally provide?

Rally has extensive help material on their website and offers e-mail as a way to ask for specific support. The paid editions have a special function in the tool itself that we haven’t tried.

We did try asking a support question via e-mail and received a detailed and helpful answer within a business day. While this is hardly scientific evidence of good support, it does look positive.