To begin this article, it would be a good idea to, remember this key point:
Agile Manifesto Value #1
Individuals and interactions over processes and tools
Tools work at the service of people. People, particularly intelligent people, can never be slaves to tools. People talking to each other, working together and solving problems is much more important than following a process or conforming to a tool. This also means someone who tells you a tool will solve your Agile problems knows nothing about Agile. A disaster for software development over the past decade has been companies investing gobs of money into “project management tools” or “test management tools” to replace, in some cases, successful practices of software development, in favor of the process the tool or tool vendor told that company was “best practice”. That made certain tool vendors rich and many software development teams unhappy.
If your team wants to be Agile, and work in line with the value outlined above, I have a couple of rules to remember when it comes to tools and Agile.
Rule #1 – Tools need to fit teams. Teams should not fit into tools!
Rule #2 – Test teams need to grow in their technical responsibility.
These two ideas sum up how I feel about most tools that are part of an Agile development project. For example, when you add management tools, this will add overhead, reduce efficiency and limit the Agile idea of self-directing teams. However, communication tools can help in distributed development situations and scaling into large organizations. Some tools work to provide services to teams and others do not. This article is mainly about tools and how they relate to test teams. We are going to talk a lot about the different toolsets- that are most important to test teams. We won’t talk about unit test frameworks since test teams very rarely manage them, but we will discuss test case managers in the Agile world.
I’m also going to talk about the changes that the test teams need to undergo to successfully implement or use these new tools.
Often when beginning an Agile project, there is a need for some re-tooling in order to be successful. The same set of tools we used in traditional development may not be enough to support rapid communication, dynamic user story change and ranking, and increased code development.
The groups of tools of specific use to test teams we will focus on are:
• Application Lifecycle Management (ALM) – such as Rally, Atlassian’s JIRA Studio and IBM’s Rational Team Concept
• Test Case Management Tools – such as HP’s Quality Center
• Continuous Integration Tools – such as CruiseControl, Bamboo, and Hudson ( Including Source control tools – such as PerForce, and Visual Source Safe (VSS) type tools)
• Automation Tools – we will not talk about specific tools in this section.
Application Lifecycle Management Tools (ALM)
Sometimes when we talk about tools, we discuss them in relation to the Application Lifecycle. Naturally then, the tool vendors refer to their large suites as Application Lifecycle Management Tools, or ALM. This is the major category of tooling and it is often an enormous undertaking to get these tools implemented. However, there are some minor category tools in the ALM spectrum that we can pull out that relate to testers.
Application Lifecycle Management (ALM) tools have been making a splash in the development world for a few years now. Approximately 15 years ago, Rational (before Rational/IBM) had an entire suite of tools we would now call ALM. From RequisitePro to Rose to ClearCase through ClearQuest, the idea was to have one set of integrated tools to manage software development projects from inception to deployment. Now, there are multitudes of tool sets in this space. The use of these tools has grown in the past couple of years as teams have became more distributed, faster and more dynamic while at the same time coming under increased pressure from executives to be more measurable and manageable. When Agile as a development framework grew explosively, the anti-tool development framework got inundated by tools.
ALM tool suites generally manage user stories/requirements, call center/support tickets, also help track planning poker, store test cases associated with user stories, bugs – there are many tools that do this – but the big leap forward with current ALM tools is them being linked to source control, unit testing frameworks, and GUI test automation tools. Lite ALM tools satisfy the need for speed required by the rapid nature of software development – it’s a communication tool.
There are great reasons for using ALM tools, just as there are great reasons not to use them! Let’s start with great reasons to use them:
- If you’re working on products with many interfaces to other teams and products, ALM tools can be very beneficial in communicating what your team is doing and where you are in the project.
- ALM tools are essential if you’re working with distributed teams.
- If you are working on projects with large product backlogs, or with dynamic backlogs often changing value, a management tool can be very helpful.
- ALM tools are useful if you have teams offshore or separated by big time zone differences
- You are scaling Agile – for highly integrated teams, enterprise communication tools may be essential.
- If, for whatever reason, the test team is still required to produce too many artifacts- mainly the high overhead artifacts like test cases and bug reports, build reports, automation result report, and status reports (ugh!) – most ALM tools will have all the information you need to report readily available.
There are many good reasons to use ALM tools. But the tool can’t control you or the use and benefit of going Agile will be lost.
Most Agile lecturers, including Ken Schwaber, will tell you using Agile in distributed teams will reduce the efficiency. This does not mean Agile cannot be successful with offshoring, but it means it is tougher and chances of ending up with smiles all around are lower. Nothing beats all being in the same place, communicating easily and openly every day and working on our software in my bullpen in the office. If you are in that situation – good for you! However, this is the subject of another magazine article we will bring you in the New Year.
Now the great reasons not to use ALM tools:
- It is very easy for some people to become tool-centric, as though the tool is the project driver rather than the Teams. That is completely waterfall, bad!
- Paper trails and the production of artifacts, “because we always did it this way,” are anti-Agile. Most times the artifacts become stale or useless having little relevance to what is actually built.
- Management overheard is anti-Agile. The only “project management” if I can even use that phrase, is a 15 minute daily scrum. Not a giant, administrative nightmare, high cost overhead, project management process! When the ALM tool becomes a burden and overhead slows down the team, you will need to find ways to have the tools support you rather than slow you down.
A quick word about measurement: the only artifact about measurement in line with by-the-book Agile is burndown charts. Extreme programming espouses velocity. If your team is measuring you by test case creation, test case execution rates, user story coverage there is nothing Agile about it. End of discussion.
If you want to investigate ALM tools further, I recommend you look at a few – check out enterprise social software maker, Atlassian, maker of the famous Jira, and Rally a great, widely used product with dozens of partners for easy integration. These two are at the opposite ends of the cost spectrum.
Test Case Management Tools
Most test organizations have migrated, over the past decade, to using an enterprise wide test case management tool (TCM) for tracking test cases, bugs/issues and support tickets in the same repository. Many test teams have strongly mixed feeling about these tools- sometimes feeling chained to them as they inhibit spontaneous thinking and technical creativity.
To begin talking about test case tools in Agile, there are two important reminders:
- Test cases are not a prescribed or common artifact in Scrum.
- Most companies that are more Agile these days are following some lean manufacturing practices – such as ceasing to write giant requirement docs and engineering specs, and getting really lean on project documentation. Test teams need to get lean also. They can start by cutting back or cutting out test plans and test cases. Do you need a tool to manage test cases in Agile? Strict process says no!
If you are not using a TCM yet and are moving to Agile, or are beginning to use a test case manager, now is not a good idea If you already use one – get leaner!
Still, most ALM tools have large functionality built around test case management that can become an overhead problem for teams focusing on fast, lean, streamlined productivity.
Continuous Integration
In rapid development, getting and validating a new build can kill progress. Yet there is no need for this! In the most sophisticated and successful implementations of Agile development, test teams have taken over source control and the build process. We have already talked about continuous integration in this series (See Agile For Testers Part 3 Automated Build and Build Validation Practice / Continuous Integration for more discussion on this), let’s focus on tools. Source control tools are straightforward. Many test teams regularly use the source controls systems.
There’s no reason why a test team can’t take over making a build. There is also no reason a test team does not have someone already technically skilled enough to manage continuous integration. With straightforward continuous integration tools such as Bamboo, CruiseControl and Build Forge, test teams can do it.
My advice is:
- Learn how these tools work and what they can do
- Build an automated smoke test for build verification
- Schedule builds for when you want delivery
Test teams can use a continuous integration tool to make a big build validation process: scheduling new builds, automatic re-running of the unit tests, re-running automated user story acceptance tests and whatever automated smoke tests are built! Note, this does not mean write the unit tests – it means re-run, for example, the J-unit harness of tests. Test teams taking over the continuous integration process is, from my experience, what separates consistently successful teams from teams that enjoy only occasional success. Easy.
I suggest you visit Wikipedia on CI tools here: http://en.wikipedia.org/wiki/Continuous_integration) to see a very large list of continuous integration tools for various platforms.
Test Automation
I’m going to resist going into detail about automation here. Automation in Agile will have its own large magazine articles and white papers. Remember, when discussing automation in Agile with team members, scrum masters or anyone knowledgeable in Agile, the part of testing that a test team needs to automate is not the unit testing. That automation comes from developers. Test teams need to focus on:
- Automated smoke/ build validation tests
- Large automated regression suites
- Automated user story validation tests (typically a test team handles this. But- there are very successful groups that have developers automate these tests at the unit, api or UI level)
- New functionality testing can be difficult to automate during a sprint. You need a very dynamic and easy method for building test cases. This will be discussed a greater length in an future Agile automation methods paper.
In Agile development, the automation tool is not the issue or problem, it’s how you design, describe and define your tests that will make them Agile or not!
Here are the main points to get about test automation tools in Agile:
- The need for massive automation is essential, clear and non-negotiable. How test teams can do that in such fast-paced development environments needs to be discussed fully in its own white paper.
- That you need a more manageable and maintainable, extendible framework, like tools that easily support action-based testing is clear.
- Developers need to be involved in test automation! Designing for testability and designing for test automation will solve automation problems.
- Training is key to building a successful automation framework.
- Tools never solve automation problems- free or otherwise! Better automation design solves automation problems.
Summary:
Tools in Agile need to be used to fit people, not the other way around.
Tools need to be easy to use (from your perspective- not the tool vendor’s perspective!) and low time and management overheads.
Employ lean manufacturing ideas wherever possible- specifically, cut artifacts and documentation that is not essential.
