Regardless of your current state of tools, building an effective Continuous Integration suite of significant automated regression tests is the key to moving to a higher level of confidence in today’s development world.
In the evolution timeline of software development tools, new tools have recently proliferated. We have all been sold on collaboration, transparency and traceability – but that is not enough. Today, test tools are more often taking center stage. Integrated test platforms can push us deeper technically, help test faster, add more tools for our use and help us provide better, faster appraisals of the product under test.
In this article, I will discuss the introduction and integration of test tools into project wide tools, called ALM (application lifecycle management) and go into greater details about eXtreme Programming’s important Continuous Integration practice.
Remember: not too long ago, test teams had bug tracking tools. That was it. Then we added automation tools that pushed us to get test case managers. For years, that was where we stood. Now, imagine your user stories, test cases, automation, source control, unit test automation, API test automation, UI test automation and bug tracker all in the same tool. For many software development teams, this is already taken for granted. If you have not moved to this situation yet – get ready.
Team tools often get brought into a company where the main purpose is communication and transparency. Just as important to testers are – speed, ease, notification and access to information. Transparency and centralized communication are great for management and project progress. Using integrated test platforms, especially those inside ALM, give test teams access to information and should make your job easier.
Stand-alone tools no longer serve us in today’s fast-paced world. I have too much experience with frustrated test teams using a stand-alone tool like Quality Center where the requirements section is empty because the marketing team or business analysts would not use “the test tool.” And the bug data was piped out to Bugzilla or another tool because developers would not go into “the test tool.”
This won’t happen with the test tools in an ALM platform, for example, like TFS/Visual Studio, Rally or a custom built suite of tools like Jira, Bamboo, Bugzilla, nUnit, Subversion, etc. The tasks, storyboards, user stories, test cases, bugs, code repository and automation are all visible and accessible from one interface. The platform belongs to the product owner, designer, developer, tester, release engineer, database admin – everyone. The test team’s tool is everyone’s tool. Transparency and visibility are obvious, but there is no withholding of information due to “it’s not my tool” or lack of access. It’s a big step forward.
ALM—Application Lifecycle Management: A Brief History
The main forerunner of this ALM, all-in-one, tool explosion is Rational Software’s suite, now Rational/IBM: Requisite Pro (requirements), Rational Rose (code design), ClearCase (source control), Rational Test manager, Rational Robot (test automation), and ClearQuest (bug tracking). We now have a wide variety of tools – from enterprise wide tools to open source individual tools – so you can put all the pieces together to have the same impact.
The use of these tools has grown in the past couple of years as teams have become more distributed, faster and more dynamic while at the same time, coming under increased pressure from executives to be more measurable and manageable. When Agile, as a development framework, grew explosively, the anti-tool development framework got inundated by tools. Now every tool professes to support Agile practices and be Lean.
ALM tool suites can generally manage user stories/requirements, call center/support tickets and also help track a sprint’s planning game, store test cases associated with user stories, include source control, facilitate Continuous Integration, coverage analysis and track bugs.
There are many tools that do this, but the big leap forward with current ALM tools is the ability to be linked to source control, unit testing frameworks, and GUI test automation tools. Lite ALM tools satisfy the need for speed required by the rapid nature of software development – it’s a communication tool.
There are great reasons for using ALM tools, just as there are great reasons not to use them! Let’s start with great reasons to use them:
- If you’re working on products with many interfaces to other teams and products, ALM tools can be very beneficial in communicating what your team is doing and where you are in the project.
- ALM tools are essential if you’re working with distributed teams.
- If you are working on projects with large product backlogs, or with dynamic backlogs often changing value, a management tool can be very helpful.
- ALM tools are useful if you have teams offshore or separated by big time zone differences.
- You are scaling Agile – for highly integrated teams, enterprise communication tools may be essential.
- If, for whatever reason, the test team is still required to produce too many artifacts – like test cases, bug reports, build reports, automation result reports and status reports (ugh!) – most ALM tools will have all the information you need to report readily available.
Some reasons not to use them:
- The Agile manifesto states people and their interactions are more important than processes and tools. Clearly, tools cannot dictate process – that is a mistake. People and their interactions must be supported by tool use or do not use one.
- Bad tool use is not Lean. Tools have to be easy, fast and not create waste.
- The tools can be expensive or the open source ones have a high build/integrate/maintain cost. I have to say, if this is the problem, the cost to the team of not using an ALM will be greater.
Development and Test Tools
In the old days, developers had their IDEs (Integrated Development Environment). These tools have progressed to be more than text editors and compilers. They’ve grown into task tracking, build management, debuggers, coverage analyzers, syntax checkers, unit test harnesses, dependency analyzers, etc. These tools now make programming easier, give access to many useful built-in tools — some with a variety of libraries – lots of “stuff” to help programmers work faster, more efficiently and produce a better product faster. An IDE is now a unified platform for development, production and white box testing of code.
Testing tools developed more slowly. Test teams started to have access to sets of integrated tools in the mid-1990s mainly because teams began using multiple automation tools. The early integrated tools included a GUI/UI test tool, a performance test tool and perhaps an API test tool. Teams also needed a single bug database, maybe automatic bug creation, and, to reduce the maintenance headache, a unified interface for test case creation, execution and management. These tools tended to be “owned” by test teams. Rarely, if ever, were they used by other roles on the project team and there was no avenue to integrate the test team into the wider project.
The days of separate platforms are going the way of the dinosaur. It can be hugely beneficial for test teams to have easy access to source code, developer tools, coverage analyzers, debuggers and make their own builds as needed. It can also be convenient for developers to run test automation suites or execute test cases specifically linked to pieces of code. In addition to traceability for compliance or coverage measurement, ALM tools support tracing user stories to chunks of code to test cases to bugs. The benefits of linking or tracing items this way is a big benefit for targeted regression, gated builds and resolving issues. These are all the benefits that can be realized from a unified platform.
Continuous Integration- Not just autobuild!
The XP practice of Continuous Integration (CI) is revolutionizing software development. With the need for speed, rapid releases and very compressed development cycles, an automated build process is a no-brainer. This is not rocket science and not specific to Agile/XP. Continuous Integration tools have been around for years; there are many of them and they are very straightforward to use. It is also common for test teams to take over the build process. Implementing an automatic build process by itself is a step forward, but a team will realize more significant gains if they add to automated builds with Continuous Integration.
Continuous Integration includes the following:
- Automated build process.
- Re-running of unit tests.
- Automated smoke test/build verification.
- Automated regression test suite.
- Build report.
The ability to have unit tests continually re-run has significant advantages:
- It can help find integration bugs faster.
- Qualifying builds faster will free-up tester time.
- Testing on the latest and greatest build will save everyone time.
The positives for Continuous Integration far outweigh any resistance to their implementation. Test teams can take on more responsibility here: they can be more in control of the testing process — on how many builds they take and when they take them.
There’s no reason why a test team can’t take over making a build. There is also no reason a test team does not have someone already technically skilled enough to manage Continuous Integration. With straightforward Continuous Integration tools such as Bamboo, Cruise/CruiseControl, Git, Jenkins, Hudson—the list goes on— test teams can do it. Integrated platforms make this possible with easy access, easy notification, sharing information, linking requirements or stories to code, to test cases, to automation to bugs as needed.
I suggest you visit Wikipedia to see a very large list of Continuous Integration tools for integration into various platforms at this here.
Test teams can use Continuous Integration tools to make a very effective build validation process:
- Scheduling new builds.
- Automatic re-running of the unit tests. Note: this does not mean write the unit tests, it means re-run—for example, the JUnit harness of tests.
- Re-running automated user story acceptance tests and whatever automated smoke tests are built.
Test teams taking over the Continuous Integration process with high volume automation is, from my experience, what separates consistently successful teams from teams that enjoy only occasional success.
Significant Automation: Automation is the rule, not optional
It’s a forgone fact that significant, sophisticated, yet simple automation is mandatory. If you do not have significant, high volume automation, you are doing the product team a disservice. All the integrated development/test platforms are including significantly more test tools. Many have GUI test recorders, limited performance tools and database test tools. To have these automation tools integrated into the test and development platform is essential.
High-volume test automation is the only path to rapid development and confident deployment. Making it happen can still be a challenge. It requires perseverance, support, cooperation, expertise and more! But at the same time, having an integrated test platform will make it more effective than ever.
From TestArchitect for Visual Studio, to Selenium and Eclipse integrated with Git or Hudson, there are a wide variety of possibilities for having all your test tools integrated into a development platform beneficial to both dev and test teams. Integrating testing tools with development tools and project wide tools allows test teams to provide the maximum benefit to the whole product team.
Project Management
The project management aspects of software development are facilitated by having integrated tools. For test teams, the obvious reasons are traceability, reporting, communication and transparency. The most important reasons are: speed, ease of work, being aware of other team members’ work, more automation, more technical testing, smarter testing, form more grey box knowledge, leveraging other team members’ work, knowledge and artifacts.
Summary
Stand-alone test tools, although beneficial, are not leveraging easy technologies we have available today. To keep pace with current software development practices, efficient tools to support rapid release and confident deployment are essential.
Having a unified test platform is the first step. Integrating the test platform with the development platform and the whole ALM is the most effective infrastructure for software development. Most importantly, regardless of your current state of tools, building an effective Continuous Integration suite of significant automated regression tests is the key to moving to a higher level of confidence in today’s development world.
