Tools for Integrated Test Platforms

Regardless of your current state of tools, building an effective Continuous Integration suite of significant automated regression tests is the key to moving to a higher level of confidence in today’s development world.

In the evolution timeline of software development tools, new tools have recently proliferated. We have all been sold on collaboration, transparency and traceability – but that is not enough. Today, test tools are more often taking center stage. Integrated test platforms can push us deeper technically, help test faster, add more tools for our use and help us provide better, faster appraisals of the product under test.

In this article, I will discuss the introduction and integration of test tools into project wide tools, called ALM (application lifecycle management) and go into greater details about eXtreme Programming’s important Continuous Integration practice.

Remember: not too long ago, test teams had bug tracking tools. That was it. Then we added automation tools that pushed us to get test case managers. For years, that was where we stood. Now, imagine your user stories, test cases, automation, source control, unit test automation, API test automation, UI test automation and bug tracker all in the same tool. For many software development teams, this is already taken for granted. If you have not moved to this situation yet – get ready.

Team tools often get brought into a company where the main purpose is communication and transparency. Just as important to testers are – speed, ease, notification and access to information. Transparency and centralized communication are great for management and project progress. Using integrated test platforms, especially those inside ALM, give test teams access to information and should make your job easier.

Stand-alone tools no longer serve us in today’s fast-paced world. I have too much experience with frustrated test teams using a stand-alone tool like Quality Center where the requirements section is empty because the marketing team or business analysts would not use “the test tool.” And the bug data was piped out to Bugzilla or another tool because developers would not go into “the test tool.”

This won’t happen with the test tools in an ALM platform, for example, like TFS/Visual Studio, Rally or a custom built suite of tools like Jira, Bamboo, Bugzilla, nUnit, Subversion, etc. The tasks, storyboards, user stories, test cases, bugs, code repository and automation are all visible and accessible from one interface. The platform belongs to the product owner, designer, developer, tester, release engineer, database admin – everyone. The test team’s tool is everyone’s tool. Transparency and visibility are obvious, but there is no withholding of information due to “it’s not my tool” or lack of access.  It’s a big step forward.

ALM—Application Lifecycle Management: A Brief History

The main forerunner of this ALM, all-in-one, tool explosion is Rational Software’s suite, now Rational/IBM: Requisite Pro (requirements), Rational Rose (code design), ClearCase (source control), Rational Test manager, Rational Robot (test automation), and ClearQuest (bug tracking).  We now have a wide variety of tools – from enterprise wide tools to open source individual tools – so you can put all the pieces together to have the same impact.

The use of these tools has grown in the past couple of years as teams have become more distributed, faster and more dynamic while at the same time, coming under increased pressure from executives to be more measurable and manageable. When Agile, as a development framework, grew explosively, the anti-tool development framework got inundated by tools. Now every tool professes to support Agile practices and be Lean.

ALM tool suites can generally manage user stories/requirements, call center/support tickets and also help track a sprint’s planning game, store test cases associated with user stories, include source control, facilitate Continuous Integration, coverage analysis and track bugs.

There are many tools that do this, but the big leap forward with current ALM tools is the ability to be linked to source control, unit testing frameworks, and GUI test automation tools. Lite ALM tools satisfy the need for speed required by the rapid nature of software development – it’s a communication tool.

There are great reasons for using ALM tools, just as there are great reasons not to use them! Let’s start with great reasons to use them:

  • If you’re working on products with many interfaces to other teams and products, ALM tools can be very beneficial in communicating what your team is doing and where you are in the project.
  • ALM tools are essential if you’re working with distributed teams.
  • If you are working on projects with large product backlogs, or with dynamic backlogs often changing value, a management tool can be very helpful.
  • ALM tools are useful if you have teams offshore or separated by big time zone differences.
  • You are scaling Agile – for highly integrated teams, enterprise communication tools may be essential.
  • If, for whatever reason, the test team is still required to produce too many artifacts – like test cases, bug reports, build reports, automation result reports and status reports (ugh!) – most ALM tools will have all the information you need to report readily available.

Some reasons not to use them:

  • The Agile manifesto states people and their interactions are more important than processes and tools. Clearly, tools cannot dictate process – that is a mistake.  People and their interactions must be supported by tool use or do not use one.
  • Bad tool use is not Lean. Tools have to be easy, fast and not create waste.
  • The tools can be expensive or the open source ones have a high build/integrate/maintain cost. I have to say, if this is the problem, the cost to the team of not using an ALM will be greater.

Development and Test Tools

In the old days, developers had their IDEs (Integrated Development Environment). These tools have progressed to be more than text editors and compilers. They’ve grown into task tracking, build management, debuggers, coverage analyzers, syntax checkers, unit test harnesses, dependency analyzers, etc. These tools now make programming easier, give access to many useful built-in tools — some with a variety of libraries – lots of “stuff” to help programmers work faster, more efficiently and produce a better product faster. An IDE is now a unified platform for development, production and white box testing of code.

Testing tools developed more slowly. Test teams started to have access to sets of integrated tools in the mid-1990s mainly because teams began using multiple automation tools. The early integrated tools included a GUI/UI test tool, a performance test tool and perhaps an API test tool. Teams also needed a single bug database, maybe automatic bug creation, and, to reduce the maintenance headache, a unified interface for test case creation, execution and management. These tools tended to be “owned” by test teams. Rarely, if ever, were they used by other roles on the project team and there was no avenue to integrate the test team into the wider project.

The days of separate platforms are going the way of the dinosaur. It can be hugely beneficial for test teams to have easy access to source code, developer tools, coverage analyzers, debuggers and make their own builds as needed. It can also be convenient for developers to run test automation suites or execute test cases specifically linked to pieces of code. In addition to traceability for compliance or coverage measurement, ALM tools support tracing user stories to chunks of code to test cases to bugs. The benefits of linking or tracing items this way is a big benefit for targeted regression, gated builds and resolving issues. These are all the benefits that can be realized from a unified platform.

Continuous Integration- Not just autobuild!

The XP practice of Continuous Integration (CI) is revolutionizing software development. With the need for speed, rapid releases and very compressed development cycles, an automated build process is a no-brainer. This is not rocket science and not specific to Agile/XP. Continuous Integration tools have been around for years; there are many of them and they are very straightforward to use. It is also common for test teams to take over the build process. Implementing an automatic build process by itself is a step forward, but a team will realize more significant gains if they add to automated builds with Continuous Integration.

Continuous Integration includes the following:

  • Automated build process.
  • Re-running of unit tests.
  • Automated smoke test/build verification.
  • Automated regression test suite.
  • Build report.

The ability to have unit tests continually re-run has significant advantages:

  • It can help find integration bugs faster.
  • Qualifying builds faster will free-up tester time.
  • Testing on the latest and greatest build will save everyone time.

The positives for Continuous Integration far outweigh any resistance to their implementation. Test teams can take on more responsibility here: they can be more in control of the testing process — on how many builds they take and when they take them.

There’s no reason why a test team can’t take over making a build. There is also no reason a test team does not have someone already technically skilled enough to manage Continuous Integration. With straightforward Continuous Integration tools such as Bamboo, Cruise/CruiseControl, Git, Jenkins, Hudson—the list goes on— test teams can do it. Integrated platforms make this possible with easy access, easy notification, sharing information, linking requirements or stories to code, to test cases, to automation to bugs as needed.

I suggest you visit Wikipedia to see a very large list of Continuous Integration tools for integration into various platforms at this here.

Test teams can use Continuous Integration tools to make a very effective build validation process:

  • Scheduling new builds.
  • Automatic re-running of the unit tests. Note: this does not mean write the unit tests, it means re-run—for example, the JUnit harness of tests.
  • Re-running automated user story acceptance tests and whatever automated smoke tests are built.

Test teams taking over the Continuous Integration process with high volume automation is, from my experience, what separates consistently successful teams from teams that enjoy only occasional success.

Significant Automation: Automation is the rule, not optional

It’s a forgone fact that significant, sophisticated, yet simple automation is mandatory. If you do not have significant, high volume automation, you are doing the product team a disservice. All the integrated development/test platforms are including significantly more test tools. Many have GUI test recorders, limited performance tools and database test tools. To have these automation tools integrated into the test and development platform is essential.

High-volume test automation is the only path to rapid development and confident deployment. Making it happen can still be a challenge. It requires perseverance, support, cooperation, expertise and more! But at the same time, having an integrated test platform will make it more effective than ever.

From TestArchitect for Visual Studio, to Selenium and Eclipse integrated with Git or Hudson, there are a wide variety of possibilities for having all your test tools integrated into a development platform beneficial to both dev and test teams. Integrating testing tools with development tools and project wide tools allows test teams to provide the maximum benefit to the whole product team.

Project Management

The project management aspects of software development are facilitated by having integrated tools. For test teams, the obvious reasons are traceability, reporting, communication and transparency. The most important reasons are: speed, ease of work, being aware of other team members’ work, more automation, more technical testing, smarter testing, form more grey box knowledge, leveraging other team members’ work, knowledge and artifacts.


Stand-alone test tools, although beneficial, are not leveraging easy technologies we have available today. To keep pace with current software development practices, efficient tools to support rapid release and confident deployment are essential.

Having a unified test platform is the first step. Integrating the test platform with the development platform and the whole ALM is the most effective infrastructure for software development. Most importantly, regardless of your current state of tools, building an effective Continuous Integration suite of significant automated regression tests is the key to moving to a higher level of confidence in today’s development world.

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

Source: From I.M.Testy (BJ Rollison’s blog) I just finished reading Implementing Automated Software Testing by E.Dustin, T. Garrett, and B. Gauf and overall this is a good read providing some well thought out arguments for beginning an automation project, and provides strategic perspectives to manage a test automation project. The first chapter made several excellent ...
As our world continues its digital transformation with excitement in the advancement and convergence of so many technologies- from AI, machine learning, big data and analytics, to device mesh connectivity, nor should we forget VR and AR- 2017 promises to be a year that further transforms the way we work, play and take care of ...
I recently came back from the Software Testing & Evaluation Summit in Washington, DC hosted by the National Defense Industrial Association. The objective of the workshop is to help recommend policy and guidance changes to the Defense enterprise, focusing on improving practice and productivity of software testing and evaluation (T&E) approaches in Defense acquisition.
The Cloud demands that we be as nimble as possible, delivering features and fixes in almost real-time fashion. Both customer and provider rely on software development that can maintain quality while being light on its feet and constantly moving. In addition, Cloud-oriented systems tend to be highly complex and dynamic in structure — more than ...
One of my current responsibilities is to find ways to automate, as much as practical, the ‘testing’ of the user experience (UX) for complex web-based applications. In my view, full test automation of UX is impractical and probably unwise; however, we can use automation to find potential UX problems, or undesirable effects, even in rich, ...
Many organizations rely on HP Quality Center to design test plans and track test results. TestArchitect’s Quality Center integration makes working with QC as easy as pie. TestArchitect (TA) is a three-in-one tool for Test Management, Test Development, and Test Automation. Users can create and manage test assets, execute tests, track and analyze test results, ...
Are you frustrated with vendors of test automation tools that do not tell you the whole story about what it takes to automate testing? Are you tired of trying to implement test automation without breaking the bank and without overloading yourself with work? I experienced first-hand why people find test automation difficult, and I developed ...
LogiGear Magazine – April 2014 – Test Tool and Automation
When automated tests are well-organized and written with the necessary detail, they can be very efficient and maintainable. But designing automated tests that deal with data can be challenging if you have a lot of data combinations. For example, let’s say we want to simulate a series of 20 customers, along with the number of ...
LogiGear Magazine – The Big Testing Issue – April 2012
We’re excited to share with you the latest and greatest features of TestArchitect Gondola, as well as how to use them. So, check them out below! Gondola Studio UI/UX ImprovementsGondola Studio’s new Test Execution Dialog makes it easy to configure and run your test. You can choose the browser or device you’d like to run ...
“Happy About Global Software Test Automation: A Discussion of Software Testing for Executives” Author: Hung Q. Nguyen, Michael Hackett, and Brent K. Whitlock Publisher: Happy About (August 1, 2006) Finally, a testing book for executives!, November 17, 2006 By Scott Barber “Chief Technologist, PerfTestPlus” Happy About Global Software Test Automation: A Discussion of Software Testing ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news