Contact Us

Software Testing 3.0: Case Study #1

By | | Test Methods & Metrics

Introduction

Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development that finally delivers on the promise of Software Testing. For more details on the evolution of Software Testing and Software Testing 3.0 see:

  1. The Early Evolution of Software Testing
  2. Software Testing 3.0: Delivering on the Promise of Software Testing

Following is a case study of Software Testing 3.0 principals in action.

MX Logic Inc.:
Automation Expands Testing Coverage, Reduces Testing Time
Environment Prior to LogiGear

MX Logic is a leading provider of email and web managed security services. The company was running a suite of 423 software tests manually to test their security software. Manual Testing was so time consuming that:

  • Manual Testing of their software’s functionality was taking approximately 240 person-hours to run
  • Build acceptance testing was done on an ad-hoc, time-permitting basis

LogiGear’s Solution

LogiGear implemented an Automated Testing program for MX Logic that made use of:

    • Low-cost off-shore LogiGear testing resources in Vietnam. These resources are all trained by LogiGear University in Software Testing 3.0 concepts and tools.
  • TestArchitect, a tool set that integrates the latest methodologies and technologies in one easy-to-use package.

The Automated Software Testing solution implemented by LogiGear allowed the company:

    • To automate existing tests and expanded test coverage to 945 automated test cases that run in 150 to 160 machine hours (the same tests run manually would consume 480 to 500 man hours).
    • To expand testing coverage from 50% to 90%
    • To use a subset of these tests to implement regular automated build acceptance testing that runs in 2 machine hours (the same would take 1 person day manually)
“LogiGear has delivered an innovative turnkey solution that is fully automated, low cost and has expanded our testing coverage by 90%.

While reviewing any Test Automation initiative it is important to understand that a successful Automation project requires care and maintenance. Not only does LogiGear provide MX Logic with an excellent Automation tool in TestArchitect, it provides me with the cost effective resources to enhance and maintain our Automation projects. With a comparable feature set to more expensive licensed tools, my Automation budget can be utilized on both the tool and the engineers to support it for the same price. And this guarantees the success of our Automation, where others fail!”

– Jamie Tischart, Director,
Quality Assurance, MX Logic

Coverage expanded from 50% to 90%

Automated test coverage expanded from 10% to 80%

Twice as many tests tested in less time (150 machine hours vs. 240 person hours)

Added regular build-acceptance testing

Added production portal performance testing
    • To reuse the automated test suite release after release with only the addition of a small number incremental test cases to test new functionality
  • To avoid staff growth in their domestic operations
Using the difference in cost between fully-burdened U.S. based resources and Vietnam based resources, LogiGear estimates that this level of Automated Testing effort done manually would have cost the company an additional $1 million or more. LogiGear’s Automated Software Testing efforts have expanded testing coverage and effectiveness while avoiding significant additional expense.
LogiGear’s Automated Software Testing efforts expanded testing coverage and effectiveness while avoiding significant additional expense.

In addition, LogiGear created performance testing to help MX Logic monitor their customer experience on their production portal. They run this daily to ensure that their customer experience is not negatively affected by any new features or Internet traffic spikes.

Next Up

LogiGear will be producing an additional 300 automated test cases to cover new functionality that MX Logic will be adding to their software. This will bring the total number of test cases to nearly four times the company’s original manual test suite.

LogiGear Corporation

LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers public and corporate software-testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast and cost-effective results. Since 1994, LogiGear has worked with hundreds of companies from the Fortune 500 to early-stage startups, creating unique solutions to exactly meet their needs. With facilities in the US and Vietnam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs. For more information, contact Joe Hughes + 01 650.572.1400

TAG:

LogiGear Corporation
LogiGear Corporation provides global solutions for software testing, and offers public and corporate software testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast, cost-effective results. Since 1994, LogiGear has worked with Fortune 500 companies to early-stage start-ups in, creating unique solutions to meet their clients’ needs. With facilities in the US and Viet Nam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs.

The Related Post

Something Old, Something New: Requirements and Specifications
Differences in interpretation of requirements and specifications by programmers and testers is a common source of bugs. For many, perhaps most, development teams the terms requirement and specification are used interchangeably with no detrimental effect. In everyday development conversations the terms are used synonymously, one is as likely to mean the “spec” as the “requirements.”
August 2009 - Three Kinds of Measurement and Two Ways to Use Them
This article was originally featured in the July/August 2009 issue of Better Software magazine. Read the entire issue or become a subscriber. People often quote Lord Kelvin: “I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express ...
July 2009 - Issues About Metrics About Bugs
This article was originally featured in the May/June 2009 issue of Better Software magazine. Read the entire issue or become a subscriber. In my travels, I’ve worked with a number of companies that have attempted to assess the quality of their testing — or worse, their testers — using poorly considered metrics. Sometimes the measurement ...
The Great Reads of Software Testing
Are you looking for the best books on software testing methods? Here are 4 books that should be on your reading list! The Way of the Web Tester: A Beginner’s Guide to Automating Tests By Jonathan Rasmusson Whether you’re a traditional software tester, a developer, or a team lead, this is the book for you! It ...
LogiGear Magazine March Issue 2018: Under Construction: Test Methods & Strategy
LogiGear Magazine March Issue 2018: Under Construction: Test Methods & Strategy
Harry Robinson Talks Training
At VISTACON 2011, Harry sat down with LogiGear Sr. VP, Michael Hackett, to discuss various training methodologies. Harry Robinson Harry Robinson is a Principal Software Design Engineer in Test (SDET) for Microsoft’s Bing team, with over twenty years of software development and testing experience at AT&T Bell Labs, HP, Microsoft, and Google, as well as ...
Key Success Factors for Keyword Driven Testing
Introduction Keyword-driven testing is a software testing technique that separates much of the programming work of test automation from the actual test design. This allows tests to be developed earlier and makes the tests easier to maintain. Some key concepts in keyword driven testing include:
LogiGear Magazine - March 2015 - Testing Strategies and Methods - Fast Forward to Better Testing
LogiGear_Magazine–March_2015–Testing_Strategies_and_Methods-Fast_Forward_To_Better_Testing
Who Cares About Test Plans? You Should!
Test plans have a bad reputation, and perhaps, they deserve it! There’s no beating around the bush. But times have changed. Systems are no longer “black boxes” where QA Teams are separated from design, input, and architecture. Test teams are much more technically savvy and knowledgeable about their systems, beyond domain knowledge. This was an old ...
Jaideep Khanduja: Progressive Software Testing Approach by Acquiring Soft Skills
In software testing, we need to devise an approach that features a gradual progression from the simplest criteria of testing to more sophisticated criteria. We do this via many planned and structured steps, each of which brings incremental benefits to the project as a whole. By this means, as a tester masters each skill or area ...
A Primer on Passwords
Please note: This article was adapted from a blog posting in Karen N. Johnson’s blog on July 24, 2007. Introduction The password field is one data entry field that needs special attention when testing an application. The password field can be important (since accessing someone’s account can start a security leak), testers should spend more ...
Software Testing 3.0: Organization and Process Ramifications
Introduction Many companies have come to realize that software testing is much more than a task that happens at the end of a software development cycle. They have come to understand that software testing is a strategic imperative and a discipline that can have a substantial impact on the success of an organization that develops ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe
SHARE: Facebooktwitterlinkedinmail