Agile is a philosophy focused on delivering constant value to customers incrementally and frequently, based on communication and feedback. These two ingredients are vital to a successful Agile recipe.
Agile is no longer a buzzword or an unknown territory in the industry. Agile has progressed leaps and bounds the last few years and has matured to a widely accepted methodology. Testing in Agile projects required a paradigm shift for traditional testing roles. It required a change in tester’s attitudes from a relay race oriented approach to an upfront involved role. The Agile approach focuses on getting things right the first time, reducing the need for QA testers to get something over the finish line. But it’s easier said than done. How does it happen in reality? Does it actually happen?
Agile is a philosophy focused on delivering constant value to customers incrementally and frequently, based on communication and feedback. These two ingredients are vital to a successful Agile recipe and testers have an important role to play in creating value in Agile projects throughout different phases of iteration.
I have outlined the QA activities as three phases in Agile projects. However, this is no golden rule and it can be flexible according to the project situation. The Agile QA tester’s role is not limited to a set of pre-defined processes, as the methodology will dictate roles based on the situation.
Pre-iteration: This is the time where requirements are analyzed in detail by the BAs and acceptance criteria are detailed for that story. As QA testers are immediate consumers of those requirements, it is important to verify the requirements early and often.
Story verification (Requirement Verification): Agile testing is all about giving feedback early, not necessarily only by testing the requirements, but doing requirement testing early. QA testers really need to look at the requirement / stories upfront for clarity and testability. This will determine the requirements that are unambiguous and testable (I believe unambiguous in Agile is not much different in context compared to a typical waterfall project).
- Requirements should be small enough to make sense in the context
- Acceptance criteria (stories are generally used for acceptance criteria) should not be duplicated or overlapping from different stories
However, doing this can be very difficult and can only be achieved with strong communication between Development/Business Analysts/Quality Assurance.
Testable: The testability aspect of the story requires scanning through the story to see what needs to be done in order to test the story. These factors are generally:
- Finding hidden requirements
- Environment
- Test data
- Dependency on other requirement
Getting these details early helps the story to be prioritized accordingly in the backlog, and allows smooth execution of the story in the iteration.
QA testers also participate in the iteration planning meeting to give a testing perspective so the team can come up with a developer estimate. Participating in the iteration planning is a big role, as some of the implicit requirements are escalated by QA testers.
QA activities In the iteration
Once QA testers are happy with the acceptance criteria of the story, they can help define the acceptance tests for the story. Acceptance tests are requirements in terms of tests that can be executed in order to understand what is expected from the requirements. These acceptance tests are generally automated and used to drive development. Acceptance tests should not cover too many corner case scenarios as this would create unnecessary delay and could end up producing too many similar automated tests.
People often fail to understand that acceptance testing in Agile projects is different from traditional projects. Unlike traditional projects where acceptance testing happens at the end of the software lifecycle, in Agile projects acceptance testing is performed before the software is delivered. Acceptance tests also tend to be automated so they can run as regression tests.
Automated testing is very important for any Agile project. Frequent builds require short feedback cycles, hence regression testing must be quick and accurate. It needs to be noted that automated testing in Agile is different from traditional automated testing. In Agile projects, automated testing is practiced by all levels – developers, QA testers and business analysts. Involvement from everyone increases the relevance of the tests and often helps identify the right tests. But, this does not mean that everyone needs to be writing test code.
It’s always been debatable who owns test automation in Agile projects. For me, it’s more of a responsibility than a role. In my experience, the most effective test automation is achieved when developers and QA testers work together.
Use automation purposefully
Automation in Agile can be quite controversial. Many people try to automate everything and end up in a trap of having a long feedback cycle. Automation is meant to provide early feedback on the latest code, and the key is to identify what is worth automating and what is not.
Every automated test has a cost associated with it. The cost of automation should be compared to the cost of not running it. The questions that need to be asked are: what if a test is not automated? What will be lost and what would be the cost of fixing the stuff around the code for which we are losing the coverage? Is it cheaper to test manually? The answers may not always be as straight forward as finding the value of a test. It’s often a contextual decision depending on the size of the project and the number of people involved. In other words, a longer feedback cycle equals more people contributing time in getting instant feedback.
The typical QA activities in the iteration are to continuously measure the quality of the software. QA testers participate in the story handover to the developers. This helps them understand the testing requirements of the story so they are enabled for Test Driven Development (TDD). Also, handing over the acceptance tests and making developers understand the testability aspect of the story often helps catch common defects. These activities require a high level of communication between the developer and business analysts to clarify things on the fly and make sure the product is built right first time.
QA testers can help resolve issues beforehand by actively participating in the overall process. They may even pair up with developers to work on the story or tests for the story in order to get a better understanding of the requirements. It’s also imperative that once the story is delivered, it is tested properly, in a proper environment. Once the QA testers are happy with the stories/requirements, they sign off on that piece of work and hand it over for further testing.
It is also important to think beyond the written requirements and experiment with exploratory testing to execute ‘out of the box’ scenarios and undertake negative testing to help assure the software is robust. Exploratory testing is not at all about executing pre-defined testing scenarios, it is the art of exploring the software beyond test cases and at the same time keeping the focus around specific requirements.
I have attempted to cover most aspects of the basic activities of a QA tester in an Agile project. However, the role is not limited to these activities and QA testers should play more of a collaborative role when possible.
Pankaj Nakhat
