Spotlight Interview with Jonathan Rasmusson

Author of The Agile Warrior, Rasmusson answers questions from LogiGear’s testing staff about test automation in agile projects.

As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. Developer by trade, Jonathan Rasmusson engages with LogiGear’s inquisitive staff to give us insight on how he addresses various agile test automation challenges. An excerpt from chapter seven of his book, titled “Planning Agile Projects’ Estimation – The Fine Art of Guessing” will be included in next month’s issue focused on Test Process Improvement. More excerpts can be found at The Pragmatic Bookshelf.

1. How does test case automation fit in among different sprints? How can you automate testing on new functionality during a sprint? How do you make decisions during a spring to automate a test or not?

Generally when creating software in agile we like to do our testing in the same iteration (or sprint) that the story is developed in–this ensures that what we’ve produced works. It’s more efficient than abandoning the story and picking it up again in the next iteration to complete the testing. So generally, it fits in towards the end of the iteration.

In having said that, getting all your automated testing done in a single iteration isn’t always possible. For example, at the start of a new project we may not have a mature automated testing strategy in place. It’s something we’ll probably discover and figure out as we go. But at a minimum, the unit tests should be there along with whatever else we can get going early in the project.

Deciding which tests to automate (and which not to) is a decision every team faces on their project. Unit tests are cheap and easy – every project should be writing a ton of these. Integration tests are a bit more complex and involve making trade-offs between costs, scalability and fragility. For example a nice end-to-end integration or smoke screen test can be extremely valuable for a team that just wants to know whether the latest push of their software into a development environment is working or not.

Automated GUI tests on the other hand can be extremely fragile and problematic. It’s not that you should never do them, you just have to be careful as you don’t want to waste a lot of teams on fixing a changing UI test because the GUI hasn’t settled down.

2. What type of test automation should we do in agile projects?

At a minimum, teams should be doing a lot of unit testing. These tests developers write to ensure that their software (at the method and class level) works as expected. Every team should be writing many of these.

After that, it really depends on the nature of your project and your team. If your system is primarily back-office, integration tests (tests verifying all your various subsystems) that are hooked up correctly can be extremely valuable. If you have a GUI / web front end, you may want some high-level smoke screen tests using a tool like Selenium or something.

3. What are all the challenges for a test automation team in agile?

There are a couple of challenges with test automation on any team. One is having the courage and discipline to do it. Automated testing is hard work. It means not moving forward and delivering new functionality. It’s hard for some people to stop, make the investment, and automate.

The second challenge of course is knowing what to automate. Teams need to figure and discover on each project which tests are going to pay off and which are going to give a lot of bang for your buck. We know we can’t automated everything (that would be too expensive) but we also know that not automating is going to kill us. So we just need to be flexible, try lots of different tests, keep what works, and drop anything that gets in the way.

4. About test automation script maintenance, should it be considered as a separate user story/ sprint?

No. While it’s OK for teams to reserve 10-20% of an iteration for general bug fixing, refactoring, or test script maintenance, I wouldn’t recommend turning it into a story. Why? Because it’s not something our customers would find very interesting. As much as we can, I prefer reserving stories for pieces of business functionality our customers would get excited about. Testing, refactoring, and general maintenance are generally things they don’t care about.

5. What is the best time to document testing in agile projects, at the beginning from just the user story or at the end after the function is complete and fully understood?

While I like the idea of automating tests at the start of the sprint, I have never been able to pull it off in practice. I think you could do it if you had a really good idea of what was required and a team with the people, skillset, and desire to do it. I just haven’t seen it yet. That would have to be a pretty mature and skilled team.

Also, quite often on new stories we don’t know what the end result is going to look like. It’s often not until the story is nearly complete, and we’ve gone through many iterations with the customer that we (including the customer) truly understand what is required─that makes automating ahead of time hard.

6. If other groups (marketing, development, project management) have cut back on documentation and measurement to become “leaner,” should testers still be required to write test cases, show traceability to user stories, write a test plan and produce a dashboard of metrics on test cases and bug counts?

I don’t see the connection. If one group in the organization is doing something different to become more lean or efficient– good for them. That’s to be applauded. Conversely if other groups aren’t being efficient or lean, that shouldn’t stop us from doing whatever it is we feel we need to do to serve our customers better.

So if writing test cases, tracking traceability, and extensive test plans aren’t adding any value, they should be dropped regardless of what others in the organization are doing.

7. If my team does not do a hardening sprint, when do we run a full regression test suite?

That’s something each team needs to decide for themselves. While I am generally not a fan of hardening sprints, they are sometimes required. The team just takes on a lot of technical debt, or that they start slowing down because they haven’t invested enough in their build and automation tools, so taking a sprint to firm these things up can help.

One obvious time to do a full on regression test is right before any major product release, e.g. a User Acceptance Testing (UAT). That’s something you’ll obviously want to do before going live. Having said that, I believe the goal of every agile project should be to make UAT redundant or a non-event. In other words, the team does such a good job with it’s automated and in house iteration testing, that when UAT comes around, no bugs are found; they’re not there. The team has been building and releasing software of such a high quality that it is no longer required.

I am not saying drop UAT. You’ve got to earn the right to do that. But every day more and more teams are making UAT a non essential activity instead of spending that time and money on more valuable things (improving and adding more features to the existing software).

8. I am on an agile team that does offshore testing. Often, I do not attend the daily stand-up due to time zones. Can agile be successful this way? What are some suggestions to help me be successful in my testing?

I am no expert in offshore testing. Not really my specialty. But I do find it interesting, that whenever projects get into trouble, they bring everyone together and put them in the same room. This is often the single greatest thing you can do to improve the productivity of your team. 

 

Jonathan Rasmusson

As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. Developer by trade, Jonathan Rasmusson engages with LogiGear’s inquisitive staff to give us insight on how he addresses various agile test automation challenges. An excerpt from chapter seven of his book, titled “Planning Agile Projects’ Estimation – The Fine Art of Guessing” will be included in next month’s issue focused on Test Process Improvement. More excerpts can be found at The Pragmatic Bookshelf.
Jonathan Rasmusson
As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together.

The Related Post

Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part One of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile” LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations ...
Application development and delivery teams are under constant pressure to release quality features as quickly as possible. CIOs rate delivering applications faster, with higher quality and with strong control on application development as their key priorities. What’s more, supporting this type of agile environment is particularly complex to IT teams that are also tasked with supporting ...
Mark Levison has over twenty years experience in the IT industry, working as a developer, manager, technical lead, architect, and consultant. He discovered Agile in 2001 and is now a Certified Scrum Trainer and Agile Coach with Agile Pain Relief Consulting. Levison has introduced Scrum, Lean and other Agile methods to a number of organizations and coaches from ...
How to fit automated testing into scrum, and keep testers in sync with other teams One of the benefits of the approaches of agile projects is their friendliness towards testing. The testing activities, and the testers with it, are integrated into the teams, and testing and quality are redefined as team responsibilities. Automation nowadays is a must-have ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Four of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
Agile Automation Michael Hackett – Senior Vice President – LogiGear Corporation Michael Hackett Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the ...
Testing in Agile Part 1 – INTRODUCTION TO AGILE In case you missed the first part of the series in our last magazine issue from Michael Hackett, Agile’s impact on software development teams is huge. For test teams it can be even more pronounced — and good, especially if your existing projects have been problematic.
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Three of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
Testing is often looked upon by many as an unmanageable, unpredictable, unorganized practice with little structure. It is common to hear questions or complaints from development including: What are test teams doing? Testing takes too long Testers have negative attitudes
Over the years many Agile proponents have come out strongly against offshoring some of the development team, and in particular against having a remote testing team. We made use of not one, but two separate outsourcing providers located in two distant locations. While we had many challenges, what we found was that by starting with ...
“Agile is to software development what poetry is to literature. It is very difficult to do well, very easy to do poorly, and most people don’t understand why a good poem is good and a bad poem isn’t…” – from the web

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe