Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand service in early 2013, because I love ‘real’ books and what an awesome cover!
What an amazing amount of detail in one book. As I was reading it, the test ideas were jumping out of the page; so much so that I wanted a mobile application project to test. I could dazzle the project team with all of ‘my’ wonderful test ideas (while referring to the book at the same time). Don’t worry Jonathan, I would give you credit…eventually.
I like the way the book is structured – the order of chapters maintain a great flow through the entire process of testing mobile applications and devices. From the introduction to mobile technology, through to performance and security. The book will be a definite reference point for me if developing a strategy for mobile application testing in the future!
Chapters 2 and 3 were incredibly valuable! As mentioned above, test ideas jump off the pages and into your brain.
Chapter 2 walks you though and overview of mobile technology and puts a lot of things into perspective; namely just how complex these little devices are. We all know that understanding what you are testing and the technology behind it is important right? Well this chapter goes a long way to helping you with that.
Each mobile device component is listed with examples of how they are used and what could wrong, among other things. Each of these would provide very valuable input to any mobile testing strategy.
Chapter 3 explains many different types of mobile testing tours (if you haven’t heard of these, try here and here as a start pointing). I had done some research into testing tours prior to reading this book; however I had never looked into tours for mobile testing. This chapter provides a significant list of possible tours, accompanied by a significant amount of detail for each one. The User Tour stuck in my head; possibly due to the following:
“…go to a local coffee shop and observe other people as they use the devices. Do they get frustrated? Talk to them (without being creepy) and ask them about their devices, what they like and don’t like. Watch for emotion: are they happy? Angry? Frustrated? Does technology make them feel dumb?”
I wrote about this type of concept here. I think there is immense value to be gained through such a process, and not just at the ‘end’ of the project, but also initially as the application is being designed.
This of course (using Jonathan’s excellent structure) flows directly on to chapter 4: Test Using Different Perspectives. I SLICED UP FUN is then introduced – an awesome mobile testing mnemonic; and awesome thinking tool.
If you are testing mobile devices then I urge you to look at this and use it. Jonathan walks you through this by actually testing an application (of your choice of course). I would encourage all readers of the book to actually pickup a mobile device and test while Jonathan takes you through the process (not the best word to use, but you get it).
Obviously this will take you longer, but I don’t believe this book is a ‘from start to finish’ publication. It’s a set of tools to help you test better and what other ways are there to learn how to test better than actually testing?
None.
One aspect of this chapter that stands out for me is the bug story for each letter of the mnemonic. After Jonathan has finished explaining the letter and providing testing tips he tells a bug story from his experience that relates to the letter. Story telling in a book, who would’ve thought? Seriously though, a lot of testing books don’t tell stories from experience, they preach theory. Of course they too have their place, but experience is REAL, and real adds significant value to my testing.
I would also like to quote a section of the book that I completely agree with…
“Usability is the single most important factor for a mobile app. It is so crucial, I recommend that you study mobile usability works so that you can find and report important usability problems quickly. As I’ve mentioned before, if your app isn’t very usable, people will just delete it. (The deletable offence.) If it annoys them, they may just go on to a social networking service and rant about your app. (A rantable offence.) Or, they may just quietly give your app a poor rating on application stores.”
Now, who said that all testing can be automated?
Chapter 5 takes you through some interesting information in relation to logging and diagnosing bugs. One thing that we as testers seem to be reasonably good at is logging the bugs, however I think there is a lot more work to be done in the diagnosing space! Quoting from the book, which is quoting Cem Kaner…
“The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers. The best tester is the one who gets the most bugs fixed.” -Cem Kaner
Before you jump on me, I agree that it isn’t a tester’s job to fix bugs (if you can, then cool), but we can do enough investigation to make our bug reports so compelling that they are hard to ignore.
There is a lot of information on bug advocacy online, most prominent is Cem Kaner’s paper; Bug Advocacy: How to Win Friends, and Stomp Bugs.
Jonathan does a great job of sharing his thoughts on the subject, and of course putting a mobile testing slant on it.
He focuses on the important information to put in a mobile testing bug report. One very important message that shines through is the value in being exact with your steps to reproduce.
Mobile devices are, well, mobile.
The amount of varying scenarios more than doubles to that of a desktop application where your PC is stationary. It is extremely important to be as exact as possible when recording your steps. A change of location by 10 feet could make a difference when trying to reproduce a bug (read from page 214 to see what I mean).
Chapter 6 is all about the mobile testing strategy. The question that always comes to my mind in relation to mobile testing is what platform to use? It seems it is also the most common question Jonathan gets too!
The first part of this chapter covers all sorts of different elements that you need to consider when trying to answer that question. Check out Jonathan’s four different strategies for choosing which platforms to test with.
In this chapter, Jonathan walks you through a strategy example for a particular platform choice. Even the simplest of choices quickly adds up to a huge amount of possible combinations to be considered. This is a great example of just how complex the mobile testing space can be.
The second part of chapter 6 looks at Jonathan’s ‘general’ process when creating a mobile testing strategy. There are five steps to the process, to which Jonathan explains in great detail. I enjoyed this and will definitely pull this piece out so I can carry it from project to project. In closing Jonathan talks about military strategy and it’s obvious alignment to strategies for almost anything. He runs through OODA, which is brilliant. You’ll need to read the book to see how this fits in.
Guidance and planning are next on the menu.
Chapter 7 begins by highlighting the importance of structure, which stands true for any testing project, let alone in the complex world of mobile. However, as Jonathan points out, don’t overdo it! Mobile projects in general are extremely fast moving, and you don’t want to structure things so much that they slow you down. A happy balance is required.
This is a great chapter focusing on coverage, guidance (on the move as well), checklists, maps, how-to guides, session based testing, and models (including modelling the users)!
This last one was particularly interesting for me as I had experienced it first hand.
Not too long ago I had a problem with my car and the mobile (pun not intended) mechanic paid a visit. All went well until he went to use the new tablet they had just installed in his truck.
It was a new way of recording the job real-time with data being sent straight to the server instead of the old paperwork and manual input. The screen would just not recognise his gestures. After some expletives and getting rather wet in the rain that had just started, I offered to have a look for him (being that I ‘worked’ in IT). Guess what, all my gestures were recognised straight away! Seems my hands were a hell of a lot cleaner than his. Long story short, they now have wet wipes in the truck so they can clean their hands before lodging the jobs.
Another noteworthy mention for this chapter: TAP IT UP. I’ll leave this to you and the book. I will say though, it’s awesome!
I recall when I first started reviewing the book I made a note to myself to make sure Jonathan had written about the differences in native and web applications on mobile devices. Enter chapter 8. Jonathan’s explanation of web application history is very interesting indeed. I was only a youngster when the web came about, so the majority of ‘amazing’ discoveries had been made by the time I started using it. Now I find it fascinating to read about. Jonathan walks us through how web applications work, all while reminding us how their native brothers and sisters differ.
He provides a valuable explanation of all the components that make up web applications, and while reading I had various test ideas pop into my head. All of which were ideas that I could have used on previous projects (for PC web applications). Jonathan does a good job with a pretty dry subject. It’s as though he’s telling a story instead of just throwing theory at us. My patience runs pretty thin when reading theory on certain technologies, but having my mobile device along side of me helped immensely (something that Jonathan encourages throughout the book).
One particularly good section of this chapter is Mobile Web Challenges, and each of the considerations detailed after this. Yes, you guessed it! More test ideas. In case you hadn’t noticed, this book is solid gold for those!
Chapter 9 gives you a great reminder; PS: Don’t Forget about Performance and Security! These two subjects are only an overview, as we all know that each could have their own books (or two, maybe three). It gets started with performance, and Jonathan provides a great analogy for performance benchmarks via the use of cars. Seems he’s a bit of a car nut, not unlike myself, so this was a particularly good analogy for me (and more interesting than most).
Jonathan then calls on some expertise from John Carpenter to describe some performance areas to watch for when testing mobile applications. This is a great little section that I’ll be returning to (like many others). Oh yes, there is mention of a stop watch; as silly as that may sound these days, it’s very useful. I’m actually using one on my current project, just for some baseline metrics. It’s good to know the experts still use them.
Next up we scratch the surface of security testing for mobile applications. Jonathan describes some basics of security testing, however is careful to warn readers that there is a lot more to it. Things like password protection, application input attacks, and technical analysis. A useful highlight into the VERY complex world of mobile applications security testing. Jonathan then closes out the chapter briefly covering some other common test approaches such as accessibility, localization, and duration testing (OK, not so commonly known – duration testing is using the application for extended periods of time which can yield some great results when compared to the usual short bursts of testing).
Chapter 10 captures Jonathan’s final thoughts and a very cool little section called The Seven Deadly Sins of Mobile Apps. Good fun.
But wait, there’s more – appendix 1 and 2! 1 provides a list of, and details, some basic test tools to get you started. I particularly like; Your Brain & Imagination. What better tool is there? 2 lists some really great quick tests, which will also get you off to a great start testing mobile applications.
In total, 446 pages of brilliance! Have I mentioned I’m not a huge fan of reading? Not sure I can get away with saying that anymore, with all the proof reading gigs I have on the go! However, what I’m trying to say is that this is a monster of a book for my standards, but it was easy for me to read. Jonathan has a story telling style of writing that maintained my interest, even through the ‘dry’ subjects. I now have loads of ammunition; I just need to find a mobile testing project to unleash it!
