Letter from the Editor – September 2015

In every year since 2011, we have devoted one edition of our magazine to the topic of mobile testing. In this year’s issue on mobile, we focus on testing from the point of view of the user experience.

Most teams start with UI testing, and it may seem basic — until you look at the importance and uniqueness of UI testing for mobile apps and websites. There’s a lot to understand, but the most important thing is that mobile users aren’t very bug tolerant. Their expectation is flawless performance.

First, the problem of mobile responsiveness. The Wikipedia definition of responsive web design (RWD) is: …”an approach to web design aimed at crafting sites to provide an optimal viewing and interaction experience — easy reading and navigation with a minimum of resizing, panning, and scrolling — across a wide range of devices (from desktop computer monitors to mobile phones).”

Responsiveness testing is key for any mobile software product. That is the ability for your site UI to be accessed by all the defined devices with varying screen sizes and orientations (portrait or landscape). This is different from functionality or even UI testing, since UI testing focuses on the correct behavior of UI controls such as combo boxes, text boxes, buttons, or links. Responsiveness testing examines the layout and look and feel of, for example, a 12.9” iPad vs. a 9.7” iPad, not UI functionality.

Second, what mobile devices do you test? I just had a new client request that I specify what platforms to use for mobile testing. That is a huge question with no easy answer. The mobile world is increasingly fractured. The devices, operating systems and browsers, rather than converging, are becoming increasingly diversified. There are so many manufacturers making devices with varying hardware components. For example, if your app uses geolocation, do you have to test devices’ gyroscopes or accelerometers, gravity sensors, compass sensors and magnetometers? It’s a complicated answer.

So many devices, and so little time. How many devices do you need to test? How and where will you execute your tests? On real devices? Buy a Cloud service? Use emulators? At the UI? At an API? We have been having these problems in mobile testing since it began. The same issues are here today and it is getting ever more complex as the device world diversifies.

These problems would diminish, or go away entirely, with a convergent mobile device market. But not only is the device/hardware market fracturing, the OS world is getting more complicated as well. For the Android OS alone, the various versions of Lollipop, KitKat and Jellybean all have significant usage, with lesser use of Ice Cream Sandwich and Gingerbread. They are definitely not all the same. So for Android, what do you test?

Companies continue to struggle with a mobile testing strategy. There are more questions than answers, and this is not necessarily a bad thing!

As I learned from my first and greatest testing manager, Cem Kamer, “half of our job is education.” Wow.

Bring your test strategy and coverage plans to the team. Educate your team to these complexities. Marketing, POs (product owners), product management, as well as programmers have to be involved in your decision-making. What you test — the hardware, sensors and screen size, and OS combinations — as well as how to test — emulators, cloud, real devices — needs to be knowledgeably and openly discussed and then agreed upon.

In these Agile and Lean days of no or few test plans, someone has to be having these conversations; someone has to be asking these questions. That person is you.

Our last issue of 2015 will be on Test Automation. That issue will also contain the Editorial
Calendar for 2016. Until then, good luck and have fun testing!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

Continuous Testing… what is it? When we first decided to do a magazine issue dedicated to the DevOps practice of Continuous Testing, I joked with someone: “It’s about testing continuously.” And their reply was: “Yeah. What else would it be?” I was joking, but clearly the joke didn’t land. Continuous Testing is about testing continuously, ...
Hello everyone – I’m hoping each one of us is having a great October. This time of the year is always my favorite, with the changing of the seasons, Fall was always my favorite time of year; it signified change and renewal – but I don’t want to digress to much from what’s going on ...
I once consulted for a company to give a week-long course on testing and QA. It was a survey course covering a wide range of topics. I was setting up and chatting with students in the room. One man came over to me and said: “I have been testing for 6 months and I am completely ...
This is our first Trends issue in our 10- year history. Trends are important to help foresee what is on the horizon and coming next.
Testing Embedded systems and testing the Internet of Things could each have their own issue of LogiGear magazine. But these days they are referred to presupposing knowledge of the other, so we thought it would be a good idea to tackle the two together in this issue to give a broad understanding of the landscape ...
How do you test software? How do you validate it? How do you find bugs? These are all good questions anyone on your project team or anyone responsible for customers may ask you. Can you articulate your test strategy─not your test process, but explain your approach to testing? I find that this can be a ...
Testers need to learn their craft and hone in on their skill set. That means building skills, sharpening their tools, and becoming creative detectives. There is no cookie-cutter tester and no best practice. The best circumstance is a fully-skilled, aggressive tester mixed with curiosity, nimbleness, and agility.
Automation is a mantra in testing. Anyone associated with software development wants more test automation, but it’s often misunderstood. People who do test automation know how difficult it can be. But some people do not understand that automation is code, and that it needs to have architecture and design just like production code. They do ...
I remember the times when test teams sat in their own area and we were not allowed to “bother” developers.
There is a growing software development dynamic of teams without Testers. When I first went into Software Quality, I learned one thing right away: My role was user advocate. My main job was to find bugs. This is the Lean principle called Amplified Learning. We learn about behavior by testing. Even then, validation was not ...
Everything is mobile. What else can we say? Everything. If your product or service is currently not, it will be very soon. As Apple says: “There’s an app for that.” There is an app for everything. The race for mobile apps has consumed the software development world. I did a few projects at Palm Computing in the ...
As part of my work, I spend a lot of time at client’s sites and talk to various software development organizations. I am beginning to see a problem arise regarding Test Automation. There is too much automation! Surprised? While there are still many teams struggling to make progress with Test Automation, many teams have been doing ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe