Are Testers’ Ethnographic Researchers?

People who follow me on twitter or via my blog might be aware that I have a wide range of interests in areas outside my normal testing job. I like to research and learn different things, especially psychology and see if it may benefit and improve my skills and approaches during my normal testing job. One area I have being looking at for awhile is the social science of ethnography. The approaches used when carrying out research appears to have many similarities to software testing and I feel we could benefit and maybe improve our testing skills by examining ethnography.

In my opinion, there are two areas in which we can learn from ethnography:

  • To improve our understanding of users and how they differ by using ethnographic methods
  • Use ethnographic methods to test software in an exploratory way.

I should start by explaining what my understanding of ethnography is according to anthropologist Brian Hoey. He defines ethnography as “equated with virtually any qualitative research project…where the intent is to provide a detailed, in-depth description of everyday life and practice.” He continues to state that an ethnographer is more than documenting events and details of everyday life but that the ethnographer attempts to explain how such occurrences represents the cultural construction an individual or community lives in.

The problem with trying to describe and define ethnography is that it has wide and varied meanings. To me, it is a branch of the study of humanity (anthropology) in which the researcher actively gets involved and participates with the study group rather than just sitting back and observing. The reporting is using qualitative (words) measurements rather than rely on quantitative (numbers) measurements.

One of the key factors when approaching ethnographic research is to be aware that participation, rather than just observation, is one of the keys to the approach. Does this not sound familiar to testing, especially exploratory testing? Actively using the software under test to find out about its characteristics and behavior are similar to an ethnographic researcher living within a community and participating with that community to learn about its beliefs and characteristics. There appears to be very close parallels between ethnographic research and exploratory testing. Wikipedia states that “one of the most common methods for collecting data in an ethnographic study is direct, first-hand observation of daily participation.”

How similar is that to testing software?

Another approach within ethnography is the use of grounded theory to explain the results from the participation. This is when the data is used to provide theories about the data. This is different from grand theory in which the theory is defined without the use of real life examples and therefore has a danger of not fitting the actual data gathered afterwards. Is this similar to scripted and exploratory, grand theory vs. grounded theory?

Grounded theory is a constantly evolving set of conclusions that can continue indefinitely based upon the changing data being obtained by the ethnographic researcher. One of the questions that are asked about ethnographic research is: When does this process end?

One answer is: never! Clearly, the process described above could continue indefinitely. Grounded theory doesn’t have a clearly demarcated point for ending a study. Essentially, the project ends when the researcher decides to quit. (http://www.socialresearchmethods.net/kb/qualapp.php) How similar is this to testing? When do we stop testing?

Many articles have been written on this subject and mainly we stop when we can learn nothing new, no time or ran out of money. See Michael Bolton’s article where he lists twelve heuristics for test efficiency methods. I feel that ethnographic research stops because of similar reasons.

One interesting section I saw within the Wikipedia article on ethnography was about the process of ethnographic research in which to aid the researcher areas were split and the research asked questions:

  • Substantive Contribution: “Does the piece contribute to our understanding of social-life?”
  • Aesthetic Merit: “Does this piece succeed aesthetically?”
  • Reflexivity: “How did the author come to write this text…Is there adequate self-awareness and self-exposure for the reader to make judgements about the point of view?”
  • Impact: “Does this affect me? Emotionally? Intellectually?” Does it move me?
  • Expresses a Reality: “Does it seem ‘true’ – a credible account of a cultural, social, individual or communal sense of the ‘real’?”

I thought about this and started to change the context to be about software testing:

  • Substantive Contribution: “Does the testing carried out contribute to our understanding of the software?”
  • Aesthetic Merit: “Does the software succeed aesthetically?” Is it suitable for the end user?
  • Reflexivity: “How did the author come to write this test…Is there adequate self-awareness and self-exposure for the reader to make judgements about the point of view?”
  • Impact: “Does this affect me? Emotionally? Intellectually?” Does it move me?
  • Expresses a Reality: “Does it seem ‘true’ – a credible account of a requirement’?”

By doing this I found I suddenly had a set of heuristics to measure against the software testing that has been carried out, yet again more similarities between the two crafts.

Another area in which ethnographic research can be useful to software testing is when you need to test software that has a lot of UI interactions. Using the methods of ethnography a tester could go visit the users and observe and participate in their daily routine to find out the common tasks carried out and what oddities are seen.

The oddities are the things of greatest interest since these are the things that would not normally be planned for and without active participation with the users would not normally be uncovered until it is too late.

There are many studies being carried out to determine if ethnographic research should be used when designing software system. However, my concern with this is that it appears to be stuck in the design up front way of working which is not a flexible iterative approach. In my view it is easier, quicker and cheaper to ensure that testers use ethnographic methods when testing to ensure the design is suitable for users or even better get the users involved earlier and observe them earlier.

The more I have delved into the study of ethnography the more and more I have seen similar patterns to software testing. This makes me aware that software testing is not solely a hard science but a craft that encompasses many disciplines outside of the typical number crunching and algorithm creating world of software development. Within the testing profession we need to look outside of the box and find approaches, methods, structures that can improve the discipline. To ensure our craft grows we need to ensure we do not narrow out field of vision or thought.


To read original post please visit http://steveo1967.blogspot.com/2011/01/are-testers-ethnographic-researchers.html

 

John Stevenson

Tweet John Stevenson: @steveo1967
“The opinions expressed in this blog are my own views and not those of any company I have or do work for”Having been involved in testing for over 20 years and within the IT industry for more than 24 years I am still surprised with how exciting I find it and how much I continue to learn about things that are new.I have a passion for learning and love to learn about new things. I have an interest in many things such as social science, psychology, photography, and gardening. I keep involved within the testing community and write a testing blog (www.steveo1967.blogspot.com) and can be found regularly tweeting (@steveo1967).
John Stevenson
I have a passion for learning and love to learn about new things. I have an interest in many things such as social science, psychology, photography, and gardening.

The Related Post

Most have probably heard the expression ‘less is more‘, or know of the ‘keep it simple and stupid‘ principle. These are general and well-accepted principles for design and architecture in general, and something that any software architect should aspire to. Similarly, Richard P. Gabriel (a major figure in the world of Lisp programming language, accomplished poet, and currently ...
Introduction Many companies have come to realize that software testing is much more than a task that happens at the end of a software development cycle. They have come to understand that software testing is a strategic imperative and a discipline that can have a substantial impact on the success of an organization that develops ...
At VISTACON 2011, Harry sat down with LogiGear Sr. VP, Michael Hackett, to discuss various training methodologies. Harry Robinson Harry Robinson is a Principal Software Design Engineer in Test (SDET) for Microsoft’s Bing team, with over twenty years of software development and testing experience at AT&T Bell Labs, HP, Microsoft, and Google, as well as ...
VISTACON 2010 – Keynote: The future of testing THE FUTURE OF TESTING BJ Rollison – Test Architect at Microsoft VISTACON 2010 – Keynote   BJ Rollison, Software Test Architect for Microsoft. Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches ...
Reducing the pester of duplications in bug reporting. Both software Developers and Testers need to be able to clearly identify any ‘Bug’, via the ‘Title’ used for the ‘Bug Report’.
Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests. Test design is the single biggest contributor to success in software testing and its also ...
Introduction Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development that finally delivers on the promise of Software Testing. For more details on the evolution of Software Testing and Software Testing 3.0 see: The Early Evolution of Software Testing Software Testing ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction When thinking of the types of Software Testing, many mistakenly equate the mechanism by which the testing is performed with types of Software Testing. The mechanism simply refers to whether you are using Manual or ...
Introduction Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development that finally delivers on the promise of software testing. For more details on the evolution of software testing and Software Testing 3.0 see: Software Testing 3.0: Delivering on the Promise of ...
Jeff Offutt – Professor of Software Engineering in the Volgenau School of Information Technology at George Mason University – homepage – and editor-in-chief of Wiley’s journal of Software Testing, Verification and Reliability, LogiGear: How did you get into software testing? What do you find interesting about it? Professor Offutt: When I started college I didn’t ...
From cross-device testing, to regression testing, to load testing, to data-driven testing, check out the types of testing that are suitable for Test Automation. Scene: Interior QA Department. Engineering is preparing for a final product launch with a deadline that is 12 weeks away. In 6 weeks, there will be a 1 week quality gate, ...
Internet-based per-use service models are turning things upside down in the software development industry, prompting rapid expansion in the development of some products and measurable reduction in others. (Gartner, August 2008) This global transition toward computing “in the Cloud” introduces a whole new level of challenge when it comes to software testing.

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe