Much has been written over the past few years about SMAC—Social, Mobile, Analytics and the Cloud—as the new platform with exponential growth. In this article we will answer the question: what is this and what does it mean to software testing?
What is SMAC?
S—Testing Social
M—Testing Mobile
A—Testing Analytics, or Using Analytics to test
C—Testing the cloud, or Testing in the cloud, or testing using the Cloud
Not only as individual parts of the system but also testing the integrated product or service and adapting that testing based on the output of analytics based on real usage as well as the prediction of use.
To me, SMAC is the natural evolution of mobile as a platform. It is the convergence of these technologies, also called the SMAC stack. That is leveraging of each other to build and develop systems and products with great functionality, and deploy them very quickly into the marketplace.
The most common phrase people are using to describe the outcome of SMAC is “greater than the sum of its parts.”
SMAC is also coupled with the Internet of Things. Since connected, smart devices are flooding into the market, often controlled by mobile devices and data from these devices is often part of the data collected and stored in the cloud for analytics.
This technology use is exploding and getting bigger everyday. If you don’t realize how big and mobile the world has become, this article is not for you. Mobile and web services use is growing exponentially every day, making it pointless to toss out numbers and percentages of just how much these technologies are being used, because these numbers get outdated as soon as they are published.
Evolving as a Platform
Not too long ago, the desktop expanded to a web server, application server, database, varieties of web browsers—that was a big jump for teams whose testing had been restricted to desktops. New environments, new services, new technologies to test.
Today, mobile has evolved into a platform. From concerns about iOS versus Android and all the varieties of android, Mobile is a platform.
It’s Social. Mobile uses social media APIs or web services or Web APIs, or even just using any web service in larger and larger importance every day.
A common example of the IoT has been the smart home, managing your energy usage remotely from your mobile device with the energy company collecting user and usage data, coupling that into big data with weather/temperature data. This data captured from a variety of sources using APIs and web services and smart devices and put it in a data warehouse in the cloud. Obviously, the data has to be kept safe, it has to be ready when systems or people need it and the cloud databases need to respond well and quickly.
The product will do some analytics to make sense of the data. And, as an example broadcast a notification of a “brown out” (reduced energy available or shut down to certain areas during peak usage usually during the hottest time of the year), and is sent out over email and social media, perhaps both Facebook and Twitter.
How many places do we need to inject testing in this example?
We could describe examples in consumer and corporate or enterprise app; banking, finance, real estate, retail, sales and marketing. Banks are now accepting payments through Facebook and Twitter. LinkedIn is being integrated extensively into secure enterprise apps. For people who think SMAC is only consumer apps—you are wrong—the application of this stack is far reaching. Mobile access to corporate internal assets—such as CRM is now taken for granted.
But for us, in this article, let’s focus on the testing aspects.
Terms and tests
Social is most often referred to as the huge social media players but also includes product integration with the massive number of web service APIs such as integration with Amazon, Pandora, eBay and thousands more. The most common use of social media in SMAC might be the most straightforward to test: real-time engagement and feedback. Correct integration of the various social apps enables these.
Mobile is the device, all the platforms and hardware and applications that goes along with it, cameras, geolocation and location services, example, features unique to Wi-Fi versus 3 or 4 G. Part of your mobile strategy, for example must include: cross-platform, responsive, location aware, memory, CPU and battery usage.
Analytics is the most complex, and probably newest to most testers. This is testing the gathering, storage, manipulation, analysis, and use of data for a variety of purposes such as learning and optimizing the user experience or predicting behavior.
There are many, many places that need testing in this aspect of data warehousing—from the algorithms and system availability to performance.
Cloud, simply, is testing the cloud infrastructure the product uses. It’s like testing a database server as a remote location for all the things you already know to test here, such as; data integrity, performance, security, fail-over. Databases—with and without “big data” attributes need their own sets of tests.
Obviously, functionality has to be tested, bugs need to be found, and User Stories need to be validated at each place.
The Central theme of SMAC is that the Integration’s the thing.
The Stack presents its own issues.
According to Admonsters when people refer to “the Stack” they mean that the technology stack is a set of components or layers in software offering that provides broad functionality. Microsoft Office is an easy example. The combination of Word, Excel, PowerPoint and Exchange improves personal productivity for billions of people.
What this means to testing holistically, is that you may have a mobile test strategy; you may have a web service test strategy, etc. But putting these four technologies together into a stack makes them significantly more useful to your company’s business, and also significantly more difficult to test—even if only from a scale, skill, and tool perspective.
The SMAC test strategy focuses on security, performance, integration and usability. All these technologies get integrated into one platform while needing seamless usability.
Testing
There will be various interfaces to test:
First, the customer interface: the functionality, usability, performance. This is pretty well understood.
The IoT device interfaces. Testing embedded systems is also well understood but new to many testers.
There may also be secondary customer, internal customer, alternative users, partners who may be providing or consuming data using different interfaces—either different user interfaces or APIs into their own systems.
There will often be unique interfaces for internal employees into the social media integration, as well as interfaces into various places in the data warehouse—raw data, processed data, output data, etc.
Focus on Analytics
The most complex and interesting testing for traditional testers may be the testing associated with the analytics.
There are many aspects of testing here. The data itself needs to be tests and there can be large varieties of data as well as large amounts. Sensors can be streaming massive amounts of data and user behavior data to media data to dozens more varieties of data.
One of the goals with analytics today is for designers, developers (programmers and testers) and the system itself to learn about the users and usage from the data. The more the system is used, the more data it collects, the better the system becomes. Many of these SMAC products are built to gather information with the sole purpose of predicting how users will use it and what they will do next. This leads us to a unique position in software development. This business intelligence (BI) is used by us to design tests. We will design tests based on real usage from this data. We will do A/B testing where we may have the system designed so we can vary a task or function and measure what users do or prefer as well as build and test varieties of a task or function and swap them in and out based on data collected. This BI exists to impact our test case coverage, automation, and risk decisions.
Things you do with data: collect it, perhaps clean it up, store it, search it, mine it, analyze it, send it somewhere for you to use, give or sell it to someone else. This needs testing at every step.
Testing the cloud
Cloud testing, in many cases, is similar to traditional server testing. See the evolving as a platform section for the type of tests it includes.
Automation implications
There are a few special impacts to test automation. You may need to expand your suite of tools to include tools that test the constituent parts as well as end-to-end workflows and paths. Not many automation tools can do mobile, web service, desktop UI (for data warehouse tests), SQL tests and even run in the cloud. Finding the right test automation tool or tools is important to learn more about the criteria you should use; you can refer to this article that I wrote on test tool selection.
SMAC Development and DevOps
Combined with speed of integrating web service apps and functionality, using cloud infrastructure and the speed of ease and speed of deployment—for most teams, SMAC systems can be built and deployed quite rapidly. This puts an added emphasis on greater team collaboration and early, shift left testing as well as effective test automation. Regardless of doing the DevOps practice of Continuous Testing or not—be sure to leverage technology and take advantage of virtualization for test environments and data.
Also, built into SMAC is the data collection and analytics commonly used in the Continuous Monitoring practices in DevOps.
Skills
As you have seen in this article, there is a wide assortment of skills—many new skills that a test engineer working on the SMAC stack will need proficiency in. For many teams this will require a big upgrade in skills and technology. See our Leader’s Pulse article for getting your teams these skills.
In addition to the technical skills and understanding the platform for better test design, there are some testing skills that need to be especially strong: defect analysis and isolation, creative test case development, test case development based on analytic data and A/B Testing test case development.
Summary
SMAC is set of technologies that used together, as a stack, are becoming more and more common for sophisticated and rapid product development.
We need a new test strategy to satisfy this demand.
- Social- SOA testing (service oriented architecture with many APIs/web services).
- Mobile- we have been doing mobile testing a few years. The platforms have matured, the tools have matured, and the practices are more common.
- Analytics- this may be new to many testers. The impact of so much data on our testing is a new frontier in our work.
- Cloud- as a test platform or a platform to test may be new to many testers as well.
Over the next 2 issues, we will be publishing articles on testing SMAC testing web services testing mobile testing data, business intelligence, data warehouses and testing in the cloud as well as testing internet of things.
