What is Ethereum Smart Contract Testing? What are its challenges? If you’re new to Smart Contract Testing, this in-depth guide will prepare you on how to test smart contracts successfully.
Blockchain stands out due to its enormous implications. Everyone has heard of it, but few people know what the ramifications are for testers or how to test a smart contract. For brevity, this article focuses on introducing blockchain, smart contracts, and how a tester can automate Ethereum smart contract testing.
Smart Contracts
In the following sections, this article dives into smart contract testing. The goal of this article is to give you an understanding on smart contracts, and how to test them.
Topics explained are:
- What is an Ethereum Smart Contracts?
- Ethereum Smart Contract Examples
- Ethereum Smart Contract Testing
- Smart Contract Testing Challenges
- Requirements and Strategy of Testing a Smart Contract
- A Smart Contract Under Test
If you are still new to Blockchain, our infographic The 411 on Blockchain and Smart Contracts which contains a wealth of resources for you to learn about the fundamentals of blockchain.
What is an Ethereum Smart Contract?
Ethereum, one of the most popular public blockchains, made its premiere in 2016. Ethereum was hailed as a revolutionary blockchain infrastructure because of its improvements, including the enhanced consensus mechanism Proof-of-Work and other security tighten-ups. Most notably, Ethereum allows you to add user-defined rules for validating and creating new transactions or blockchain data such as balance, owner, or friend list. This capability is the corner stone of building smart contracts.
Proposed by Nick Szabo, “a smart contract is a computer protocol intended to digitally facilitate, verify, or enforce the negotiation or performance of a contract. Smart contracts allow the performance of credible transactions without third parties. These transactions are trackable and irreversible” (http://www.fon.hum.uva). The aim of smart contracts is to provide security that is superior to traditional contract law and to reduce other transactions costs associated with contracting. With present implementations, based on blockchain, “smart contract” is mostly used more specifically in the sense of general purpose computation…used by the Ethereum Foundation…a smart contract is not necessarily related to the classical concept of a contract, but can be any kind of computer program.” (Wikipedia)
We define an Ethereum Smart Contract simply as an account with attached information and rules. This internal data can be changed passively only by its data-governing rules. User transactions sending to this account to change the internal data will trigger these rules.
Here is one interesting point if you haven’t noticed: since a smart contract is simply an ETH account, one smart contract can call other contracts, just like programming objects have the ability to call other programming objects.
The below illustration visualizes the relationships between a smart contract and related concepts.
It’s crucial to keep in mind that in contrast to the data, the rules are immutable during the contract lifecycle. In the Ethereum world, smart contracts are developed in Solidity—a dedicated language compiled and executed within an Ethereum Virtual Machine (EVM). The examples used in this article are written in Solidity.
Ethereum Smart Contract Examples
Ethereum Native Smart Contract
We’d easily recognize that the obvious Ethereum Smart Contract example is the native smart contract of the Ethereum blockchain itself. You may call it the ‘capo dei capi’ of all smart contracts. This contract’s data contains all user ETH balances.
The rule to execute is very simple: “if you want to send 10 ETHs, you better have more than 10 ETHs”. This rule is triggered by any transactions attempting to modify the balances of a certain sender and a certain receiver.
Token Contracts
A second famous example is a token contract which is created to distribute and manage user tokens; this is not to be confused with an Initial Coin Offering (ICO). Prospective service providers create a certain amount of tokens (say 1,000,000 tokens) with the promise that token buyers will be able to exchange these tokens for their to-be-developed services (e.g. an affordable DNA-based cancer diagnostics).
The contracts work independently as a central bank of ‘token’ currencies. They manage user balances and transactions while transparently voiding the problem of double-spending.
Lottery Contracts
Another example of Ethereum Smart Contracts is the lottery contracts which simulate a real lucky draw. With a contract-based lottery, no bad actors can influence the randomization or alter the picking results. Particularly, the contract is an Ethereum account which can receive ETH from lottery ticket buyers around the world. You can buy 10,000 lottery tickets for let’s say 0.0015 ETH per ticket (totaled around $2,951 at the time of writing). The smart contract then registers you as a ticket holder in a contribution list.
At a predefined moment, a special transaction will be sent to the lottery contract. The lottery contract will randomly pick a lucky contributor and reward him with an ETH prize, say 0.08 ETH per lottery ticket. Since you bought 10,000 lottery tickets, you’ll receive around $158,178. This prize is subjective to some commissions and fees.
Ethereum Smart Contract Testing
Smart Contract Testing Challenges
The biggest challenge in blockchain and smart contract testing is to design and perform consensus scenarios. Brainstorming the scenes of multiple factors decentralizing across networks are never enough and orchestrating facilities to simulate such contexts aren’t easy. It would be costly in executing a test case to ensure the consensus works properly. For example, validating a byzantine fault tolerance algorithm in a decentralized network requires several to hundreds of machines and networks.
Requirements and Strategy of Testing A Smart Contract
Requirements of Testing a Smart Contract
With rules to validate and change the internal data of Ethereum accounts, smart contracts could be considered as programs running on the blockchain infrastructure. As a result, functional tests and non-functional tests both are required for validating and correcting the contract’s behaviors before being released it into the wild.
Non-Functional Testing
Regarding non-functional testing, both security and performance of the contract under test must be considered. While performance testing ensures an optimal transaction fee for the contract behaviors, security testing ensures that our smart contracts don’t expose common vulnerabilities such as overflows/underflows, visibility/delegate call, and reentrancy.
The example below shows an untested contract that is vulnerable to cheating. In the parallel/decentralized world, no one can ensure that the operations are executed in the predefined order. The seller of ProductX could be cheated by a malevolent buyer if the buyer intentionally changes the order of transaction execution.
What happens if, while the seller is still processing the price-updating transaction (he wants to increase it from $100 to $150), a malevolent buyer pushes a high-gas transaction to buy all in-stock products of the seller at price $100 (assuming the buyer got insider information about the price hike) and then resell it to the seller at $150? That scenario is possible if both methods are called in parallel. It’s formally called “front-running”.
Functional Testing
In functional testing, all business rules or requirements should be verified in various cases including valid/invalid arguments, boundary values, and argument combinations. Additionally, the fact that a contract could call or trigger other smart contracts to perform its business leads to the requirement of many types of ‘mock’ contracts or oracles for better integration tests. Also, the interfaces between the contract under test and user applications (front-end) should also comply with industry standards.
Let’s take a look at the below smart contract example. It allows you to deposit ETH to a list of balances. It also lets you withdraw your deposited ETH as long as your withdraw is smaller than your balance. All well and good, except there’s a vulnerability that we haven’t covered. Could you spot it? We’ll dive into a test for this contract in the following section.
Smart Contract Testing Strategy
You can’t test blockchain successfully/efficiently without Test Automation. Due to the combinatorial explosion of many inputs and environmental factors in the test case preconditions, manually testing a contract is not only too hard, it’s inefficient.
For example, creating a will-be-late transaction (with a low gas limit) and then an in-front transaction (with tons of gas) to simulate a ‘front running’ scenario seems to be impossible for manual testers, especially in regression testing.
Regarding the testing types, the smart contract should first pass the unit testing gate and integration testing gate before turning to performance/system testing in the ‘test net’ all before it hits the production stage.
As a best practice, unit, and integration tests should be done in the local infrastructure (e.g. Ganache) for every code commit. While unit tests just need to cover all contract methods, integration tests should cover all kinds of oracles and front-end applications.
On the other hand, performance tests should analyze contract code statically or dynamically and measure the consumed gas in order to generate a performance assessment. Finally, the system test must cover all weird scenarios (i.e. common security vulnerabilities) to ensure its resistance against most known attacks in the decentralizing context.
Not one should be mentioned on smart contract testing tools—a private infrastructure is mandatory for the contract development and testing. We’d recommend Ganache and Geth as excellent candidates in building up such testing infrastructures. Regarding testing frameworks, Truffle is the most popular. Meanwhile, Populus is a good alternative for Python developers.
A Smart Contract Under Test
To illustrate the importance of testing smart contracts, we’ve written a smart contract test example below. This test targets the aforementioned smart contract in our previous section.
This test was designed to detect the “re-entrance” vulnerability. We want to see what happens if the sender’s account is actually a smart contract and inside its payable method, the sender can maliciously call the ‘withdraw’ method multiple times.
Since the smart contract under test only checks that you have more ETH in your balance than the amount you want to withdraw once, that IF statement in smart contract will surely pass when you call “withdraw” the second time within the payable method. As a result, you can withdraw as much ETH as you want until the smart contract runs out of money.
Once we ran the said test, we got the following result shown below.
Since we got a total of 10 wei in the beginning, we withdraw 7 wei then deposit 3, we’d expect to see the money in our pocket to be 6. However, we now got 9 wei thanks to our clever cheating method. The test has failed. Imagine that we release this vulnerable contract into the Ethereum network, how much money would we lose if we miss such an important test? That’s why it’s always worthwhile to invest in a comprehensive smart testing strategy.
Conclusion
Smart contracts control their internal data run as backend services for a wide variety of DApps. When people talk about blockchain testing, they usually refer to the testing of those smart contracts and their respective front-end applications. Despite the essential features supported by the blockchain infrastructures, we should focus on the functionality, security, and performance of our smart contracts.
Nevertheless, unlike web standard protocols (web services or RESTful) used between web apps or mobile apps and their back-ends, compliance testing of a smart contract is a must for the long-term product compatibilities. We’ve covered everything you need to know on how to test smart contracts. It’s clear that blockchain and Ethereum Smart Contract testing stand out in emerging technologies. If you’re looking to create your own blockchain—then talk to us, our company MoWeDe is offering blockchain Dapp Services, and, LogiGear is here to help with your smart contract testing needs.
References
Anastasia Mavridou, Aron Laszka, Designing Secure Ethereum Smart Contracts: A Finite State Machine Based Approach, FC 2018
Georgios Konstantopoulos, How to Secure Your Smart Contracts, Medium 2018. https://medium.com
Matthew Di Ferrante, Mechanism Design Security in Smart Contracts, Medium 2018. https://medium.com/@matthewdif
Loi Luu, Duc-Hiep Chu, Hrishi Olickel, Prateek Saxena, Aquinas Hobor, Making Smart Contracts Smarter, Proceedings of the 23rd ACM SIGSAC Conference on Computer and Communications Security (CCS), ACM (October 2016) 254-269.
Szabo, Nick. “Smart Contracts.” Literature Review on Reaction Time, www.fon.hum.uva.
