Avoid Smart Contract Hacks with Fuzz Testing

Quite possibly nothing in web3 is more critical—and difficult to do well—than smart contract testing. For individual developers and small teams, the requisite testing tools are often too expensive and hard to use. Fortunately, a new breed of testing techniques are emerging, ones that are both affordable and accessible.

In my previous article, I talked about one of the most popular techniques: fuzzing. Fuzzing is a dynamic testing technique capable of identifying errors and vulnerabilities that standard tests don’t typically identify. I also talked about how one of the most powerful fuzzing tools—Diligence Fuzzing—just released support for one of the most powerful development frameworks: Foundry. This combination complements your manual audits by providing new vulnerability detection techniques that you can use to avoid costly contract rewrites.

Last time we looked at Diligence and Foundry in detail and how they worked together. This time, we’ll review why the integration is a big deal, and then run through a detailed tutorial on exactly how to implement fuzzing with Diligence Fuzzing and Foundry.

Why Fuzzing Is Important

As even novice dapp developers know, smart contracts running on blockchains tend to be immutable. In other words, if you deploy a contract and someone (including yourself) discovers a security loophole, there’s nothing you can do to prevent a malicious party from exploiting that weakness. So deploying defect-free contracts is incredibly important.

Projects that handle the flow of assets of enormous value, therefore, typically spend thousands of dollars and many months auditing and stress testing their contracts.

Fuzzing is a testing method that can supplement your testing practice and find defects that otherwise would have made it to production. Fuzzing works by inputting millions of invalid, unexpected, or (semi) random data points into your smart contract to cause unexpected behavior and stress test the code. The fuzzing tool identifies vulnerabilities and flags anything that doesn’t meet expectations through the Fuzzing dashboard. It’s an incredible tool for identifying edge cases that could result in those dreaded and expensive smart contract hacks.

Diligence Fuzzing is a Fuzzing as a Service (FaaS) offering by ConsenSys, inspired by coverage-based fuzzing approaches like AFL and libFuzzer. Released in closed beta earlier this year, Diligence Fuzzing has routinely outperformed its counterparts in terms of overall code coverage, time to coverage, and number of bugs found.

Foundry is an open-source framework for Ethereum development—and probably the most popular. With Foundry, you can create smart contracts, deploy, and more.

The ability to use Diligence Fuzzing with Foundry was just recently released, pairing the leading fuzzing tool with the lead development framework. So let’s jump into how easy it is to use the combination to test your Ethereum smart contracts.

Testing an Ethereum Smart Contract with Foundry and Diligence Fuzzing

Step 1: Install Python and Node

The tools we are using in this tutorial are extremely diverse. In order to install Diligence Fuzzing, we will require Python and pip. You can do so by following the instructions available for your OS here.

Check that they’ve been installed correctly by running the following commands:

$ python --version
$ pip --version

Next let’s install Node and npm using the instructions available here and check their version numbers by running:

$ node -v
$ npm -v

Step 2: Create a Foundry Project

Let’s now create a Foundry project!
First, let’s install foundryup, a package that will make installing Foundry a breeze. (Without it, we would have to build Foundry from source using Rust and Cargo.)

$ curl -L https://foundry.paradigm.xyz | bash

Finally, let’s install Foundry by simply running:

$ foundryup

If all goes well, you should see a downloading screen in your terminal that looks something like this:

.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx

 ╔═╗ ╔═╗ ╦ ╦ ╔╗╔ ╔╦╗ ╦═╗ ╦ ╦         Portable and modular toolkit
 ╠╣  ║ ║ ║ ║ ║║║  ║║ ╠╦╝ ╚╦╝    for Ethereum Application Development 
 ╚   ╚═╝ ╚═╝ ╝╚╝ ═╩╝ ╩╚═  ╩                 written in Rust.

.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx

Repo       : https://github.com/foundry-rs/
Book       : https://book.getfoundry.sh/                      
Chat       : https://t.me/foundry_rs/                         
Support    : https://t.me/foundry_support/
Contribute : https://github.com/orgs/foundry-rs/projects/2/

.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx.xOx

foundryup: installing foundry (version nightly, tag nightly-6672134672c8e442684d7d9c51fa8f8717b0f600)
foundryup: downloading latest forge, cast, anvil, and chisel

Out of the tools that foundryup installs for us, the one we’re most interested in is Forge. Using forge, we can initialize a sample Foundry project as follows:

$ forge init fuzz_project

This will create a new folder in your repository called fuzz_project. Open the project in your favorite code editor (like VS Code).

Step 3: Compile Contract and Perform Normal Testing

One of the main reasons Foundry has witnessed a monumental rise in popularity is because of its focus on testing.

Foundry is one of the very few frameworks that allows developers to test Solidity smart contracts using Solidity itself (instead of a JavaScript testing framework like Chai or Mocha).

In the sample project that Foundry has created for us, you will find a sample contract in the src/ folder and a test contract in the test/ folder.

The main contract is extremely basic. It allows you to set a number and increment that number.

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

contract Counter {
    uint256 public number;

    function setNumber(uint256 newNumber) public {
        number = newNumber;
    }

    function increment() public {
        number++;
    }
}

Now, let’s take a look at the corresponding test contract.

// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.13;

import {Test, console2} from "forge-std/Test.sol";
import {Counter} from "../src/Counter.sol";

contract CounterTest is Test {
    Counter public counter;

    function setUp() public {
        counter = new Counter();
        counter.setNumber(0);
    }

    function testIncrement() public {
        counter.increment();
        assertEq(counter.number(), 1);
    }

    function testSetNumber(uint256 x) public {
        counter.setNumber(x);
        assertEq(counter.number(), x);
    }
}

Notice how simple this is in contrast to the huge amount of boilerplate you’re required to write while testing contracts using JS frameworks.
Testing is also extremely simple. All you have to do is run…

$ forge test

…to get output that looks something like this:

[⠢] Compiling...
[⠑] Compiling 22 files with 0.8.21
[⠊] Solc 0.8.21 finished in 3.60s
Compiler run successful!

Running 2 tests for test/Counter.t.sol:CounterTest
[PASS] testIncrement() (gas: 28334)
[PASS] testSetNumber(uint256) (runs: 256, μ: 27398, ~: 28409)
Test result: ok. 2 passed; 0 failed; 0 skipped; finished in 8.32ms
Ran 1 test suites: 2 tests passed, 0 failed, 0 skipped (2 total tests)

Step 4: Install Diligence Fuzzing

In order to perform Diligence Fuzzing in our Foundry project, we will need to install it first. This is extremely simple to do. Run the following command:

$ pip3 install diligence-fuzzing

Just installing the CLI tool is not enough though. In order to perform fuzzing, we will require an API key from ConsenSys.
To obtain this, create a free Diligence Fuzzing account. Next, choose a subscription tier that meets your needs. For this tutorial, the free tier should be more than sufficient.
Once you’ve created an account, you will be redirected to a dashboard that looks something like this:

Next, create an API key here. You can name the key anything you want.

Once the key is created, keep it handy. We will require it in a later step.

Step 5: Perform Diligence Fuzzing

Believe it or not, performing fuzzing, one of the most advanced testing techniques for web3 ever created, can be done using a single command. And by using Diligence, you seamlessly pick up the foundry fuzz tests and start fuzzing without any additional work.

$ fuzz forge test -k <your_api_key>

If all goes well, you should see output that looks something like this:

Parsing foundry config
Compiling tests
Collecting tests
Collecting and validating campaigns for submission
Preparing the seed state
Submitting campaigns
You can view campaign here: https://fuzzing.diligence.tools/campaigns/cmp_a39a98d29554496b91f4a977804d468d
Done 

You can obtain detailed information about your test results by visiting the campaign above. It should look something like this:

Conclusion

Fuzzing with Diligence Fuzzing and Foundry makes it easy to add powerful fuzzing to your testing tools. With it, you should be able to catch many of the pressing vulnerabilities that may plague your smart contracts. With this integration, in fact, there’s no reason not to include fuzzing in your workflows.