Build workflow for Blockchain's development

TLDR: We were having a bad time with Blockchain smartcontract's development, and fortunately we had found solutions to save our asses. We'd like to share it with you, so maybe it could be useful to you also.

Table of Contents

1. Welcome to the Blockchain
2. Problems with our development/deployment
3. Writing Upgradable smartcontracts
4. Automatic deployment to the blockchains
5. Problem with the Truffle's migration process
6. Our completed workflows

Blockchain is fun!

It's a huge public database (chain of data blocks), which is being shared across many computers. Everyone can query data inside it, or add a new record by sending transactions. Everything is stored historically, so you can trace the changes easily.
With ethereum chains, it's even funnier with SmartContract - a program that runs on chains. Developers can write and compile a program, and upload it into the chain. Then everyone can run it to read its data, update its data with transactions, or even use it to run another program.

Tips: You can use this tool Smartcontract UI to interact with smartcontracts easily.

We were working on many projects based on blockchains, mainly in decentralized finance. Many of them are programs (written with Solidity) that involve managing users' account balance, allowing users to trade their assets, or to stake their assets to the liquidity pools and gain interests.

Problems with our development/deployment

However, we were usually facing a big problem: Smartcontracts in ethereum chains are immutable. It means once you deployed it on the chain, there is no way to change it. It works just like a contract we have in real life: you surely won't change the contract's contents once you signed it, if you want to, you'd need to have another contract. Same in ethereum chains, if we make changes in the contract code, we'd need to deploy it again to a new smartcontract.

And since a smartcontract has its data, when we deployed to a new smartcontract, we'd need to migrate all data from the old contract to the new one, while keeping support on both contracts until all users moved to the new contract.

Another problem we were having is the contract's deployment. After finishing the code and passing all tests on the local environment, we rely on someone to deploy it to the test chains so we can test its communication with other contracts. And since the contract's address will be changed each deployment, it's a really big pain for us to update all the addresses on each deployment.

After the contract's deployment, we have another problem with the interaction with our contracts. We provide ABIs (Application Binary Interface) to other people, so they can use them to interact with our contracts. It's also needed for our team to write applications that interact with contracts, so we'll need to keep them always updated with our deployed contracts.

SmartContract's development is fun, but its deployment was a pain for us for a long time.

Writing Upgradable smartcontracts with OpenZeppelin

We've decided to improve our deployment process. Thanks to OpenZeppelin, we're now able to upgrade our contracts smoothly.
When we deploy the contract on the first time, OpenZeppelin will create a Proxy Contract, points it to our actual contract, and finally, it deploys all contracts. Later, every time we make changes to the contract's code, OpenZeppelin will deploy it, and points the Proxy Contract to the newly deployed contract, keeping the old contract's state. Now we don't have to worry about the migration between contract's deployments. Our users and devs always connect with the Proxy Contract address, which points to the latest deployed contract.
It was a good change, that helped us during the release process. However, we were still having other problems with the deployment: We'd still need someone to build the contracts and deploy them to the chains. It requires access to our deployer's wallet, and we cannot give the wallet access to everyone in the team.

Automatic deployment with Github Actions and Workflows

After many tries, we finally integrate our build & deployment process with Github workflows. We also added some tweaks (like caching dependencies for faster build in future, configuring Truffle environment correctly, having wallet's private key in Github's secret,..)
Our deployment is automatic now! Every time we merge features into a development branch, a workflow will be triggered to build the contracts and upload them into the test blockchain. And when they are ready for production, we just need to tag the version. Github will deploy the contracts into the production blockchain.
We also release the contract's built artifact to Github release page every time we release the contracts to production, along with the prerelease of the contract's ABI on each push to the development branches. In this way, our users and devs can always update the contract's ABI quickly and easily.

Memorize Truffle's migration process

Everything seems OK, but actually, we're still missing a piece: We use actions/cache@v2 to cache the Truffle's build. So in the next deployment, we can continue the migration without doing it from the beginning.
However, the cache will be removed after some inactivity time, and when it's removed, or if there is some problem in the cache (cause of a wrongly configured deployment), our migration process will be restarted from the beginning. Therefore, all the proxy contracts will be changed. We will need to notice our users and devs, and migrate all contract's state.
Again, we had to try other approaches, and finally, we found a solution: Saving the Truffle's build on a deployed branch. So for each deployment, instead of taking the previous build from the cache, we will pull it from the deployed branch, and continue with it. It has some more advantages:

• The Truffle's build will be there always
• We can inspect the Truffle's build if there was any problem with the deployment
• We can also customize the deployed branch to give more informations

Submission Category: DIY Deployments

Yaml File or Link to Code

• README: https://github.com/ActionsHackathon21/deploy-upgradable-smartcontract-to-blockchain/blob/main/README.md
• Workflow file (development): https://github.com/ActionsHackathon21/deploy-upgradable-smartcontract-to-blockchain/blob/main/.github/workflows/migrate-to-testnet.yml
• Workflow file (release): https://github.com/ActionsHackathon21/deploy-upgradable-smartcontract-to-blockchain/blob/main/.github/workflows/migrate-to-mainnet.yml
• License: https://github.com/ActionsHackathon21/deploy-upgradable-smartcontract-to-blockchain/blob/main/COPYING

Configuration

• You can config the branch postfix which holds the development's artifacts with the DEPLOY_BRANCH_POSTFIX variable.
• You can also configure the development branches which you want to deploy to testnet, with branches key.
• To config the blockchain you want to deploy to, use the WALLET_SECRET , RPC, NETWORK_ID and CONFIRMATIONS variables Important! You should store the wallet secret in GitHub's secret (Settings > Secrets). On this project, I stored as DEV_WALLET_SECRET and PROD_WALLET_SECRET secrets

Flows

• Use actions/checkout@v2 to checkout source code from the repository
• Use actions/setup-node@v2 to setup nodejs
• Use actions/cache@v2 to cache dependencies
• Install build dependencies (yarn, node-gyp, node-gyp-build)
• Install dependencies from yarn.lock
• Synchronize previous built artifacts from the deployment branch
• Build and Migrate smartcontracts
• Push new built artifacts into the deployment branch
• Release smartcontracts' JSON (including ABI)