Hi all, in the previous article, I have covered how to create a simple transaction system for our blockchain. So today, I will build the core component of our cryptocurrency - the peer-to-peer (p2p) network. It's not only required for a blockchain to work, but after building it, we can really release our coins! Yes, we are at that stage already.
I have also created a tutorial on Youtube, check it our for better understanding.
What is a p2p network?
First, we have to understand what is a peer-to-peer network. In the last parts, I have talked about it a few times, but it's not really ideal yet, so I will dig deeper about it in this article.
Before we get on to that, we need to understand the client-server model first. In our life, pretty much everything we use currently follows the client-server model. The network works by having a server, and every machines will connect to that server. You can send requests to the server, and the server can send back responses. Think of the system as a company, and the boss is the server. The boss is the one who makes decisions and controls every action of the application.
But in a distributed peer-to-peer model, machines (called nodes) can send messages to each other without having any 3rd-party system involved. Think of it as a group of friends working together. People can work independently, and decisions will be made by the majority.
In a cryptocurrency's network, people should be able to at least send transactions and suggest new blocks.
Without futher ado, let's code!
What and how we are trying to achieve
We would need a network where nodes can connect with and send messages to each other in a peer to peer fashion. After that, we will add functionalities like broadcasting transactions, suggesting new blocks, send the chain and chain's information.
I will be using WebSocket - a TCP-based protocol, please learn about it to have a better understanding with what I will be doing.
Also, please read the comments in the code, I use them to explain individual functionalities.
Setup
I will use a light package called ws just to stay simple in this article.
Simply install it using npm:
npm install ws
Basic concepts of ws
// Import the package
const WS = require("ws");
// Create a server
const server = new WS.Server({ port: "SOME PORT" });
// Listens for connections
server.on("connection", async (socket, req) => {
// This event handler will be triggered every time somebody send us connections
});
// Get the socket from an address
const socket = new WS("SOME ADDRESS");
// Open a connection
socket.on("open", () => {
// This event handler will be triggered when a connection is opened
})
// Close a connection
socket.on("close", () => {
// This event handler will be triggered when the connection is closed
})
// Listens for messages
socket.on("message", message => {
// "message" is message, yes
})
A basic node
Create a new file, call it anything you want.
Basically, we will have the basic variables for PORT, the list of peers we are connecting (PEERS), our address (MY_ADDRESS).
I use process.env.abcxyz so that you can configure the node easily through command line.
// BTW, I will import the required stuff too
const crypto = require("crypto"), SHA256 = message => crypto.createHash("sha256").update(message).digest("hex");
const { Block, Transaction, JeChain } = require("./jechain");
const EC = require("elliptic").ec, ec = new EC("secp256k1");
const MINT_PRIVATE_ADDRESS = "0700a1ad28a20e5b2a517c00242d3e25a88d84bf54dce9e1733e6096e6d6495e";
const MINT_KEY_PAIR = ec.keyFromPrivate(MINT_PRIVATE_ADDRESS, "hex");
const MINT_PUBLIC_ADDRESS = MINT_KEY_PAIR.getPublic("hex");
// Your key pair
const privateKey = process.env.PRIVATE_KEY || "62d101759086c306848a0c1020922a78e8402e1330981afe9404d0ecc0a4be3d";
const keyPair = ec.keyFromPrivate(privateKey, "hex");
const publicKey = keyPair.getPublic("hex");
// The real new code
const WS = require("ws");
const PORT = process.env.PORT || 3000;
const PEERS = process.env.PEERS ? process.env.PEERS.split(",") : [];
const MY_ADDRESS = process.env.MY_ADDRESS || "ws://localhost:3000";
const server = new WS.Server({ port: PORT });
console.log("Listening on PORT", PORT);
// I will add this one line for error handling:
process.on("uncaughtException", err => console.log(err));
The MINTING address should never be changed, and we are going to change the old genesis block too:
const initalCoinRelease = new Transaction(MINT_PUBLIC_ADDRESS, "04719af634ece3e9bf00bfd7c58163b2caf2b8acd1a437a3e99a093c8dd7b1485c20d8a4c9f6621557f1d583e0fcff99f3234dd1bb365596d1d67909c270c16d64", 100000000);
We are releasing coins and send it to a guy with the address above, which is basically just from this private key: 62d101759086c306848a0c1020922a78e8402e1330981afe9404d0ecc0a4be3d
Remember to replace the old mint key pair with the new one too.
Now, let's have a way to connect to other nodes, as well as listen to other nodes' connections.
To implement this system, we need a function to connect, and use server.on("connection") for listening to connections.
The connect function should be able to connect to an address, then send it our address, then, the connection handler of that address will connect to our address using the message given.
A message is a string, in this case, a JSON, which have a form like this:
{
"type": "...",
"data": "..."
}
What we need in this case is:
{
"type": "TYPE_HANDSHAKE",
"data": ["Our address and our connected nodes' address", "address x", "address y"]
}
I will create a function to generate messages for convenience:
function produceMessage(type, data) {
return { type, data }
}
Now, let's implement the main system:
// THE CONNECTION LISTENER
server.on("connection", async(socket, req) => {
// Listens for messages
socket.on("message", message => {
// Parse the message from a JSON into an object
const _message = JSON.parse(message);
switch(_message.type) {
case "TYPE_HANDSHAKE":
const nodes = _message.data;
nodes.forEach(node => connect(node))
// We will need to handle more types of messages in the future, so I have used a switch-case.
}
})
});
// THE CONNECT FUNCTION
async function connect(address) {
// Get the socket from address
const socket = new WS(address);
// Connect to the socket using the "open" event
socket.on("open", () => {
// Send our address to the target
socket.send(JSON.stringify(produceMessage("TYPE_HANDSHAKE", [MY_ADDRESS])));
});
}
To actually do stuff in the future, we would want to store connected sockets and addresses into one array. Also, by doing this, we can send other nodes the address of the node that has just been connected to us.
let opened = [], connected = [];
// I will use "opened" for holding both sockets and addresses, "connected" is for addresses only.
async function connect(address) {
// We will only connect to the node if we haven't, and we should not be able to connect to ourself
if (!connected.find(peerAddress => peerAddress === address) && address !== MY_ADDRESS) {
const socket = new WS(address);
socket.on("open", () => {
// I will use the spread operator to include our connected nodes' addresses into the message's body and send it.
socket.send(JSON.stringify(produceMessage("TYPE_HANDSHAKE", [MY_ADDRESS, ...connected])));
// We should give other nodes' this one's address and ask them to connect.
opened.forEach(node => node.socket.send(JSON.stringify(produceMessage("TYPE_HANDSHAKE", [address]))));
// If "opened" already contained the address, we will not push.
if (!opened.find(peer => peer.address === address) && address !== MY_ADDRESS) {
opened.push({ socket, address });
}
// If "connected" already contained the address, we will not push.
if (!connected.find(peerAddress => peerAddress === address) && address !== MY_ADDRESS) {
connected.push(address);
}
// Two upper if statements exist because of the problem of asynchronous codes. Since they are running
// concurrently, the first if statement can be passed easily, so there will be duplications.
});
// When they disconnect, we must remove them from our connected list.
socket.on("close", () => {
opened.splice(connected.indexOf(address), 1);
connected.splice(connected.indexOf(address), 1);
});
}
}
To connect to all prefixed peers, you can add this line in:
PEERS.forEach(peer => connect(peer));
Integrate our blockchain into the network.
What do we need to do?
Alright, now that we have our node running, let's start diving into the real deal part of the article - cryptocurrency. To create a cryptocurrency, we would need to be able to broadcast transactions, suggest newly mined blocks. New nodes should be able to ask other nodes for their chains too.
0. Adding necessary stuff
Because when we send messages, we are effectively parsing objects to JSON, meaning that the methods of an object (in this case, they are the transactions, blocks, blockchains) will disappear. We can solve this problem by making our methods static, so we can re-use them without having to touch the real objects themselves.
And in the previous article, the validation methods of blocks and transactions are not really ideal, so let's update them while we are turning everything static.
static hasValidTransactions(block, chain) {
let gas = 0, reward = 0;
block.data.forEach(transaction => {
if (transaction.from !== MINT_PUBLIC_ADDRESS) {
gas += transaction.gas;
} else {
reward = transaction.amount;
}
});
return (
reward - gas === chain.reward &&
block.data.every(transaction => Transaction.isValid(transaction, chain)) &&
block.data.filter(transaction => transaction.from === MINT_PUBLIC_ADDRESS).length === 1
);
}
static isValid(blockchain) {
for (let i = 1; i < blockchain.chain.length; i++) {
const currentBlock = blockchain.chain[i];
const prevBlock = blockchain.chain[i-1];
if (
currentBlock.hash !== Block.getHash(currentBlock) ||
prevBlock.hash !== currentBlock.prevHash ||
!Block.hasValidTransactions(currentBlock, blockchain)
) {
return false;
}
}
return true;
}
static isValid(tx, chain) {
return (
tx.from &&
tx.to &&
tx.amount &&
(chain.getBalance(tx.from) >= tx.amount + tx.gas || tx.from === MINT_PUBLIC_ADDRESS) &&
ec.keyFromPublic(tx.from, "hex").verify(SHA256(tx.from + tx.to + tx.amount + tx.gas), tx.signature)
)
}
static getHash(block) {
return SHA256(block.prevHash + block.timestamp + JSON.stringify(block.data) + block.nonce);
}
Related methods
constructor(timestamp = Date.now().toString(), data = []) {
this.timestamp = timestamp;
this.data = data;
this.prevHash = "";
this.hash = Block.getHash(this);
this.nonce = 0;
}
mine(difficulty) {
while (!this.hash.startsWith(Array(difficulty + 1).join("0"))) {
this.nonce++;
this.hash = Block.getHash(this);
}
}
addBlock(block) {
block.prevHash = this.getLastBlock().hash;
block.hash = Block.getHash(block);
block.mine(this.difficulty);
this.chain.push(Object.freeze(block));
this.difficulty += Date.now() - parseInt(this.getLastBlock().timestamp) < this.blockTime ? 1 : -1;
}
addTransaction(transaction) {
if (Transaction.isValid(transaction, this)) {
this.transactions.push(transaction);
}
}
1. Transactions
First, I will create a handy-dandy sendMessage function to send messages to nodes easier.
function sendMessage(message) {
opened.forEach(node => {
node.socket.send(JSON.stringify(message));
});
}
Now, let's handle the messages!
A message for broadcasting transactions will look like this:
{
"type": "TYPE_CREATE_TRANSACTION",
"data": "the transaction object goes here"
}
In our message handler, we will create a new case which simply uses the handy-dandy addTransactions method we have created in the last part.
switch(_message.type) {
...
case "TYPE_CREATE_TRANSACTION":
const transaction = _message.data;
JeChain.addTransaction(transaction);
break;
}
And you can send a transaction like this:
sendMessage(produceMessage("TYPE_CREATE_TRANSACTION", someTransaction));
// You must also add the transaction to your pool:
JeChain.addTransaction(someTransaction);
2. Mining and sending new blocks
Now, let's handle the new block's suggestion messages.
This is by far the hardest, most bulky part to implement, so let's get going shall we?
The message will look like this:
{
"type": "TYPE_REPLACE_CHAIN",
"data": [
"new block",
"new difficulty"
]
}
How would we handle this message? The simplest thing we would do first is to check if the block is valid or not, then we will add it to the chain and update the difficulty. The block is valid when:
- It has valid transactions (the transactions are in our transaction pool, the transactions are valid according to our old methods).
- It has a valid hash (matches with the block's information (also called "block header")).
- It has a valid difficulty (it can't be greater or less than difficulty plus/minus 1).
- It has a valid timestamp (must not be greater than the time they sent us and less than the previous block's timestamp). This is not really a fulfill way to adjust difficulty, but at least it shouldn't create too much damage.
switch(_message.type) {
...
case "TYPE_REPLACE_CHAIN":
const [ newBlock, newDiff ] = _message.data;
// We are checking if the transactions exist in the pool by removing elements from transactions of the block if they exist in the pool.
// Then, we simply use `theirTx.length === 0` to check if the all elements are removed, meaning all transactions are in the pool.
const ourTx = [...JeChain.transactions.map(tx => JSON.stringify(tx))];
const theirTx = [...newBlock.data.filter(tx => tx.from !== MINT_PUBLIC_ADDRESS).map(tx => JSON.stringify(tx))];
const n = theirTx.length;
if (newBlock.prevHash !== JeChain.getLastBlock().prevHash) {
for (let i = 0; i < n; i++) {
const index = ourTx.indexOf(theirTx[0]);
if (index === -1) break;
ourTx.splice(index, 1);
theirTx.splice(0, 1);
}
if (
theirTx.length === 0 &&
SHA256(JeChain.getLastBlock().hash + newBlock.timestamp + JSON.stringify(newBlock.data) + newBlock.nonce) === newBlock.hash &&
newBlock.hash.startsWith(Array(JeChain.difficulty + 1).join("0")) &&
Block.hasValidTransactions(newBlock, JeChain) &&
(parseInt(newBlock.timestamp) > parseInt(JeChain.getLastBlock().timestamp) || JeChain.getLastBlock().timestamp === "") &&
parseInt(newBlock.timestamp) < Date.now() &&
JeChain.getLastBlock().hash === newBlock.prevHash &&
(newDiff + 1 === JeChain.difficulty || newDiff - 1 === JeChain.difficulty)
) {
JeChain.chain.push(newBlock);
JeChain.difficulty = newDiff;
JeChain.transactions = [...ourTx.map(tx => JSON.parse(tx))];
}
}
break;
}
But turns out, there's one really dangerous problem. If one miner mines a block, he wouldn't really know if his block came first or the other one sent to him came first. Yes, this does happen due to many impacts, one of them is internet problem. Imagine if someone mined a block before you, and he had sent the block to other nodes already, but due to some internet problem, you can manage to finish mining the block after the message is received, so the block sent will be invalid, but it's in fact valid and you will be left behind.
There are many approaches to this, but I have coined out a really simple system which leverages the use of majority's support.
We can affectively implement this functionality using a boolean variable called checking and setTimeout. Basically, the idea is that if the block's prevHash is equal to the latest block's prevHash, then it's probably a block that needs checking for replacement. We will set checking to true to indicates that we are checking, then, we will request other nodes for their latest block. We will wait for a period of time (which I have set to 5s) using setTimeout, then we will set checking to false, cancelling the process, and the block that appeared the most is likely the block we need. I will also implement a system to skip on all similar blocks after we have had the correct answer.
let check = [];
let checked = [];
let checking = false;
...
if (newBlock.prevHash !== JeChain.getLastBlock().prevHash) {
...
// If this case was found once, simply just dismiss it
} else if (!checked.includes(JSON.stringify([newBlock.prevHash, JeChain.chain[JeChain.chain.length-2].timestamp || ""]))) {
checked.push(JSON.stringify([JeChain.getLastBlock().prevHash, JeChain.chain[JeChain.chain.length-2].timestamp || ""]));
const position = JeChain.chain.length - 1;
checking = true;
sendMessage(produceMessage("TYPE_REQUEST_CHECK", MY_ADDRESS));
setTimeout(() => {
checking = false;
let mostAppeared = check[0];
check.forEach(group => {
if (check.filter(_group => _group === group).length > check.filter(_group => _group === mostAppeared).length) {
mostAppeared = group;
}
})
const group = JSON.parse(mostAppeared)
JeChain.chain[position] = group[0];
JeChain.transactions = [...group[1]];
JeChain.difficulty = group[2];
check.splice(0, check.length);
}, 5000);
}
Note that the upper code is more of a proof-of-concept, we often would like to check if the block is valid or not just to be safer. There are faster, cleaner, more secure ways than this, but this should do our job.
Let's create a way to handle TYPE_REQUEST_CHECK. We will send back TYPE_SEND_CHECK, so let's make one for that as well.
The message will look like this:
{
"type": "TYPE_REQUEST_CHECK",
"data": "address to send back"
}
{
"type": "TYPE_SEND_CHECK",
"data": ["block", "transaction pool", "difficulty"]
}
The handler:
case "TYPE_REQUEST_CHECK":
// Find the address and send back necessary data.
opened.filter(node => node.address === _message.data)[0].socket.send(
JSON.stringify(produceMessage(
"TYPE_SEND_CHECK",
JSON.stringify([JeChain.getLastBlock(), JeChain.transactions, JeChain.difficulty])
))
);
break;
case "TYPE_SEND_CHECK":
// Only push to check if checking is enabled
if (checking) check.push(_message.data);
break;
So the handler is finally done!
You can mine blocks like this:
if (JeChain.transactions.length !== 0) {
// Note that technically you can choose your own transactions to mine, but I would like to mine all transactions at once.
JeChain.mineTransactions(publicKey);
sendMessage(produceMessage("TYPE_REPLACE_CHAIN", [
JeChain.getLastBlock(),
JeChain.difficulty
]));
}
3. Sending chains
For new nodes that have just joined the network, there are 2 ways to get the latest chain. You can either get a chain from a trust-worthy source, or you can ask for the chain in the network. Note that the size of a message is limited, so we won't be able to send the whole chain, we will send its blocks and information one by one.
We can implement the second solution like this:
let tempChain = new Blockchain();
...
case "TYPE_SEND_CHAIN":
const { block, finished } = _message.data;
if (!finished) {
tempChain.chain.push(block);
} else {
tempChain.chain.push(block);
if (Blockchain.isValid(tempChain)) {
JeChain.chain = tempChain.chain;
}
tempChain = new Blockchain();
}
break;
case "TYPE_REQUEST_CHAIN":
const socket = opened.filter(node => node.address === _message.data)[0].socket;
// We will send the blocks continously.
for (let i = 1; i < JeChain.chain.length; i++) {
socket.send(JSON.stringify(produceMessage(
"TYPE_SEND_CHAIN",
{
block: JeChain.chain[i],
finished: i === JeChain.chain.length - 1
}
)));
}
break;
case "TYPE_REQUEST_INFO":
opened.filter(node => node.address === _message.data)[0].socket.send(
"TYPE_SEND_INFO",
[JeChain.difficulty, JeChain.transactions]
);
break;
case "TYPE_SEND_INFO":
[ JeChain.difficulty, JeChain.transactions ] = _message.data;
break;
Note that you can send request chain to a trust-worthy node, or base on the majority.
Testing in localhost
To test, I will start 2 new consoles with different PORT, MY_ADDRESS, and PRIVATE_KEY. For the first one, I will set our peers to be empty, and the key to be what the initial coin release points to. For the other one, I will set the peer list to be the first node to test if our "handshake" functionality work. Then, I'm going to create a transaction in the first node and mine in the second node. After 10 seconds, we will print out the opened array and the chain.
First node:
setTimeout(() => {
const transaction = new Transaction(publicKey, "046856ec283a5ecbd040cd71383a5e6f6ed90ed2d7e8e599dbb5891c13dff26f2941229d9b7301edf19c5aec052177fac4231bb2515cb59b1b34aea5c06acdef43", 200, 10);
transaction.sign(keyPair);
sendMessage(produceMessage("TYPE_CREATE_TRANSACTION", transaction));
JeChain.addTransaction(transaction);
}, 5000);
setTimeout(() => {
console.log(opened);
console.log(JeChain);
}, 10000);
Second node:
setTimeout(() => {
if (JeChain.transactions.length !== 0) {
JeChain.mineTransactions(publicKey);
sendMessage(produceMessage("TYPE_REPLACE_CHAIN", [
JeChain.getLastBlock(),
JeChain.difficulty
]));
}
}, 6500);
setTimeout(() => {
console.log(opened);
console.log(JeChain);
}, 10000);
It should look like this:
Nodes have connected to each others, the block is mined, the chain is synced!
Releasing our coin (testing publicly)
Simply host a node publicly (by using port forwarding, also, for each router, you would have a different way to do port forwarding, just simply look up online to see what suits your model), using your PC or a VPS hosting service.
I have tested with my friend here:
My node which is the first node:
His node which is the second node:
We have done some port forwarding and connected to each others' public IP address.
Note that this network is not meant to be production-ready, but this network should be fine for now.
Source code
The full source code used in this article can be found in this Github repo. Or here with JeChain.
Shoutouts
I want to give appreciation to my friend Apple who have contributed to the code used in the article, and Trey - a really cool guy who enlightens me in constructing the network.
- Check out Apple's Github account.
- Check out Trey's website.
Contacts
I have also created a tutorial on Youtube, check it our for better understanding.
Top comments (2)
Nice work. I am quite the beginner and i have been following this tutorial from the part 1. Please how did you generate the pairs that you used here?
const MINT_PRIVATE_ADDRESS = "0700a1ad28a20e5b2a517c00242d3e25a88d84bf54dce9e1733e6096e6d6495e";
const MINT_KEY_PAIR = ec.keyFromPrivate(MINT_PRIVATE_ADDRESS, "hex");
const MINT_PUBLIC_ADDRESS = MINT_KEY_PAIR.getPublic("hex");
// Your key pair
const privateKey = process.env.PRIVATE_KEY || "62d101759086c306848a0c1020922a78e8402e1330981afe9404d0ecc0a4be3d";
const keyPair = ec.keyFromPrivate(privateKey, "hex");
const publicKey = keyPair.getPublic("hex");
I know you asked us to refer to your previous YouTube video for steps to generate the keypairs but i didn't see it in the Creating Transaction video.
Please assist me or point me in the right direction.
Thanks for this useful info! I also advise to read this guide about p2p payment app development