loading...
Cover image for Introduction to the Bitcoin Network Protocol using Python and TCP Sockets

Introduction to the Bitcoin Network Protocol using Python and TCP Sockets

alecbuda profile image Alec Buda ・4 min read

Blockchain technology is built around consensus algorithms which allow distributed nodes to share a common ledger. A fundamental dependency of these algorithms is a common network protocol to enable communication between participating nodes. Today, let's write a Python program from scratch to interact with a real Bitcoin node.

This post will assume you're familiar with the fundamentals of blockchain technology. If you aren't, I would recommend checking out the Bitcoin White Paper by Satoshi Nakamoto.

Bitcoin nodes communicate with each other using the TCP protocol. Nodes will typically listen on port number 8333. For a detailed description of the bitcoin network protocol check out this resource.

Today, we are going to write a Python program to connect to a Bitcoin node and fetch the details of a specific transaction. Here is a diagram of the message flow that will be developed.

Alt Text

Before we start coding our program, we must make one point clear. Interacting with a Bitcoin node using raw TCP sockets is reinventing the wheel. This has already been done by python packages such as python-bitcoinlib.

If you want to write sophisticated applications you should definitely use the correct tool for the job. With that said though, programming with TCP sockets is a great way to improve your low level understanding of a network protocol.

To begin, let's import the dependencies our program will require.

#!/usr/bin/env python

# Filename:                     bitcoin-network-tutorial.py
# Command to run the program:   python bitcoin-network-tutorial.py

# Import dependencies
import socket
import time
import random
import struct
import hashlib
import binascii

Let's now define the methods required for constructing the "version" request message.

# Binary encode the sub-version
def create_sub_version():
    sub_version = "/Satoshi:0.7.2/"
    return b'\x0F' + sub_version.encode()

# Binary encode the network addresses
def create_network_address(ip_address, port):
    network_address = struct.pack('>8s16sH', b'\x01', 
        bytearray.fromhex("00000000000000000000ffff") + socket.inet_aton(ip_address), port)
    return(network_address)

# Create the TCP request object
def create_message(magic, command, payload):
    checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[0:4]
    return(struct.pack('L12sL4s', magic, command.encode(), len(payload), checksum) + payload)

# Create the "version" request payload
def create_payload_version(peer_ip_address):
    version = 60002
    services = 1
    timestamp = int(time.time())
    addr_local = create_network_address("127.0.0.1", 8333)
    addr_peer = create_network_address(peer_ip_address, 8333)
    nonce = random.getrandbits(64)
    start_height = 0
    payload = struct.pack('<LQQ26s26sQ16sL', version, services, timestamp, addr_peer,
                          addr_local, nonce, create_sub_version(), start_height)
    return(payload)

The struct module is used for packing binary data. The hashlib module is used for generating message checksums. For a full understanding of the code, you'll need to cross reference the data encoding with the protocol documentation.

Next, let's add a method for creating the "verack" request message. The verack command name is derived from "version acknowledge".

# Create the "verack" request message
def create_message_verack():
    return bytearray.fromhex("f9beb4d976657261636b000000000000000000005df6e0e2")

With the mandatory messages out of the way, we may now create our "getdata" method for retrieving the details of a specific transaction.

# Create the "getdata" request payload
def create_payload_getdata(tx_id):
    count = 1
    type = 1
    hash = bytearray.fromhex(tx_id)
    payload = struct.pack('<bb32s', count, type, hash)
    return(payload)

Please note that not all nodes will be able to return arbitrary transaction data; some will prune their history to save disk space.

We'll also create a method for printing TCP data to the terminal.

# Print request/response data
def print_response(command, request_data, response_data):
    print("")
    print("Command: " + command)
    print("Request:")
    print(binascii.hexlify(request_data))
    print("Response:")
    print(binascii.hexlify(response_data))

We may now add our main method which will connect to a bitcoin node and execute the desired message flow.

if __name__ == '__main__':
    # Set constants
    magic_value = 0xd9b4bef9
    tx_id = "fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20"
    peer_ip_address = '104.199.184.15'
    peer_tcp_port = 8333
    buffer_size = 1024

    # Create Request Objects
    version_payload = create_payload_version(peer_ip_address)
    version_message = create_message(magic_value, 'version', version_payload)
    verack_message = create_message_verack()
    getdata_payload = create_payload_getdata(tx_id)
    getdata_message = create_message(magic_value, 'getdata', getdata_payload)

    # Establish TCP Connection
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((peer_ip_address, peer_tcp_port))

    # Send message "version"
    s.send(version_message)
    response_data = s.recv(buffer_size)
    print_response("version", version_message, response_data)

    # Send message "verack"
    s.send(verack_message)
    response_data = s.recv(buffer_size)
    print_response("verack", verack_message, response_data)

    # Send message "getdata"
    s.send(getdata_message)
    response_data = s.recv(buffer_size)
    print_response("getdata", getdata_message, response_data)

    # Close the TCP connection
    s.close()

We found the IP address of the node using Bitnodes. Details of the transaction we elected to query can be found on a block explorer.

Execute the program on a terminal with the command python bitcoin-network-tutorial.py. A sample output is provided below.

Command: version
Request:
b'f9beb4d976657273696f6e000000000064000000f4de76b762ea00000100000000000000c8c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d010000000000000000000000000000000000ffff7f000001208d0f2f736a397699b60f2f5361746f7368693a302e372e322f00000000'
Response:
b'f9beb4d976657273696f6e000000000066000000fe4aee167f1101000d04000000000000c2c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d0d040000000000000000000000000000000000000000000000000b63185e17ebcdb3102f5361746f7368693a302e31382e302fbc29090001'

Command: verack
Request:
b'f9beb4d976657261636b000000000000000000005df6e0e2'
Response:
b'f9beb4d976657261636b000000000000000000005df6e0e2'

Command: getdata
Request:
b'f9beb4d9676574646174610000000000220000007b00a9b50101fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20'
Response:
b'f9beb4d9616c65727400000000000000c0000000d2f50d9ef9beb4d9616c65727400000000000000a80000001bf9aaea60010000000000000000000000ffffff7f00000000ffffff7ffeffff7f01ffffff7f00000000ffffff7f00ffffff7f002f555247454e543a20416c657274206b657920636f6d70726f6d697365642c2075706772616465207265717569726564004630440220653febd6410f470f6bae11cad19c48413becb1ac2c17f908fd0fd53bdc3abd5202206d0e9c96fe88d4a0f01ed9dedae2b6f9e00da94cad0fecaae66ecf689bf71b50'

That concludes the tutorial! Stay tuned for more.

ULTRA CONFIG GENERATOR

Have you heard of Ultra Config Generator? If you haven't, I highly recommend you check it out.

We designed the product to allow network engineers to generate and automate network configuration in a highly flexible, efficient and elegant manner. Our customers love the application and I hope that you will too.

Take care until next time!

Ultra Config

Posted on by:

alecbuda profile

Alec Buda

@alecbuda

Passionate Engineer and Entrepreneur. Founder of Ultra Config.

Discussion

markdown guide
 

i'm getting an empty response for every "s.send" . Is it because txn is outdated?

 

Hello, I just reran the script in its entirety and it still works for me.

Did you copy everything as it appeared in the blog?

Best Regards,

Alec

 

Hi, for s.send(verack_message), I get timeout. The version message works, but every host I connect to does not respond to the verack message. Is anything changed since last updated?
Thank you