ChatGPT created a script in 2 hours which would take 20-30 dev hours [prompts included]

I'm working on an open source project which creates integration tests for web apps in seconds by analysing server activity (https://github.com/Pythagora-io/pythagora).

In order for us to track server activity we needed to create a proxy that intercepts TCP packets that are going out of the server to a specific port and, in some specific cases, injects the response from the service on that port.

I'm an experienced web dev but I never worked on this kind of low level tech (TCP layer). So, I tried figuring out how to do this in C or Python but I couldn't make it work for 2 days straight. The problem I stumbled upon was updating ack and seq numbers so I ended up thinking that it will just take me too long to develop properly and debug.

So, I decided to see how much would it cost for someone who knows more about handling TCP packets to create this script for me.

I went to Upwork and the proposals I got were 20-30 developer hours to create this.

But, I thought, before I pay someone to do it, let's see what will ChatGPT suggest for this problem.

I ended up playing with ChatGPT for 2 hours and managed to create the proxy that work exactly as expected! It took only 16 messages.

I was mind blown by this so I thought to share my conversation with ChatGPT with everyone here. It might give someone an idea how to use GPT for whatever you're making.

I wanted to make it easy for it to create the script so I told it to listen on a port and when it recognised a string "brija", return the string "yoyobrija" as a response. In reality, we're using a different if condition and returning different response but I thought this will make it easier for me to write a meaningful prompt.

Ok, here it is.

This is ChatGPTs first try:

import socket

# Set the listen port and target port
listen_port = 16379
target_port = 6379

# Create a socket to listen for incoming connections
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Bind the socket to the listen port
listen_socket.bind(('', listen_port))

# Start listening for incoming connections
listen_socket.listen(1)

# Loop indefinitely
while True:
    # Accept an incoming connection
    connection, address = listen_socket.accept()

    # Receive data from the connection
    data = connection.recv(4096)

    # If the data contains the string "brija", return 5 packets with the content "yoyobrija"
    if b"brija" in data:
        for i in range(5):
            connection.send(b"yoyobrija")
    # Otherwise, forward the data to the target port
    else:
        # Create a socket to connect to the target port
        target_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

        # Connect to the target port
        target_socket.connect(('localhost', target_port))

        # Send the data to the target port
        target_socket.send(data)

        # Receive the response from the target port
        response = target_socket.recv(4096)

        # Forward the response back to the source
        connection.send(response)

        # Close the connection to the target port
        target_socket.close()

    # Close the connection
    connection.close()

Right away it was able to intercept TCP packets. However, the returned packet wasn't full which is something I got stuck on myself when I tried to do this.

It looks like the issue is that you are returning the "yoyobrija" string directly as the response to the client, but this string is not a valid Redis protocol response. In order to return a valid Redis protocol response to the client, you need to follow the Redis protocol.
Here is an example of how you can return a Redis protocol "bulk string" response to the client:

if b"brija" in data:
    # Build the Redis protocol response
    response = b"$8\r\nyoyobrija\r\n"

    # Send the response to the client
    connection.send(response)

This will send a Redis protocol bulk string response with the value "yoyobrija" to the client.

Alternatively, you could also use the redis-py library to build and send the Redis protocol response to the client. Here is an example of how you could do this using redis-py:

import redis

if b"brija" in data:
    # Build the Redis protocol response using redis-py
    response = redis.encode_command("yoyobrija")

    # Send the response to the client
    connection.send(response)

To return a string of any size in multiple packets, you will need to split the string into smaller chunks and send them to the client one at a time. Here is an example of how you can do this:

import redis

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [redis.encode_command(chunk) for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

This will send the "yoyobrija" string to the client in chunks of 1024 bytes. You can adjust the chunk size as needed to suit your needs.

I apologize for the mistake. The encode_command function is not a part of the redis library. Instead, you can use the pack function from the struct module to build a Redis protocol response.
Here is an example of how you can use the struct module to build a Redis protocol bulk string response with the value "yoyobrija":

import struct

if b"brija" in data:
    # Build the Redis protocol response
    response = struct.pack("!cIc", b"$", 8, b"\r\n") + b"yoyobrija" + b"\r\n"

    # Send the response to the client
    connection.send(response)

This will send a Redis protocol bulk string response with the value "yoyobrija" to the client.
Alternatively, you can also use the redis-protocol library to build Redis protocol responses. Here is an example of how you can use the redis-protocol library to build a Redis protocol bulk string response with the value "yoyobrija":

import redis_protocol

if b"brija" in data:
    # Build the Redis protocol response using redis-protocol
    response = redis_protocol.encode("yoyobrija")

    # Send the response to the client
    connection.send(response)

At this point, I saw that it's using a package called "struct". I don't have much experience with Python so I just went ahead and asked it how to get it installed.

The struct module is a built-in module in Python and does not need to be installed. You can use it in your script by simply importing it:

import struct

Now, this is where I saw that it's better to use this package instead so I directed it to use struct more.

To split a string into smaller chunks and build a Redis protocol response for each chunk using the struct module, you can use the following code:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack("!cIc", b"$", len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

This will split the "yoyobrija" string into chunks of 1024 bytes and build a Redis protocol bulk string response for each chunk. The struct.pack function is used to build the Redis protocol response, and the !cIc format string specifies that the response should contain a single byte (!c), a 32-bit unsigned integer (I), and another single byte (c). The \r\n bytes are used to separate the response elements.

The error you are seeing is occurring because the struct.pack function expects the first argument (the format string) to be a bytes object, but you are passing it a string. To fix the error, you need to pass the format string as a bytes object by prefixing it with b.

Here is the corrected code that should fix the error:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!cIc", b"$", len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

Now, we started to go back and forth a bit.

I apologize for the mistake. It looks like the issue is occurring because you are using the b"$" byte string as the first argument to the struct.pack function, but the $ character is not a valid format string character in the struct module. Instead, you should use the s format character to represent a string in the format string.

Here is the corrected code that should fix the error:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!sIc", b"$", len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

I apologize for the error. It looks like the issue is occurring because you are using the b"$" byte string as the first argument to the struct.pack function, but the $ character is not a valid format string character in the struct module. Instead, you should use the s format character to represent a string in the format string, and you should pass the actual string you want to encode as the first argument.

Here is the corrected code that should fix the error:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!sIc", b"$" + bytes(str(len(chunk)), 'ascii'), len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

Here is the corrected code that should fix the error:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!sIc", b"$" + bytes(str(len(chunk)), 'ascii'), len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

I apologize for the mistake. It looks like the issue is occurring because you are using the b"!sIc" format string, but the s format character expects a bytes object as its argument, and the I format character expects a 32-bit unsigned integer as its argument. However, you are passing the length of the chunk as the second argument to the struct.pack function, which is a string.

To fix the error, you should use the b prefix to specify that the arguments to the struct.pack function are bytes objects, and you should also use the b prefix to specify that the chunk length is a bytes object.

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!

I apologize for the mistake. It looks like the issue is occurring because you are using the bytes(str(len(chunk)), 'ascii') expression to convert the chunk length to a bytes object, but this will not work because the bytes function expects an iterable of integers as its first argument, and the str(len(chunk)) expression is not an iterable of integers.

To fix the error, you can use the str.encode method to encode the chunk length as a bytes object:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!sIc", b"$" + str(len(chunk)).encode('ascii'), len(chunk), b"\r\n") + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

At this point, I was tired with it writing the entire code over and over again so I wanted to get it to output only the lines to change. This failed 😅 but it still did the job.

I apologize for the mistake. It looks like the issue is occurring because you are using the str(len(chunk)).encode('ascii') expression to encode the chunk length as a bytes object, but this will not work because the str(len(chunk)) expression returns a string, and the encode method expects an integer as its argument.

To fix the error, you should pass the chunk length as an integer to the struct.pack function and use the !b format string to specify that it is a bytes object:

import struct

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [struct.pack(b"!b", b"$") + str(len(chunk)).encode('ascii') + chunk + b"\r\n" for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

To split a string into smaller chunks and build a Redis protocol response for each chunk using the redis_protocol module, you can use the following code:

import redis_protocol

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [redis_protocol.encode(chunk) for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

I apologize for the mistake. It looks like the issue is occurring because the redis_protocol.encode function expects a string as its argument, but you are passing it a bytes object.

To fix the error, you can use the .decode method to convert the bytes object to a string before passing it to the redis_protocol.encode function:

import redis_protocol

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [redis_protocol.encode(chunk.decode('utf-8')) for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response)

I apologize for the mistake. It looks like the issue is occurring because the connection.send function expects a bytes object as its argument, but you are passing it a string.

To fix the error, you can use the .encode method to convert the string to a bytes object before passing it to the connection.send function:

import redis_protocol

if b"brija" in data:
    # Set the chunk size
    chunk_size = 1024

    # Split the string into chunks
    chunks = [b"yoyobrija"[i:i + chunk_size] for i in range(0, len(b"yoyobrija"), chunk_size)]

    # Build the Redis protocol response for each chunk
    responses = [redis_protocol.encode(chunk.decode('utf-8')) for chunk in chunks]

    # Send the responses to the client
    for response in responses:
        connection.send(response.encode('utf-8'))

And that's it. After this it worked exactly as I wanted it to work. I open sourced the final script here. Also, as I mentioned, this is a part of Pythagora which helps create integration tests super easy - you can get from 0 to 80% code coverage in 30 minutes - and it's open sourced here - https://github.com/Pythagora-io/pythagora. It would mean a world to me if you could star our repo there.

Have you had any luck using ChatGPT to create something like this? I'm curious to see other examples of how ChatGPT is utilized for development.