How does a Simple Web Server Work?

With an interest on the security of systems that are web and/or mobile based, research in that direction during my masters program is what brought me to poke into it. I have in mind to build some side projects during this program to get into the train of building web and mobile based software to be much better as a coder. Having had a stint in DevOps/Infrastructure I wish to transition back to product development as it mostly pertains to writing code and that’s where I want to continue growing my skills during this program.

I believe that to become a better developer you have to get a better understanding of the underlying software systems you use on a daily basis and that includes programming languages, compilers and interpreters, database and operating systems, web servers and web frameworks. And, to get a better and deeper understanding of those systems you have to rebuild from scratch, brick by brick, wall by wall.
As the great Confucius had put it: “I hear and I forget, I see and I remember, I do and I understand”

At this point I hope you are convinced that it’s sometimes a good idea to start rebuilding different software systems to learn how they work. In this post I will show how to build your own basic Web server.
To start, first things first, What is a Web server?

Courtesy of StackOverflow.com:

Overall it’s a networking server(which is virtual or software based) that sits on a physical server and waits for a client to send a request. When it receives a request it generates a response and sends it back to the client. The communication between a client and a server happens using the HTTP protocol. The “client” here is your browser or any other software that speaks HTTP(which is a language that helps different computers to communicate on the World Wide Web).
What would a simple implementation of a Web server look like? Here is my take on this comprehension. The example is in Python and it’s tested on Python 3.5:

import socket

HOST, PORT = '', 7777

listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print(“Serving HTTP on port {PORT} …”)
while True:
    client_connection, client_address = listen_socket.accept()
    request_data = client_connection.recv(1024)
    print(request_data.decode(“utf-8”))

    http_response = b"""\
HTTP/1.1 200 OK

Hello, World!
"""
    client_connection.sendall(http_response)
    client_connection.close()

Save the code above in a file called demoserver.py or download it from GitHub and run it from the command line:

$ python demoserver.py
Serving HTTP on port 7777 …

Hit Enter after typing in the following URL in your Web browser’s address bar http://localhost:7777/ and see magic in action. You should see “Hello, World!” displayed in your browser like this:

First try it and obtain the results.

Done? Great! Now let’s talk about how it actually works.

Let’s start with the web address you’ve entered which is called a URL and here is the structure;

http:// : Hyper Text Transfer Protocol
Localhost: The host name
:7777/: Port number and path(‘/’)

This is how you tell your browser the address of the webserver where it needs to connect and find the page (which is also the path and in our case the path is the root directory’/’) on the server and the server is going to fetch that for you.
Before the browser can send an HTTP request, it needs to first establish a TCP connection with the webserver. Then it sends an HTTP request over that TCP connection to the server and waits for the server to send the HTTP response back(as you can see it is a request and response cycle and all over the web that’s how connections work). And when your browser receives the response it displays it, in our case it displays “Hello World!”

Now how does the client establishes a TCP connection to the server before sending the HTTP requests and responses. To do that they both used what is called sockets. Instead of using a browser directly let’s simulate your browser manually by using telnet on the command line.

On the same computer where you’re running the webserver fireup a telnet session on the command line specifying a host to connect to localhost and the port to connect to 7777 and hit Enter:

$ telnet localhost 7777
Trying 127.0.0.1 …
Connected to localhost

At this point you have established a connection with the server running on localhost and it is ready to send and receive HTTP messages.

In the same telnet session type GET / HTTP/1.1 and hit Enter:

$ telnet localhost 7777
Trying 127.0.0.1 …
Connected to localhost.
GET / HTTP/1.1

HTTP/1.1 200 OK
Hello, World!

Fantastic! You’ve just manually simulated your browser! You sent an HTTP request and got an HTTP response back. This is the basic structure of an HTTP request;

GET: An HTTP method
/: the path(root directory in this case)
HTTP/1.1: The HTTP version

The HTTP request consists of the line indicating the HTTP method(GET, because we are asking our server to return something to us), the path / that indicates a “page” on the server we want and the protocol version.

For the sake of simplicity our webserver completely ignores the above request line as you could just type anything and you would still get back a “Hello, World!” response.

Once you’ve typed the request line and hit Enter the client sends the request to the server, the server then reads the request line, prints it and returns the proper HTTP response.

Here is the HTTP response that the server sends back to your client;

HTTP/1.1: The HTTP version
200 OK: HTTP status code
Hello, World!: HTTP response body

Let’s take it apart to understand what it actually means in detail. The response consists of a status line HTTP/1.1 200 OK, followed by an empty line and then the HTTP response body.

The response status line HTTP/1.1 200 OK consists of the HTTP version, the HTTP status code and the HTTP status code reason phrase OK. When the browser gets the response, it displays the body of the response and that’s why you see “Hello, Word!” in your browser.

And that’s how the very basic model of a webserver works.
To sum everything up; the webserver creates a listening socket and starts accepting new connections in a loop. The client initiates a TCP connection and, after successfully establishing it, the client sends an HTTP request to the server and the server responds with an HTTP response that gets displayed to the user. To establish a TCP connection both clients and servers use sockets.
Now you have a basic understanding of a working webserver that you can test with your browser or some other HTTP client. As you’ve seen and hopefully tried, you can also be a human HTTP client by using telnet and typing HTTP requests manually.

Congratulations! You have built a very simple webserver and there are things you can add like concurrency and make it adaptable to different frameworks without changing the server to suit them.