What is “from scratch”?
Sometimes we are told “Don’t reinvent wheel”. It may be right when we want to write reliable and stable software. We should rely on mature framework instead of write our own framework. However sometimes life is too boring if we don’t challenge ourselves. If we want to learn how web frame works, we should write it by ourselves. “I see and I remember. I do and I understand.”
Ok, you maybe want to write your own web framework “from scratch”. Well, now you have two problems :-). What is “from scratch” ? Does it mean that we need to reinvent “semiconductor”. Of course not. According to what you want to learn, you can “stand on the shoulders of giants”. If you want to learn low level module, you can write it beginning from TCP or HTTP Protocol. And you can write a HTTP server and make a web framework based on it.
Regarding this article, we will rely on others’ HTTP server, because I just want to know how a web framework works.
What is web framework?
Web framework is also called web application framework. It is built on the HTTP server which is built on HTTP protocol. Generally, HTTP server is only used to accept request from client and respond to client. Web framework supplies more useful and powerful utilities such as data validation or user authentication.
For example, when you shop online, you click on purchase button. Your browser will send a request to web server. Web server will parse request into data structure which web framework can understand. Web framework will verify your identity, add this product or user info into database and send a request to deduct money from your credit card and so on.
What we need mostly?
Nim programming language
Standard libraries: asyncdispatch
and asynchttpserver
.
Why Nim?
Nim is a statically typed compiled systems programming language. It has intuitive and clean syntax. Nim is efficient, expressive and elegant. Choose Nim and enjoy your life!
I love it because of three reasons:
- Indent and elegant syntax
- Static Type
- High performance
Simple usage of asynchttpserver
Nim has a built-in asynchronous HTTP server namely asynchttpserver
. You don’t need to install it, just type import asynchttpserver
to use it.
HTTP server helps us transfer contents(like HTML, Json and Text) to our client(like browser or curl). It will parse request into seq
or tables
in Nim language which our web framework can understand.
Let’s look at the code.
Http200
is a status code which tells client that everything is ok. “Hello World” is the text we responds to the client with. We also need to respond to the client with HTTP headers.
Now run the code below and enter localhost:8080
in your browser, you will get Hello World
on the screen.
# nim c -r thisfile.nim
import asynchttpserver, asyncdispatch
var server = newAsyncHttpServer()
proc cb(req: Request) {.async.} =
await req.respond(Http200, "Hello World", newHttpHeaders())
waitFor server.serve(Port(8080), cb)
Let’s look at request headers and response headers to get more intuitions.
request headers
GET / HTTP/1.1 # start line
Host: 127.0.0.1:8080 # request headers
......
response headers
HTTP/1.1 200 OK # start line
Content-Length: 11 # response headers
Now let’s extend out framework!
Serve static files
Sometimes we want to serve static files like CSS, HTML, Images and so on. It is wise to use Nginx. But for small applications, we just want to use ourselves’ framework.
In Nim, you only need to prepare strings and send strings to client. If you want to send HTML files, you read it and send it by socket. If you want to send Json text, first convert/dump it to strings and then send it by socket.
Now follow three steps:
Firstly, judge whether the file exists. Secondly judge whether we have access to files. Finally judge whether we can open files.
Let’s look at Pseudo code. resp
followed by contents we give to respond
proc in asynchttpserver
.
note: await must be used in functions
import asyncdispatch, asynchttpserver, os
proc serveStaticFile*(req: Request, dir, filename: string) {.async.} =
# exist -> have access -> can open
let path = dir / filename
# whether exists file
if not existsFile(path):
await req.respond(Http404, "File doesn't exist.", newHttpHeaders())
# whether has access to file
var filePermission = getFilePermissions(path)
if fpOthersRead notin filePermission:
await req.respond(Http403, "You have no access to the file.", newHttpHeaders())
# whether can open file
try:
let content = readFile(path)
await req.respond(Http200, content, newHttpHeaders())
except IOError:
await req.respond(Http404, "404 Not Found.", newHttpHeaders())
Basic Route
Route is used to map URL to corresponding proc which responds to client with contents.
For example:
This example demonstrates that web framework execute some actions such as fetch info from database, login, return HTML and so on according to corresponding HTTP method and URL.
- get 127.0.0.1:8080/hello -> send email
- get 127.0.0.1:8080/home -> fetch info from Database
import httpcore
proc findHandler*(httpMethod: HttpMethod, path: string) =
case httpMethod
of HttpGet:
case path
of "/hello":
echo "get 127.0.0.1:8080/hello"
# sendEmail()
of "/home":
echo "get 127.0.0.1:8080/home"
# fetchInfoFromDatabase()
else:
discard
of HttpPost:
case path
of "/hello":
echo "post 127.0.0.1:8080/hello"
# login()
of "/home":
echo "post 127.0.0.1:8080/home"
# returnHome()
else:
discard
else:
discard
Use Hash Map for static route
If client requests www.example.com/login
, our application will look up hash table to find corresponding Handler. Handler is the proc which handles request from client and generates response.
It is fast to look up URL in hash table.
type
Router* = Table[URL, procHandler]
Dynamic route
Sometimes we need dynamic URL to meet our demands. For example, we use only one procHandler
to handle login operations for different customers. So every customer will have different login URLs like www.example.com/login/1
, www.example.com/login/2
and so on.
We need URL match like www.example.com/login/{id}
to catch different ids.
First we split URL with /
, turn www.example.com/login/1
into @[“www.example.com”, “login”, “1”]
. Then we iterate hash table to decide whether URL match patterns.
Let’s look at Pseudo code.
let routeList = route.split("/")
# iterate all URL and procHandler pairs
let pathList = iterate(HandlerTable)
for idx in 0 ..< pathList.len:
# if match continue
# www.example.com => www.example.com
# login => login
if pathList[idx] == routeList[idx]:
continue
# match {id} => 2
if routeList[idx].startsWith("{"):
let key = routeList[idx]
if key.len <= 2:
raise newException(RouteError, "{} shouldn't be empty!")
let
params = key[1 ..< ^1]
Regex Route
We can use seq
to store regex route.
/post(?P<num>[\d]+)
=> /post521
or /post1314
and gets matched parameters such as num = 521
or num = 1314
.
type
ReRouter* = seq[(URL, ProcHandler)]
We just iterate URL and procHandler in Reroute to match request URL.
Let’s look at Pseudo code.
# find regex route
for (URL, ProcHandler) in reRouter:
if path.httpMethod != URL.httpMethod:
continue
var m: RegexMatch
# save matched params like id = 2
if URL.route.match(path.route, m):
for name in m.groupNames():
pathParams[name] = m.groupFirstCapture(name, URL.route)
Cookie
HTTP protocol is stateless. But we use plain Cookie to save user information.
Cookie is HTTP headers. It can carry user insensitive information. When you want to carry sensitive information, make sure encrypt it using Encryption Algorithm such as Sha-256, Sha-512 and so on. Only use Cookie with HTTPS to prevent it from man-in-the-middle attacks.
This is what cookie looks like. It is in the form of name-value
pair. Different pairs are separated by semicolon.
Cookie: username=flywind; age=21
Parse CookieJar
We use StringTable
to store different name-value
pairs.
type
CookieJar* = object
data: StringTableRef
proc parse*(cookieJar: var CookieJar, text: string) {.inline.} =
var
pos = 0
name, value: string
while true:
pos += skipWhile(text, {' ', '\t'}, pos)
# name = username
pos += parseUntil(text, name, '=', pos)
if pos >= text.len:
break
inc(pos) # skip '='
# value = flywind
pos += parseUntil(text, value, ';', pos)
# username = flywind
cookieJar[name] = move(value)
if pos >= text.len:
break
inc(pos) # skip ';'
Also we want to set cookie.
import options, times, strtabs, parseutils, strutils
type
SameSite* {.pure.} = enum
None, Lax, Strict
Cookie* = object
name*, value*: string # root => admin
expires*: string
maxAge*: Option[int]
domain*: string
path*: string
secure*: bool
httpOnly*: bool
sameSite*: SameSite
Set cookie
proc setCookie*(cookie: Cookie): string =
result.add cookie.name & "=" & cookie.value
if cookie.domain.strip.len != 0:
result.add("; Domain=" & cookie.domain)
if cookie.path.strip.len != 0:
result.add("; Path=" & cookie.path)
if cookie.maxAge.isSome:
result.add("; Max-Age=" & $cookie.maxAge.get())
if cookie.expires.strip.len != 0:
result.add("; Expires=" & cookie.expires)
if cookie.secure:
result.add("; Secure")
if cookie.httpOnly:
result.add("; HttpOnly")
if cookie.sameSite != None:
result.add("; SameSite=" & $cookie.sameSite)
Middleware
Middleware is used to execute some operations before procHandler and after procHandler. For example, we want to verify user identity, we can use middleware to do this work.
Let’s look at Pseudo code.
proc verify(user: User, next: Handler) =
# check whether user is valid
if not user.isValid:
resp 404, "You have no access to this URL."
return
# call next middleware or procHandler
next()
# If user is valid, print "Welcome" message
resp "Welcome, " & $user
There are two ways to implement middleware, one is based on function call, the other is based on hook.
Let’s look at hook way
We set before
and after
function type for every Middleware object.
type
Middleware = object
before: proc()
after: proc()
In the lifetime of application, assuming that we have two middlewares. So the order of execution is:
m1.before -> m2.before -> hello -> m2.after -> m1.after
Let’s look at Pseudo code.
proc hello() =
resp "Hello"
var app = Application()
var m1 = Middleware(...)
var m2 = Middleware(...)
app.addRoute(procHandler = hello, middlwares = [m1, m2])
Let’s look at Function call way
We use seq
to store middlewares. await next
is used to call next middleware or procHandler.
We have size
variable to get current middleware. In the beginning, size
is 0 and we get first middleware. Now we begin to execute the first middleware. When we encounter await next
. size
will increase by one, and we will call the second middleware. Finally we call the last middleware, and execute the rest program.
m1 -> m1.next -> m2 -> m2.next -> hello -> m2 -> m1
Let’s look at Pseudo code.
type
Middlewares* = seq[procHandler]
proc httpRedirectMiddleWare*() =
case request.scheme
of "http":
setScheme(request, "https")
of "ws":
setScheme(request, "wss")
else:
return
# Will call next middleware or procHandler
await next()
response.code = Http307
Exception Handler
We can map HTTP status codes to user-defined exception Handler.
Sometimes, we want to associate HTTP status code to unified pages for example custom 404 pages.
type
ErrorHandlerTable* = Table[HttpCode, ErrorHandler]
proc default404Handler*() =
response.body = errorPage("404 Not Found!", PrologueVersion)
app.errorHandlerTable.add(Http404, default404Handler)
Let’s look at Pseudo code.
if response.code in app.errorHandlerTable:
await (app.errorHandlerTable[response.code])()
More part
Regarding web framework, there are more parts than what I mentioned, you can checkout
Top comments (4)
Nice, but do you recommend
asynchttpserver
for a productive use, although the official documentation says the following?See nim-lang.org/docs/asynchttpserver....
I installed nim with snap and when im trying to compile first example im getting an error:
What i'm doing wrong?
Sorry, I don't know how to fix it. It seems something wrong with installation. You can follow these instructions. Maybe you can ask in Nim gitter or discord or Nim forum ? Nim community is very nice and I hope that you enjoy Nim.
Can anyone help me with how to serve a static directory with nim?