loading...
Cover image for Become a Fullstack JavaScript Developer, Part 3: The Backend

Become a Fullstack JavaScript Developer, Part 3: The Backend

hoangbkit profile image hoangbkit Updated on ・9 min read

This post is my curated (not exhausted) list of must-know backend related knowledge and technologies which a full stack (Javascript) developer should follow initially. In each category I picked top framework to catch up first then I'll learn more alternatives if needed.


Become a Fullstack JavaScript Developer is an original series published on avancedweb.dev where I explain my process and what I've learnt to become a full stack Javascript developer - from motivation to actual building my own product single-handedly.


Runtime (Node)

Node: is an asynchronous event driven open-source and cross-platform JavaScript runtime, designed to build scalable network applications. It has a unique advantage because millions of frontend developers that write JavaScript for the browser are now able to write the server-side code.

V8 Engine: Node runs the V8 JavaScript engine, the core of Google Chrome, outside of the browser.

Single Threaded: Node uses a single threaded model with event looping.

Asynchronous: performs non-blocking I/O operations (reading from the network, accessing a database or the filesystem)

Callback: is an asynchronous equivalent for a function. A callback function is called at the completion of a given task. Node makes heavy use of callbacks. All the APIs of Node are written in such a way that they support callbacks.

Event Loop: Node uses events heavily, there is generally a main loop that listens for events, and then triggers a callback function when one of those events is detected.

Event Emitter: All objects are instances of events.EventEmitter can emit events

File System: Node implements File I/O using simple wrappers around standard POSIX functions

CommonJS: Node implements CommonJS modules standard, ES Modules are coming behind a --experimental-modules flag for now.

RESTful (Express)

Express: one of the most simple yet powerful ways to create a web server. Its minimalist approach, unopinionated, focused on the core features of a server, is key to its success.

Routing: refers to determining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method (GET, POST, and so on).

Middleware: is a chunk or cluster of code that has access to a user’s request, the application’s response, and the next middleware to be used. With such an architecture, it becomes easy for Express js developers to add, remove, or modify various features to and from the application, giving high scalability to the application.

Template engine: enables you to use static template files in your application. At runtime, the template engine replaces variables in a template file with actual values, and transforms the template into an HTML file sent to the client. This approach makes it easier to design an HTML page.

GraphQL (Apollo)

GraphQL is a query language for your API, and a server-side runtime for executing queries by using a type system you define for your data. GraphQL isn't tied to any specific database or storage engine and is instead backed by your existing code and data.

GraphQL offers many benefits over REST APIs: clients have the ability to dictate exactly what they need from the server, receive that data in a predictable way, ability to retrieve many resources in a single request, strongly-typed which allows API consumers to know exactly what data is available and in what form it exists.

Apollo offers plenty of libraries for implementing an effective GraphQL tech stack for JavaScript applications, and their libraries are open-sourced to be more manageable. Apollo takes away all the boilerplate implementation that comes with the GraphQL reference implementation in JavaScript.

Apollo Server is a community-maintained open-source GraphQL server. It works with pretty much all Node.js HTTP server frameworks. Apollo Server works with any GraphQL schema built with GraphQL.js, so you may build your schema with that or a convenience library such as graphql-tools.

Relational Database (MySQL)

Relational database is a set of formally described tables from which data can be accessed or reassembled in many different ways without having to reorganize the database tables.

SQL (Structured Query Language): The standard user and application programming interface (API) of a relational database. SQL statements are used both for interactive queries for information from a relational database and for gathering data for reports.

Relational databases provide transactional integrity: A database transaction, by definition, must be atomic, consistent, isolated and durable. Transactions are available in most SQL database implementations, though with varying levels of robustness.

Relational databases provide consistency and availability but lack solid partitioning functionality, even though relational databases support partitioning, but due to the core concept of ‘Joins’ and other things like shared indexes, scaling them using partitions is very difficult and not optimal.

Relational databases have a strict schema for the data storage. Through the use of 'alter' statements, the schema can be changed, but it has its impact on the existing code ('application specific code'), which has to be changed in accordance with the changes made to the schema.

MySQL is the world's most popular open-source database: uses standard SQL, compiles on a number of platforms, ideal for both small and large applications, very fast, reliable, and easy to use.

NoSQL Database (MongoDB)

A NoSQL database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases.

NoSQL types: Document databases (MongoDB), Graph stores (Neo4J), Key-value stores (Redis), Wide-column stores (Cassandra)

NoSQL benefits: more scalable and provide superior performance compare to relational databases, dynamic schemas, automatic sharding, automatic replication and integrated caching

MongoDB is widely recognized as the leading NoSQL database. For developers and database administrators, MongoDB provides agility, scalability, performance and high availability.

Caching (Redis)

Caching is the term for storing reusable responses in order to make subsequent requests faster. There are many different types of caching available, each of which has its own characteristics. Application caches and memory caches are both popular for their ability to speed up certain responses.

Caching cases: in-memory data lookup, relational databases speedup, session store, token caching, website caching and fast access to any suitable data.

Caching types: object store, key-value store, native data structure store, in-memory, static file.

Caching algorithms: Least recently used (LRU), Time aware least recently used (TLRU), Most recently used (MRU), Random replacement (RR), Segmented LRU (SLRU), Least-frequently used (LFU), Least frequent recently used (LFRU)

Caching strategies: cache aside, read through, write through, write around, write back. In practice, carefully evaluate your goals, understand data access (read/write) patterns and choose the best strategy or a combination.

Caching benefits: decreased network costs, improved responsiveness, increased performance on the same hardware, availability of content during network interruptions.

Redis is a popular, open-source, in-memory data structure store that can be used as a database, cache, or message broker. Because it loads data from memory instead of from disk, Redis is faster than many traditional database solutions.

Message Brokers (RabbitMQ)

Message broker is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver.

Message brokers can do 4 important things: divide the publisher and consumer, store the messages, route messages, check and organize messages

There are self-deployed and cloud-based messaging tools: Apache Kafka, ActiveMQ, RabbitMQ, OMS, JMS, Redis, Service Bus ...

RabbitMQ is known as a “traditional” message broker, which is suitable for a wide range of projects. It is successfully used both for development of new startups and notable enterprises. The main advantage of this message broker is the perfect set of plugins, combined with nice scalability. I like RabbitMQ due to the opportunity to use many plugins. They save time and speed-up work.

Auth (JWT)

Authentication is about validating your credentials such as username and password to verify your identity. The system then checks whether you are what you say you are using your credentials.

Authentication factors determine the many different elements the system uses to verify one’s identity before granting the individual access to anything. Based on the security level, authentication factors can vary from one of the following: Single- Factor Authentication, Two- Factor Authentication, Multi- Factor Authentication

Authorization is the process to determine whether the authenticated user has access to the particular resources. It verifies your rights to grant you access to resources such as information, databases, files, etc.

JWT (JSON Web Token) is a JSON object that is defined in RFC 7519 as a safe way to represent a set of information between two parties. The token is composed of a header, a payload, and a signature.

JWT authentication is becoming very popular these days. The traditional authentication uses cookies and sessions. JWT is a type of token-based authentication. For every single request from a client to the server, a token is passed for authentication.

It's important to note that a JWT guarantees data ownership but not encryption: The JSON data you store into a JWT can be seen by anyone that intercepts the token, as it's just serialized, not encrypted.

I recommend that you store your JWT in cookies for web applications, because of the additional security they provide, and the simplicity of protecting against CSRF with modern web frameworks. HTML5 Web Storage is vulnerable to XSS, has a larger attack surface area, and can impact all application users on a successful attack.

Web Socket (Socket.IO)

WebSocket represents a long awaited evolution in client/server web technology. It allow a long-held single TCP socket connection to be established between the client and server which allows for bi-directional, full duplex, messages to be instantly distributed with little overhead resulting in a very low latency connection.

WebSocket is a thin transport layer built on top of a device’s TCP/IP stack. The intent is to provide what is essentially an as-close-to-raw-as-possible TCP communication layer to web application developers while adding a few abstractions to eliminate certain friction that would otherwise exist concerting the way the web works.

Socket.IO is a library that enables real-time, bidirectional and event-based communication between the browser and the server. Socket.IO is NOT a WebSocket implementation. Although Socket.IO indeed uses WebSocket as a transport when possible, it adds some metadata to each packet: the packet type, the namespace and the ack id when a message acknowledgement is needed.

Search Engine (Elasticsearch)

There are lots of ways to do search in your web application: via a back-end search engine such as Elasticsearch or Solr, commercial search services such as Algolia and AWS Cloudsearch, using a database with built-in search such as MySQL or MongoDB.

Full-text search, or FTS, is a technique used by search engines to find results in a database. You can use it to power search results on websites like shops, search engines, newspapers, and more.

Elasticsearch is used for a lot of different use cases: "classical" full text search, analytics store, auto completer, spell checker, alerting engine, and as a general purpose document store.

Elasticsearch is an overkilled solution for search functionality in your web application, it requires complicated installation and configuration. It is designed to deliver answers fast, really fast. To do this, most of the data structures It uses must reside in memory. To a large extent, It assumes you provide it with enough memory to do so.

My recommendation would be sticking with database built-in search when you're starting out, upgrading to a search service like Algolia to save time and cost when things getting a litle bigger. Finally coming up with an in-house search engine like Elasticsearch.

Testing (Jest)

Automated testing allows you to test your code frequently in a small amount of time. This means you can find problems and bugs before you actually push your code to production.

In general, there are three kinds of automated tests in software development: Unit Tests (Test a small unit of an application without external resources like a database), Integration Tests ( Test the application with all external resources in place.) and Functional / End To End Tests (Test the application through its User Interface).

Jest is a great JavaScript testing framework by Facebook. It’s often used for testing React components, but it’s also a pretty good general purpose testing framework.

Posted on by:

hoangbkit profile

hoangbkit

@hoangbkit

I turn ☕ into {code} using #javascript

Discussion

pic
Editor guide
 

Great article, you really hit a lot of the main technologies needed for a backend JS developer.

I hadn't thought about JWT, Search, Caching, and message brokers.

I guess they are the items I'll need to focus on when I'm a full stack JS dev.

I have heard about them through colleagues working on the backend with a Spring application.

It's pretty cool that lots of the technologies in use today are language agnostic.

 

Thanks for the article. Looking forward to reading DevOps you mentioned on a comment.
When I first saw the title, the first thing came up to my mind was this library.

GitHub logo hoodiehq / hoodie

🐶 The Offline First JavaScript Backend

hoodie

A generic backend with a client API for Offline First applications

Build Status Coverage Status Dependency Status devDependency Status

The Low-Profile Dog Hoodie Mascot

Hoodie lets you build apps without thinking about the backend and makes sure that they work great independent from connectivity.

This is Hoodie’s main repository. It starts a server and serves the client API Read more about how the Hoodie server works.

A good place to start is our Tracker App You can play around with Hoodie’s APIs in the browser console and see how it works all together in its simple HTML & JavaScript code.

If you have any questions come say hi in our chat.

Setup

Hoodie is a Node.js package. You need Node Version 4 or higher and npm Version 2 or higher, check your installed version with node -v and npm -v.

First, create a folder and a package.json file

mkdir my-app
cd my-app
npm init -y

Next…






Wonder your opinion on it, just out of curiosity, as less tech-savvy newbie.
 

hoodie is just a small layer which sync and expose your data into web localStorage. I think it's better to try strapi or parse-server which already exposed auto-generated GraphQL endpoint for you to use with very little effort.

 

Thank you very much for the explanation! It sounds interesting.
Thank you for understanding my shameless.

Honestly, after seeing people try to do front end with Jango in the Django Girls workshop, I became to have an opinion that every language has its best purpose.
Or in the opposite, every occasion has it's the most suitable language for the task. Not sure if it's the truth in general, but as a newbie, who effected a lot by the ease of use and learning.

That was the reason hoodie's tagline was super intriguing but sounds dangerous for me.

 

I really liked the rest of the series but this article contains some misleading poimts:

  • socket.io is super bloated, I heard even call it "the jquery of the back-end". Now browsers have native websocket support, and for the server you can use a much faster & smaller library such as ws.
  • Jwt: a lot of people told me not use it, I even found this gist explaining it in more details

Now for the personal opinions:

  • express has bad ts support, is not modular enough and doesn't support bidirectional comunication between middleware. On the other hand we have koa, a library which does all of that.
  • jest is cool as a tool except it's slow. Especially on windows. Back when I was using windows 10 I tried both mocha and jest, and even tho I tried following a lot of guides/tips on how to make jest faster, it still took 20s+ to run like 30 tests, the time increasing even more as I added more tests. Mocha on the other hand was blazing fast, running all of those in less than a second. On linux the difference still exists, but it'a much smaller
 

Very good article, thanks for sharing! I would add Docker to your list as well.

 

Thanks! I'm planning on putting Docker in part DevOps comming soon :D

 

thanks a lot for this great article, i really liked it and it adds lots of info to my knowledge.

It will be a nice of you, if you have an article about Docker, Kubrenetes and CI/CD

thanks in advance

Yes this mini series has 6 parts in total:

Become a Fullstack JavaScript Developer, Part 1: The Motivation (this post)
Become a Fullstack JavaScript Developer, Part 2: The Basics
Become a Fullstack JavaScript Developer, Part 3: The Backend
Become a Fullstack JavaScript Developer, Part 4: The Frontend (upcoming)
Become a Fullstack JavaScript Developer, Part 5: The DevOps (upcoming)
Become a Fullstack JavaScript Developer, Part 6: The Monorepo (upcoming)

I'll cover about what you're saying in DevOps part, stay tuned :D

 

Thanks for your articles, there are actually useful for JS beginners

 

Awesome could u recommend resources to help learn

 

I don't have specific list of resources, IMO it's better to determine the roadmap first then just look around to fill what you don't know!