What will you learn?
Understand how Javascript works in Rails. Learn to configure a Rails project with Stimulus, using esbuild to bundle your code efficiently and effectively. Set up a simple script to automatically load your Stimulus controllers and bundle your Javascript code when editing your code.
Introduction
Rails server-side HTML rendering enables the creation of powerful web interfaces without using javascript. For more complex cases which require it, Rails provides Stimulus, a minimalist framework to embed javascript in your views.
Both Rails and Stimulus aim to minimize your project's javascript code. This is desired since javascript significantly increases maintenance costs due to its web browser execution environment.
With multiple web browsers in the market, each with multiple versions, running in numerous OS of different devices, ensuring a javascript application works as expected on each platform is challenging. Compared to a Rails server, running on a specific OS on specific machines where developers can observe and diagnose errors, the javascript run environment is out of the developers' control.
To mitigate this problem, the javascript ecosystem provides tools to transpile and bundle code to maximize code compatibility and optimization. We are talking about bundlers such as esbuild and transpilers like Babel.
However, the more we dig into these solutions, the more we overload our stack with javascript code. On top of that, these tools are complex and require a good understanding of their purpose, how they work, and how to configure them properly. Testimony of this is the length of the present post.
This document is designed with newcomers in mind. To properly configure your javascript stack, it is important to understand its functioning and parts. Because of that, we will spend a significant part of the post explaining all these agents in an easy-to-understand way; to then put the theory into practice with the real configuration of a Rails project.
If you are already familiar with bundlers and are just seeking a solution to integrate Stimulus and esbuildin your project, I recommend you to jump to the solution section or even to the PR links where the final configuration is available and can be easily integrated into any other Rails project.
The Basics
How does Javascript work in Rails?
A Rails web application is an application with a web-based interface. Web interfaces are rendered in web browsers.
When we open a Rails application in our web browser, the navigator performs HTTP requests to Rails backend server in HTML format. In response, the server returns an HTTP response in HTML format. This response is rendered by the web browser, displaying the application interface.
The server generates the HTTP response content dynamically based on the request parameters. It consists of a standard HTML document. This document comprises a body section, which defines the layout of the interface (what we see on the screen), and a head section.
The head contains document meta information (name, viewport configuration, etc.) and links. These links point to static assets the application requires to render the document correctly. For example, the .css code to style our HTML or, more relevant for us, the javascript code to run our front-end business logic. Per each link present, the browser will perform an additional HTTP request to download each asset and load it. Javascript assets (.js files) are loaded by executing their inner code in a similar way we could do manually by copying the file content and pasting it into the javascript console of the web browser.
Unlike server HTML responses, asset requests are static: they always return the same content. This allows us to generate our application asset responses (the asset files) once and use them in all asset requests (this is why asset files are precompiled, generated, before launching a Rails application).
The Rails asset pipeline generates the assets. The process is simple: the pipeline processes all the asset files in the assets folder, generates the final asset files, and places them under the public folder. This folder is, as the name says, public. Web browsers can access the resources available via the link tags in the HTTP responses' head.
The Javascript code of a Rails application is located in two folders: the app/javascript folder and the node_modules folder, which contains our javascript project dependencies. To serve our javascript code as a .js asset, we must pack all this code in a set of self-contained files that the asset pipeline can process and the web browsers load. We need to bundle our javascript code.
Javascript bundling in Rails
Quoting the definition from Next.js: Bundling is the process of resolving the web of dependencies and merging (or ‘packaging’) the files (or modules) into optimized bundles for the browser. We can bundle our javascript projects with javascript bundlers.
Most of the bundlers work similarly. They provided an API with methods to generate bundles and optional parameters for customized outputs. Depending on the tool you use, these sets of parameters might vary. Most of them can also extend their capabilities with additional plugins.
When selecting a bundler for your project, there are two key aspects to consider:
Speed: the faster the bundler bundles your code, the better. In production environments, where the code does not change, speed is irrelevant (you bundle your code once). In development, however, you want to test changes in your code as fast as possible. Since you need to re-bundle all javascript code on each change for it, having a fast bundler is critical for a smooth development experience.
Optimization: bundlers use multiple optimization techniques to keep your bundle small and to take the most advantage of web browser caching capabilities. The bundle is an asset your application will serve in every HTTP request: smaller bundles and the use of caching lead to faster load times and smoother end-user experiences per request.
Rails applications are bundler-agnostic. They do not care how you bundle your javascript code. It just expects whatever comes from the bundler to be placed under app/assets, so the asset pipeline processes it. We can see this in the official jsbundling-rails gem, which consists of scripts to install different bundlers and configure a default npm build command to generate our bundles—no interaction whatsoever with the Rails configuration. This black-box bundler logic allows us to change and update our bundler system without tuning any other aspect of our Rails application.
So to bundle our javascript code in Rails we need to install a bundler, run it, and ensure the resulting bundle is placed under any of the asset paths configured in our application so that the asset pipeline can digest it. Understanding this, the question is now, what do we bundle?
Bundling Stimulus
Stimulus is a minimalist Javascript framework. It can be classified in the same category as other frameworks, such as React, Vue, or Ember. These ones are complex tools with multiple modules and utilities to build complex web applications: components, routers, HTML template engines, services, etc. Stimulus omits all these framework parts and relies on already rendered HTML (server-side HTML), and HTML data attributes to attach simple javascript logic to it.
It works with controllers, javascript classes that can be attached to any part of our page. When a Stimulus application is launched, controller classes must be registered with an identifier. The application will observe changes in the page HTML. When an element includes a data-controller attribute with a specific identifier, the application will identify the corresponding controller class and generate an instance of it attached to the element. This approach makes it significantly easy to reuse our javascript logic in our views since it is decoupled from our HTML (in comparison with traditional component-oriented approaches from other frameworks).
Since Rails renders HTML in the backend, Stimulus is the best solution to sprinkle our views with simple yet powerful javascript code. Controllers are placed under the app/javascript/controllers folder. To run the Stimulus application, we define a javascript initialization script. This is usually done under app/javascript/application.js. This script creates an instance of a Stimulus application and then registers a list of imported controllers.
We must pass this application.js file to our bundler to bundle a Stimulus application since it contains the reference to our controllers and the logic of how the application can be launched. The bundler will go from there through all the imported controllers and their specific dependencies to generate a bundle containing all the required code to run the application in the browser.
And let’s not forget about the transpiler!
We have already introduced all the key concepts to start working with javascript in our Rails application. However, there is still a missing piece in our puzzle to bundle ready-for-production javascript code in our applications: the transpiler.
Many Rails developers consider that the less code you have in your Rails applications, the better. The main reason to think this is that javascript can be hard to test.
Javascript web-browser execution environment requires browser-makers to keep their browsers updated with the latest ECMAScript specification releases. This means that newer (and not so new) released javascript language features might not be compatible with some commercial web browsers or even behave differently from the specification. As a result, developing javascript code that runs in all web browsers can be challenging. Imagine going one by one testing your code not just in all commercial web browsers but in different versions of each one (which might have different implementations of the javascript language).
To solve part of this problem, we can use transpilers. A transpiler rewrites javascript code targeting a specific ECMAScript version. By doing so, we can take advantage of the latest javascript features without compromising the browser compatibility of our code. Transpilers use polyfills, which specify how new javascript features can be translated into javascript code compatible for older versions.
In our case, we will use Babel with CoreJS to transpile our code. This transpiler can be attached to most of the bundlers, running the transpiling process in parallel with the bundling, generating both highly optimized and compatible javascript code.
The Problem
Configure a Rails application with Stimulus, esbuild and babel.
The Solution
esbuild
We will use esbuild in our project. esbuild is a bundler that stands for its speed. It does not have the same level of maturity or popularity as other available bundlers, such as webpacker, and it still misses some useful optimization options (such as code splitting, which is still in the project roadmap at the time I write this post). Still, its growing popularity and blazing-fast bundling times are reasons enough to choose it as our preferred option.
Start by installing esbuild in your project. Open a terminal window, navigate to your Rails project, and install the esbuild package:
yarn add esbuild
Once installed, the esbuild CLI should be available:
# In your terminal window running this command:
esbuild --version
# Should return the installed esbuild version number
0.15.10
We can bundle javascript code with esbuild --bundle. It requires two parameters: entryPoints, the list of javascript files we want to bundle, and outdir, the directory path where the bundler outcome will be placed.
# Produces dist/entry_point.js and dist/entry_point.js.map
esbuild --bundle entry_point.js --outdir=dist
The command has multiple optional parameters to optimize the resulting bundle to your needs. You can find the entire list in the official documentation. To properly configure the bundle with the right options, it is important to understand how each customization option works and what your needs are.
For example, minify optimizes your bundle size using different compression techniques, like removing whitespaces from code. In production environments, the minify option is a must-use; we want to reduce the weight of our assets as much as possible. In development, however, removing whitespace leads to code that is harder to read and thus more difficult to debug, so you should not use this option.
A good way to learn about the different options and their outcomes is also by experimenting with them. Let’s start by bundling something simple. We will set up the javascript entry point for our application, consisting of a simple script that prints the current date-time in an infinite loop in the console:
// app/javascript/application.js
// Entrypoint for our Javascript code. It might already exists in your project.
// This is just a simple example to show how bundling works.
console.log('Loading Javascript asset')
function printTime() {
console.log(new Date())
}
function setTimeLogger() {
setInterval(() => printTime(), 1000)
}
console.log('Starting script')
setTimeLogger()
And now, let’s bundle it. We will point the bundle directory to the app/assets folder, as the Rails asset pipeline requires:
esbuild --bundle app/javascript/application.js --outdir=app/assets/builds --minify
app/assets/builds/application.js 200b
app/assets/builds/application.js.map 536b
⚡ Done in 14ms
The resulting bundle looks like this:
(()=>{console.log("Loading Javascript asset");function o(){console.log(new Date)}function e(){setInterval(()=>o(),1e3)}console.log("Starting script");e();})();
The minify option removes all line breaks and whitespace and renames functions and variables with shorter names. This results in a significantly smaller bundle than the one generated without the minification option:
esbuild --bundle app/javascript/application.js --outdir=app/assets/builds
app/assets/builds/application.js 278b
⚡ Done in 4ms
At the same time, this refactoring makes it extremely hard to read the code and map variables and methods from the bundle to the actual code. If there are bugs in our code, error messages will be hard to understand and trace back in our code. Remember, in case of doubt, to play with the different esbuild parameters and inspect the resultant bundle when exploring new bundle optimizations in your project.
The bundle generated is ready to be processed by the pipeline and loaded by the web browser. If you want to test how it works on the browser, you can always copy its code and paste it into the browser console:
To automate bundling, we can define a npm build command that runs esbuild.
// package.json
...
"scripts": {
...
"build:esbuild": "esbuild --bundle app/javascript/application.js --outdir=app/assets/builds --minify --sourcemap",
...
...
This makes it possible to bundle our code by running the command yarn build:esbuild in our terminal, which is way better than typing the previous esbuild command. Still, this bundling system is not optimal. The CLI approach can lead to very long bundle commands, hard to read and maintain. Instead, we can use the esbuild javascript API to define our bundle script in javascript, way easier to work with:
// config/esbuild.mjs
import path from 'path'
import esbuild from 'esbuild'
import rails from 'esbuild-rails'
import babel from 'esbuild-plugin-babel'
esbuild
.build({
bundle: true,
// Path to application.js folder
absWorkingDir: path.join(process.cwd(), 'app/javascript'),
// Application.js file, used by Rails to bundle all JS Rails code
entryPoints: ['application.js'],
// Destination of JS bundle, points to the Rails JS Asset folder
outdir: path.join(process.cwd(), 'app/assets/builds'),
// Enables watch option. Will regenerate JS bundle if files are changed
watch: process.argv.includes('--watch'),
// Split option is disabled, only needed when using multiple input files
// More information: https://esbuild.github.io/api/#splitting (change it if using multiple inputs)
splitting: false,
chunkNames: 'chunks/[name]-[hash]',
// Remove unused JS methods
treeShaking: true,
// Adds mapping information so web browser console can map bundle errors to the corresponding
// code line and column in the real code
// More information: https://esbuild.github.io/api/#sourcemap
sourcemap: process.argv.includes('--development'),
// Compresses bundle
// More information: https://esbuild.github.io/api/#minify
minify: process.argv.includes('--production'),
// Removes all console lines from bundle
// More information: https://esbuild.github.io/api/#drop
drop: process.argv.includes('--production') ? ['console'] : [],
// Build command log output: https://esbuild.github.io/api/#log-level
logLevel: 'info',
// Set of ESLint plugins
plugins: [
// Plugin to easily import Rails JS files, such as Stimulus controllers and channels
// https://github.com/excid3/esbuild-rails
rails(),
// Configures bundle with Babel. Babel configuration defined in babel.config.js
// Babel translates JS code to make it compatible with older JS versions.
// https://github.com/nativew/esbuild-plugin-babel
babel()
]
})
.catch(() => process.exit(1))
This is an example of our project’s esbuild bundle configuration. As you can see, we can use javascript to add comments and keep the bundle options better organized. Via process.argv we can use different option values based on custom build parameters. This can be useful to define multiple configurations in the same code (for example, enabling or disabling minification in development or production). Please make sure you define your bundle configuration in a .mjs file. This is required since the script uses the import method to require the esbuild dependencies.
With the javascript code set, we can now update our build script in our package.json file:
// package.json
...
"scripts": {
...
"build:esbuild": "node config/esbuild.config.mjs",
...
...
You can now run yarn build:esbuild again to check the configuration works properly and the code is bundled.
Babel
Babel is the most popular Javascript transpiler. The transpiler refactors our bundle code to maximize its compatibility with web browsers. To install Babel, go to your project folder, open the terminal and run:
# in your terminal window:
yarn add --dev core-js @babel/core @babel/preset-env esbuild-plugin-babel
We listed multiple packages here. Let’s quickly describe their purpose:
@babel/core: the official Babel npm package.esbuild-plugin-babel: this package allows us to integrate theBabeltranspiling process in theesbuildpipeline. We already imported and used this package in ouresbuildconfiguration file:
// config/esbuild.mjs
...
plugins: [
...
// Configures bundle with Babel. Babel configuration defined in babel.config.js
// Babel translates JS code to make it compatible with older JS versions.
// https://github.com/nativew/esbuild-plugin-babel
babel()
]
...
@babel/preset-env:Babeluses repositories of plugins and configuration options to transpile javascript code. These repositories are called presets. The preset-env module is an official Babel preset for compiling ES2015+ syntax.core-js: this is a special kind of preset, a polyfill. The polyfill contains a library of javascript features and defines the code refactor needed for each one to make it compatible with older javascript versions.
Since we already told esbuild to use Babel, we need to configure Babel. To do so, create a babel.config.js file in your project root and import the defined presets:
const presets = [
[
// Javascript 2015+ syntax transpiler
'@babel/preset-env',
// Javascript Polyfill
{
useBuiltIns: 'usage',
// Make sure version number matches the Core.js version installed in your project
corejs: '3.28.0'
}
]
]
module.exports = {
presets,
// Fixes Core-JS $ issue: https://github.com/zloirock/core-js/issues/912
exclude: ['./node_modules']
}
Presets can be configured with multiple optional parameters. Similar to what we mentioned for esbuild, we encourage you to read the docs and find the setup that adapts best to your needs.
To finish the configuration, we must tell Babel the web browsers we want our code to be compatible with. Create a new file called .browserslist.rc and specify your desired browser compatibility. You can find all the possible ways to specify the list of compatible browsers in the official docs. We will use a simple configuration for our project:
# .browserslist.rc
# Babel Preset configuration
# --------------------------
# Defines web-browser compatibility parameters for Babel to transpile your JS code.
# This configuration is used by babel.config.js.
# More information in here.
# https://github.com/browserslist/browserslist
# Support browsers with a market share higher than 5%
>10%
With everything set, you can run esbuild and see how Babel transpiles your code. To do a little experiment, we will update our script with a new method that uses a technique called destructuring assignment to define its parameter. Its syntax is not available in some older javascript versions:
// app/javascript/application.js
...
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment
function destructuringAssignment({ param }) {
return param
}
...
Now compiling our bundle, we can see how the method is refactored. The new code does the same destructuringAssigment does. However, its syntax is now compatible with all web browsers which do not implement this kind of assignment:
(() => {
...
function destructuringAssignment(_ref) {
let {
param
} = _ref;
return param;
}
...
})();
//# sourceMappingURL=application.js.map
Stimulus
Rails provides multiple commands and generators to create applications with Stimulus pre-installed or install the framework from scratch. In this post, we will configure stimulus from scratch without using those helpers.
Let’s start installing the stimulus package. Open a terminal window in your project folder and execute:
yarn add @hotwired/stimulus
Let’s add a sample controller. We will use the example from the official Stimulus documentation. In a Rails application, Stimulus controllers are placed under the app/javascript/controllers folder. Each controller file must be suffixed with_controller.js (it is a convention).
// app/javascript/controllers/hello_controller.js
import { Controller } from '@hotwired/stimulus'
export default class extends Controller {
connect() {
this.element.textContent = 'Hello World!'
}
}
Now we need to create the launch script. This script is responsible for creating a Stimulus application instance, registering all our project controllers, and launching the application in the web browser. The official Stimulus documentation also provides an example of how this script should look like. Since the script launches the application, we should place the code in application.js:
// app/javascript/application.js
import { Application } from "@hotwired/stimulus"
import HelloController from "./controllers/hello_controller"
// Creates & launches a Stimulus application instance
window.Stimulus = Application.start()
// Registers controller in Stimulus application instance
Stimulus.register("hello", HelloController)
Each controller in our project must be registered in the Stimulus application instance with an identifier. We chose the identifier of a controller by its file and path name, excluding the _controller.js filename suffix.
This script has multiple problems. Registering our controllers one by one can be tedious: any time a new controller is created, deleted or renamed, we need to update the file. On top of that, the idea of manually writing the identifiers can lead to multiple problems: what happens if someone makes a type in the identifier? What happens if different developers use different naming conventions?
We can solve this problem by automating the controller registering process with a script: the script could look into the controllers folder, get the list of controller files, import the respective classes defined, and register them with the identifier corresponding to the file name. Luckily, the esbuild-rails package already provides the resources to implement this script.
Installing the package first:
yarn add esbuild-rails
And then using its esbuild plugin in our bundle configuration:
// config/esbuild.mjs
...
plugins: [
...
// Plugin to easily import Rails JS files, such as Stimulus controllers and channels
// https://github.com/excid3/esbuild-rails
rails()
]
...
We can now define our script to import our controllers automatically:
// Entry point for the build script in your package.json
import '@hotwired/turbo-rails'
import { Application } from '@hotwired/stimulus'
// General Controllers
// -------------------
// import all Stimulus controller files under the controllers folder
import controllers from './**/*_controller.js'
// Auxiliary Methods
// -----------------
// Infer Stimulus controller name from its file
function controllerName(defaultName) {
const namespaces = [
...new Set(
defaultName
.split('--')
.filter((ns) => !['..', 'controllers'].includes(ns))
)
]
return namespaces.join('--')
}
const application = Application.start()
// Set flag to true to get debbug information in the web browser console
application.debug = true
window.Stimulus = application
controllers.forEach((controller) => {
Stimulus.register(controllerName(controller.name), controller.module.default)
})
Bundling our javascript code, we can see how the controllers are attached to the bundle, indicating we successfully integrated Stimulus in our Rails application:
// app/assets/builds/application.js
...
// We can see our controllers & the initialization script bundled
application.debug = true;
window.Stimulus = application;
controller_default.forEach((controller) => {
Stimulus.register(controllerName(controller.name), controller.module.default);
});
controller_default2.forEach((controller) => {
Stimulus.register(componentControllerName(controller.name), controller.module.default);
});
})();
...
Finally, copying the bundle file content into our web browser console and with the application.debug flag set to true; we can see how the Stimulus application is launched. If then, we type window.Stimulus in the console, we can access the application instance object:
Last touches
We already explained how to bundle our javascript code. But we are missing the last part: linking it in our HTML.
To attach a link to our application.js asset we need to use the javascript_include_tag helper in app/views/layout/application.html. This file defines a wrapper for all the views in our application. By placing the javascript helper in the head section, we make sure every HTML formatted HTTP response includes in its head a link to the application javascript asset:
app/views/layout/application.html.haml
head
...
= javascript_include_tag("application", media: "all", "data-turbolinks-track" => true)
...
The helper renders a link tag pointing to the corresponding .js asset. By setting the name parameter to application, we point to our generated bundle.
Once this is set, you can start developing and testing your javascript logic. Remember to run the yarn build:esbuild command with the --watch option. This will enable the esbuild watch flag, which triggers automatic bundles when you change your controller files.
yarn build --watch --development
To deploy your application, you need to run the asset pipeline before launching your Rails application:
# In a terminal window
bundle exec rails assets:precompile
The command will try to bundle the javascript code by running the build script. Since we only defined the script for development, we need to add the missing one for production, so the pipeline can generate the bundle properly:
# package.json
"scripts": {
...
"build": "yarn build:esbuild --production",
...
...
Running the pipeline, we will see the command being executed, the bundle being generated and placed under the public folder, from which web browsers will request the asset:
bundle exec rails assets:precompile
yarn install v1.22.19
[1/4] 🔍 Resolving packages...
success Already up-to-date.
✨ Done in 0.29s.
yarn run v1.22.19
$ yarn build:esbuild --production
$ node esbuild.config.mjs --production
Dynamic import can only be supported when transforming ES modules to AMD, CommonJS or SystemJS. Only the parser plugin will be enabled.
app/assets/builds/application.js 150.7kb
✨ Done in 1.75s.
yarn install v1.22.19
[1/4] 🔍 Resolving packages...
success Already up-to-date.
✨ Done in 0.34s.
yarn run v1.22.19
$ yarn build:tailwind-config-viewer && yarn build:tailwind
$ tailwind-config-viewer export ./public/tailwind-config-viewer -c ./config/tailwind.config.js
$ npx tailwindcss --postcss -i ./app/assets/stylesheets/application.tailwind.scss -o ./app/assets/builds/application.css -c ./config/tailwind.config.js --minify
Rebuilding...
Done in 838ms.
✨ Done in 3.07s.
I, [2023-02-25T23:36:12.163440 #5705] INFO -- : Writing /Users/alberto-mac/Desktop/Projects/Albert/Albert/public/assets/application-dd70b2d6bf41af42627f148cd9a084b6c0a0b0961780833472ec6faf19b199e7.js
I, [2023-02-25T23:36:12.163717 #5705] INFO -- : Writing /Users/alberto-mac/Desktop/Projects/Albert/Albert/public/assets/application-dd70b2d6bf41af42627f148cd9a084b6c0a0b0961780833472ec6faf19b199e7.js.gz
Conclusions
Let’s try to wrap up all the lessons of this post. Javascript code is embedded in Rails views as assets via link tags in HTML response heads. To turn our javascript code into assets, we need to bundle it first, so Rails can process the resulting bundle in the asset pipeline and serve it to the web browser.
We can bundle our code with javascript bundlers such as esbuild. The bundler merges all our code with its dependencies in optimized files. These optimizations can be configured via different bundling parameters and third-party plugins, allowing us to shrink the size of our bundles and maximize their compatibility with web browsers, among other possibilities.
Finally, with Stimulus, Rails developers can write powerful javascript code with a simple framework that relies heavily on Rails server-side rendering to keep the entire javascript stack of your project as minimal as possible.
Remember always to keep your javascript code to the minimum possible and rely on CSS capabilities for simple UI interfaces. The disparity of javascript implementations present on the different web browsers in the market can make it difficult to build and test code that works in all possible scenarios.
And last but not least: read, experiment, and get familiar with the bundlers, transpilers, and other additional tools you find on your way to configuring your Rails application. It can be an overwhelming task due to all the different technologies involved and how complex they are, but definitely, the effort will pay off.
Code in Github
References
Any Questions?
If you have reached this far, thank you so much for reading my article. It has been quite challenging to wrap up so many concepts, tools, and ideas in a single post, and I am pretty sure there is still much I can improve. This is why I keep writing more and more (apart from the fun, of course).
Did you find the post clear enough? Or did you get lost or stuck at any point? As always, feel free to write down in the comments or reach me for any support you need to set up your own bundling process. I will be happy to give a hand if possible.
Best regards, and see you in the next post!
Top comments (0)