Ben Halpern

Posted on May 3, 2020

One thing led to another and I built my own static site generator today

#webdev #javascript #ruby #html

I started by building a static site as a small side project for my brother—but then I wanted partials... and regression tests. I thought partials which could help me inline the CSS and JS tags while breaking the code up into different files for organizational purposes in development. I like to inline the assets to avoid render-blocking latency on simple landing pages which will likely be served over unreliable network conditions.

At first I really didn't think I needed a generator at all, but one thing led to another and I kind built a basic one myself.

It consists of a build.rb file that looks like this...



prod_build = ARGV[0] == "for_prod"

# Read files
meta_html =       File.open("workspace/meta.partial.html").read
style_css =       File.open("workspace/style.partial.css").read
body_html =       File.open("workspace/body.partial.html").read
json_data =       File.open("workspace/data.json").read
scaffold_js =     File.open("workspace/scaffold.partial.js").read
dynamic_js =      File.open("workspace/dynamic.partial.js").read
analytics_html =  File.open("workspace/analytics.partial.html").read
base_html =       File.open("workspace/base.html").read
test_html = ""

unless prod_build
  test_html = File.open("workspace/test.dev.html").read
end

# Create built page
build_string = base_html
  .gsub("{{ meta }}", meta_html)
  .gsub("{{ style }}", style_css)
  .gsub("{{ html }}", body_html)
  .gsub("{{ data }}", json_data)
  .gsub("{{ scaffold_script }}", scaffold_js)
  .gsub("{{ dynamic_script }}", dynamic_js)
  .gsub("{{ analytics }}", analytics_html)
  .gsub("{{ test }}", test_html)

# Write to target page

if prod_build
  puts "Production build.... index.html"
  File.write("index.html", build_string)
else
  puts "Development build.... wip-index.html"
  File.write("wip-index.html", build_string)
end

I could DRY up this code, but I prefer it to be dumb and super explicit at this stage.

As you can see, this is just basic string find and replace. {{ could just as easily have been 💩💩 or [cromulent >>. It's completely arbitrary, but {{}} looked fancy.

base.html looks like this...



<html lang="en">
  <head>
    {{ meta }}
    <style>
      {{ style }}
    </style>
  </head>
  <body>
    {{ html }}
    <script>
      // Data
      var data = {{ data }}

      // Code
      {{ scaffold_script }}
      {{ dynamic_script }}
    </script>
    {{ analytics }}
    {{ test }}
  </body>
</html>

...I even wrote my own dependency-free JavaScript test suite. I'll share more once it's further along.

I probably should have reached for an existing static site generator instead of doing this from scratch, so why did I take this approach?

In all seriousness, I generally like to avoid dependencies when doing projects like this so it's easier to hop in for a quick change in the future without having to install a bunch of old dependencies. Building a whole toolchain myself is sort of silly, but fun!

If you don't want to be like me, you may want to check out this great thread...

What is the simplest static site framework?

Ben Halpern ・ Feb 7 '20

#discuss #html #webdev

Happy coding!

Top comments (28)

Simon Landry • May 3 '20

I like this approach because, even though you're most likely reinventing the wheel, you often learn new stuff along the road and learning is the most important thing in our industry. Hats off to you.

Ben Halpern • May 3 '20

😊

Thai Pangsakulyanont • May 3 '20 • Edited

I agree about keeping dependencies low in small projects! When I hop between multiple projects, I can feel the context switching cost if there are setups involved.

By the way, this looks like a great use-case for…

<html lang="en">
  <head>
    <?php include 'workspace/meta.partial.html'; ?>
    <style>
      <?php include 'workspace/style.partial.css'; ?>
    </style>
  </head>
  <body>
    <?php include 'workspace/body.partial.html'; ?>
    <script>
      // Data
      var data = <?php include 'workspace/data.json'; ?>;

      // Code
      <?php include 'workspace/scaffold.partial.js'; ?>;
      <?php include 'workspace/dynamic.partial.js'; ?>;
    </script>
    <?php include 'workspace/analytics.partial.html'; ?>
    <?php if (isset($_GET['test'])) include 'workspace/test.dev.html'; ?>
  </body>
</html>

# Launch a development web server
$ php -S 0.0.0.0:1234

# View production build
$ open http://localhost:1234

# View test build
$ open http://localhost:1234/?test

# Build static site
$ php index.php > index.html

Ben Halpern • May 3 '20

I haven’t used PHP in years... and this seems very appealing!

Sakis bal • May 4 '20

What stack do you primarily use? Thanks for your response beforehand.

Sean Allin Newell • May 4 '20

Blasphemy! Beautiful blasphemy!

Thai Pangsakulyanont • May 4 '20

BlasPHPemy!

Filipe Herculano • May 3 '20

That’s awesome!!! 👏

I’ve been there too, I once was tinkering with markdown parsing when I built this small library (link below) and it turned out that the live demo I made of using it could be seen as a somewhat static site generator from markdown files

I also felt kinda silly initially but now I want to revisit it as it was kinda fun too (and also try building a dependency free one from scratch like yours)

this-fifo / use-marked-hook

A react hook for parsing markdown with marked and sanitize-html

useMarked() hook

A react hook for parsing markdown with marked and sanitize-html

Live Demo

The app located at /example demonstrates how it could be used, see the live result at this-fifo.github.io/use-marked-hook/

Install

yarn add use-marked-hook

Usage

import React from "react";
import { useMarked } from "use-marked-hook";

const App = () => {
  const markdown = `**bold content**`;
  const html = useMarked(markdown);
  // html -> <p></strong>bold content</strong></p>
  return <div dangerouslySetInnerHTML={{ __html: html }} />;
};

License

MIT © Filipe Herculano

View on GitHub

Ben Halpern • May 3 '20

Neat!

Giorgos Kontopoulos 👀 • May 3 '20 • Edited

I had built a simple static site generator back before CMSs were a thing and SSGs had a name for themselves. I believe in those days everyone working on web development had something similar for organizing site development right ?

Phil Ashby • May 3 '20

v2 of my personal website was templated using Apache Velocity and some home-brew wrapper code to pick up resources from disk... I thought that was a silly idea back in about '05 and replaced it with a MoinMoin wiki. Now I'm back to Hugo :-/

Andrew Brown 🇨🇦 • May 3 '20 • Edited

This is mine. I decided to use erb since its not really much different than using handlebars.
Best part is if you don't use any ruby's gems other than standard easy to put on a lambda and use CodeBuild and CodePipeline to automatically deploy changes to S3 Static Website Hosting.

For my use case I have to compile at least a thousand pages and doing it this way is under a 1 minute

require 'erb'
require 'json'
require 'fileutils'

class Namespace
  def initialize(hash)
    hash.each do |key, value|
      singleton_class.send(:define_method, key) { value }
    end
  end

  def get_binding
    binding
  end
end

class Generator
  def self.render src_path, data={}
    html = ERB.new File.read(src_path), nil, '-'
    ns = Namespace.new data
    html.result(ns.get_binding)
  end

  def self.render_file src_path, build_path, data={}
    html = ERB.new File.read(src_path), nil, '-'
    ns = Namespace.new data
    File.open build_path, 'w' do |f|
      f.write html.result(ns.get_binding)
    end
  end

  def self.src_path name
    cwd = File.dirname __FILE__
    File.join cwd, 'src', "erb/#{name}.html.erb"
  end

  def self.build_path name
    puts "+ build/#{name}.html"
    cwd = File.dirname __FILE__
    path = File.join cwd, 'build', "#{name}.html"
    if File.exists?(path)
      File.delete path
    else
      FileUtils.mkdir_p File.dirname(path)
    end
    path
  end

  def self.json_data name
    cwd = File.dirname __FILE__
    path = File.join cwd, 'data', "#{name}.json"
    json = File.read(path)
    JSON.parse(json)
  end

  # when dealing with a file where each line is json
  def self.json_array_data name
    cwd = File.dirname __FILE__
    path = File.join cwd, 'data', "#{name}.json"
    data = []
    File.foreach(path).with_index do |line, line_num|
      data << JSON.parse(line)
    end
    data
  end
end

class PrepAnywhereGenerator < Generator
  def self.head
    src_path   = self.src_path('head')
    data       = {}
    self.render src_path, data
  end

  def self.header
    src_path   = self.src_path('header')
    data       = {}
    self.render src_path, data
  end

  def self.footer
    src_path   = self.src_path('footer')
    data       = {}
    self.render src_path, data
  end

  # /
  def self.homepage
    src_path   = self.src_path('homepage')
    build_path = self.build_path('index')
    data       = {
      head: self.head,
      header: self.header,
      footer: self.footer,
      data: self.json_data('homepage')
    }
    self.render_file src_path, build_path, data
  end

  # /textbooks
  # /textbooks/us
  # /textbooks/canada
  def self.textbooks
    src_path = self.src_path('textbooks')

    textbooks_all   = self.json_data('textbooks')
    textbooks_ca    = self.json_data('textbooks-ca')
    textbooks_us    = self.json_data('textbooks-us')

    # Canada Books
    build_path_all = self.build_path('textbooks/index')
    build_path_ca  = self.build_path('textbooks/ca')
    build_path_us  = self.build_path('textbooks/us')
    data = {
      head: self.head,
      header: self.header,
      footer: self.footer,
    }

    self.render_file src_path, build_path_all, data.merge({body_class: 'textbooks-all', data: textbooks_all})
    self.render_file src_path, build_path_us , data.merge({body_class: 'textbooks-us', data: textbooks_us })
    self.render_file src_path, build_path_ca , data.merge({body_class: 'textbooks-ca', data: textbooks_ca })
  end

  # /textbooks/:book
  def self.textbook data_key='textbook'
    src_path = self.src_path('textbook')
    books = self.json_array_data(data_key)
    books.each do |book|
      path = [
        'textbooks',
        book['permalink'],
        'index'
      ].join('/')
      build_path = self.build_path path
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        book: book
      }
      self.render_file src_path, build_path, data
    end
  end

  # /textbooks/:book/chapters/:chapter/materials/:material
  def self.material data_key='material'
    src_path = self.src_path('material')
    materials = self.json_array_data(data_key)
    materials.each do |material|
      build_path = self.build_path(self.material_path(material))
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        material: material
      }
      self.render_file src_path, build_path, data
    end
  end

  def self.material_path material
    [
      'textbooks',
      material['textbook_permalink'],
      'chapters',
      material['chapter_permalink'],
      'materials',
      material['permalink'],
      'index'
    ].join('/')
  end

  # /textbooks/:book/chapters/:chapter/materials/:material/videos/:video
  def self.video data_key='video'
    src_path = self.src_path('video')
    videos = self.json_array_data(data_key)
    videos.each do |video|
      build_path = self.build_path(self.video_path(video))
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        video: video
      }
      self.render_file src_path, build_path, data
    end
  end


  def self.video_path video
    [
      'textbooks',
      video['textbook_permalink'],
      'chapters',
      video['chapter_permalink'],
      'materials',
      video['material_permalink'],
      'videos',
      video['permalink'],
    ].join('/')
  end
end

Kane Ong • May 3 '20 • Edited

I had built one from scratch as well. Mine is mainly used for minimizing page size and optimizing page rendering, hot-reloading is included for development.

var gulp = require('gulp')
, minify = require('gulp-htmlmin')
, inlinesource = require('gulp-inline-source')
, browserSync = require('browser-sync').create()
, reload = browserSync.reload
, exec = require('child_process').exec
, jsonminify = require('gulp-jsonminify')
;

//  default hot-reloading task is `watch`
gulp.task('default', ['watch']);

// hot-reloading config
gulp.task('browserSync', function () {
    browserSync.init({
        server: {
          baseDir: 'public'
        },
    })
})

// here is the `watch` task
gulp.task('watch', ['inlinesource', 'minifies', 'browserSync'], function () {
    gulp.watch(
        ['develop/js/*.js', 'develop/*.html', 'develop/css/*.css'], 
        ['minify-index', reload]
    );
    gulp.watch('develop/html/*.html', ['minify-html', reload]);
});

gulp.task('minify-index', ['inlinesource'], function() {
    return gulp.src('public/*.html')
        .pipe(minify({
            collapseWhitespace: true,
            removeComments: true,
            removeAttributeQuotes: true,
            removeStyleLinkTypeAttributes: true
        }))
        .pipe(gulp.dest('public/'))
});

gulp.task('minify-html', function() {
    return gulp.src('develop/html/*.html')
        .pipe(minify({
            collapseWhitespace: true,
            removeComments: true,
            removeAttributeQuotes: true,
            removeStyleLinkTypeAttributes: true
        }))
        .pipe(gulp.dest('public/html/'))
});

gulp.task('inlinesource', function () {
    return gulp.src('develop/*.html')
        .pipe(inlinesource())
        .pipe(gulp.dest('public/'))
});

gulp.task('minify-json', function () {
    return gulp.src(['develop/*.json'])
        .pipe(jsonminify())
        .pipe(gulp.dest('public/'));
});

gulp.task('minifies', ['minify-html', 'minify-index', 'minify-json']);

gulp.task('deploy', ['minifies'], function (cb) {
    return exec('npm run deploy', function (err, stdout, stderr) {
        console.log(stdout);
        console.error(stderr);
        cb(err);
    });
});

The code above is about 3 years old, feel free to run it at your own risk.

Elliot Derhay • May 3 '20 • Edited

Lol I love that we're greeted with a :shrug: banner as we open the article.

Also, yeah, if you have time to build an interesting side project while building a side project, go for it. If I'm reaching for a static site generator, I like to use Nuxt at the moment—although I'm already partial to Vue and I haven't done anything crazy with it yet.

Jayme Edwards 🍃💻 • May 3 '20

Cool learning exercise. Have you checked out Svelte? It’s got a similar tooling stack to react and angular but imho much lighter weight and less quirky you can definitely build static sites with it.

Just throwing it out there. Hope you’re doing well Ben.

svelte.dev/

Matthew Daly • May 3 '20

That's how I wound up with my current site. In late 2014 I was using Octopress, which was fine, but I don't really use Ruby professionally and so I thought ideally I'd be using a Node.js solution. On a whim I rolled a very simple proof of concept for a Grunt plugin to convert Markdown and Handlebars templates into HTML, and put together a Yeoman generator to set it up with some other plugins. It went so well that in early 2015 I switched over to it and have been using it since, though lately I have been considering switching to Gatsby.

Sharad Raj (He/Him) • May 4 '20 • Edited

Same here I also made two of the static site generators both are featured on staticgen.

Python based : my_py_site

github.com/sharadcodes/my_py_site

C++ based: sudo_site

github.com/sharadcodes/sudo_site

my_py_site is capable of doing more then rendering templates, it is able to generate multiple blogs without configuration and it also parses and separates the YML front matter and the Markdown content converted to HTML. You can access that YML front matter anywhere in the templates and moreover the Meta data of pages and posts is also accessible in the templates.

You can use different layouts for any post or page as well. Just specify layout in YML front matter.

All is done under 100 lines of code

View full discussion (28 comments)

DEV Community

One thing led to another and I built my own static site generator today

What is the simplest static site framework?

Ben Halpern ・ Feb 7 '20

Top comments (28)

😊

this-fifo / use-marked-hook

A react hook for parsing markdown with marked and sanitize-html

useMarked() hook

Live Demo

Install

Usage

License

Read next

Build Live Code Editor Clone with HTML, CSS and JavaScript

Types: char and boolean

Introduction to Git: A Powerful Version Control System

Static Keyword in C# Explained in 1 Minute