Lady’s Computer: MY computer over YOUR internets

Lady’s Weblog

C·I pipelines have you down? Why not ‹ touch .grass ›?!

Lady

Published: .

This is a blogpost targeted at those who are, or are looking to become :⁠—

  • Developers of independent single‐administrator static websites,

  • Who have access to a file server that can serve their website on the internet (Neocities, a personal server, a tilde club, ⁊·c),

  • And are familiar with Git and plan on using Git to track changes to the sites they create.

If this is you, and you have an account on GitLab or GitHub, you may be considering using the pipeline features of GitLab C·I or GitHub Actions to build and deploy your site every time you push new changes to the repository. (Or, if you are very adventurous, you may be considering writing a post-receive Git hook to do this yourself.)

C·I pipelines are very trendy right now because GitLab and GitHub can make money by selling you the minutes needed to run them. And to be clear: When you are working on a team, these features are extremely useful and convenient! Also, if you are developing a library which other people will use, green pipelines can help assure library users that the code they are pulling passes its own test suite (for whatever that knowledge is worth). But if you are just building a personal website, for yourself, on your own computer?

This blogpost will walk you through an easier way. It doesn’t require any additional accounts or services, it runs entirely on your own computer, and it is 100% free and uses only programs you likely already have installed.

Prerequisites

In order to put this method into practice, you will need the following things installed on your computer :⁠—

  • GNU Make (check with make --version).

  • A program for syncing your website with a remote server. This blogpost will use Rsync, but the Neocities C·L·I push tool can be used if you are pushing to a Neocities site.

In addition, you will need basic familiarity with the command line.

A Simple Build Script

For demonstration purposes, let’s get ourselves a build script for generating a website. Deno provides a fast and convenient way to write useful scripts, so I’ll use that. The following is a simple Deno script for converting Markdown files into their H·T·M·L equivalents.

#!/usr/bin/env -S deno run --allow-read --allow-write
// build.js
// ====================================================================
//
// Copyright © 2023 Lady [@ Lady’s Computer].
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at <https://mozilla.org/MPL/2.0/>.

// We’ll use the `rusty_markdown` package to parse Markdown files.
import {
  html as markdownTokensToHTML,
  tokens as markdownTokens,
} from "https://deno.land/x/rusty_markdown@v0.4.1/mod.ts";

/**
 * Processes a directory by converting all the Markdown files in it
 * (recursively) to HTML.
 */
const processDirectory = async (path) => {
  for await (const entry of Deno.readDir(path)) {
    // Iterate over each entry in this directory and handle it
    // accordingly.
    const { isDirectory, isFile, name } = entry;
    if (isFile && /\.(?:md|markdown)$/.test(name)) {
      // This entry is a file ending in `.md` or `.markdown`. Parse its
      // text contents into H·T·M·L and write the corresponding file.
      const markdown = await Deno.readTextFile(`${path}/${name}`);
      const html = markdownTokensToHTML(markdownTokens(markdown));
      await Deno.writeTextFile(
        `${path}/${name.replace(/\.(?:md|markdown)$/, ".html")}`,
        `<!DOCTYPE html>${ html }`,
      );
    } else if (isDirectory) {
      // This entry is a directory. Process it.
      await processDirectory(`${path}/${name}`);
    }
  }
};

// Process the current directory (or the one supplied to the script).
await processDirectory(Deno.args[0] || ".");

Obviously, this script leaves a lot to be desired: It doesn’t do any templating, styling, or other processing of the input files, and so the resulting H·T·M·L pages will be very boring. But it is good enough for a demo. Feel free to substitute for this script whatever build system you like.

A Simple Makefile

Building our website is already pretty easy, but there’s nothing easier than just typing make, so next let’s create a Makefile for it. If you’ve never encountered a Makefile before, they simply (or not‐so‐simply) group targets (followed by a colon and their prerequisites) with the rules (preceded by a tab) needed to make them. There is a whole lot more information in the GNU Make manual, but for this post that’s the gist. A simple one might look like this :⁠—

build:
	deno run --allow-read --allow-write ./build.js

Make automatically builds the first target if you don’t specify one on the command line, so now when you type make it should build your site. Hooray!

While we’re at it, let’s add some rules for syncing, too.

# …continued from above

# This code assumes you have defined an Rsync filter at `.rsync-filter`
# which excludes all the files you don’t want to sync. If you’re using
# the Neocities C·L·I, it should automatically ignore anything in your
# `.gitignore`.
#
# I’m not going to go into all of the details on how Rsync works right
# now, but I encourage you to read up on it if you ever need to sync a
# local and remote copy of your website.

dry-sync:
	rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync:
	rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public

Now make dry-sync will show us what will happen if we try to sync our local repository with our webserver, and make sync will actually do it. make && make sync is an all‐in‐one way of building and syncing our website, which is pretty convenient.

The above Makefile is literally all you need to build and deploy your site, but it comes with some drawbacks. If you forget to build before you sync, you might accidentally push an older version of your site than you intended. And what about version control? It would be nice if we ensure that is up‐to‐date at the same time.

Adding Version Control Support

In fact you can add all of that functionality to the same Makefile without compromising the simplicity of running make and make sync. Let’s start with version control: git status --porcelain will give us the current status of the working tree, and be empty if all of our changes are committed. [ ! -z $(git status --porcelain) ] is how you say “git status --porcelain is not empty” in command line, so we can use that to create the following check :⁠—

if [ ! -z "$(git status --porcelain)" ]
then
  echo 'Error: There are uncommitted changes!'
  echo 'Commit changes and run `make` before syncing.'
  exit 1
fi

Let’s also test that the current branch is up‐to‐date. In Git, the current branch is signified with HEAD and its upstream can be signified with @{u}. The command git merge-base --is-ancestor will error if its first argument isn’t an ancestor of its second, so we can use it in an if statement directly.

if ! git merge-base --is-ancestor @{u} HEAD
then
  echo 'Error: This branch is currently out‐of‐date!'
  echo 'Pull in changes with `git pull` before syncing.'
  exit 1
fi

When we convert these checks into Makefile, we need to collapse them into single lines and escape any dollar signs (with additional dollar signs). So the Makefile rules for ensuring our files are committed and our branch is up‐to‐date are :⁠—

# The `@` at the beginning of the if statements tells Make not to
# bother printing anything for those lines. That’s okay, since the
# lines will echo out their own content if there is an error.

ensure-branch-up-to-date:
	git fetch
	@if ! git merge-base --is-ancestor @{u} HEAD; then echo 'Error: This branch is currently out‐of‐date!'; echo 'Pull in changes with `git pull` before syncing.'; exit 1; fi

ensure-clean:
	@if [ ! -z "$$(git status --porcelain)" ]; then echo 'Error: There are uncommitted changes!'; echo 'Commit changes and run `make` before syncing.'; exit 1; fi

We can ensure these checks run by adding them as prerequisites to our syncing targets. Let’s add a git push to our sync target as well, since we now know our branch is up‐to‐date.

dry-sync: ensure-clean ensure-branch-up-to-date
	rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync: ensure-clean ensure-branch-up-to-date
	rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public
	git push

Checking for Builds by Touching Grass

To make sure that our latest build is current, we need to add an additional rule to our build target :⁠—

build:
	deno run --allow-read --allow-write ./build.js
	touch .grass

touch is a simple command which updates the modification time on a file, creating it if it doesn’t already exist. By touching .grass (an otherwise meaningless file which we wouldn’t ever modify manually), we can easily keep track of when our site was last built. Add this file to your .gitignore file so that Git doesn’t ever touch it for you.

Because we already have checks in place to ensure that all of our changes have been committed, ensuring our build is current is just a matter of comparing times with the commit date on the latest commit. You can get the modification date of .grass with stat -f '%m' .grass (%m for “modified”), and you can get the last commit time with git log -1 --format='%ct' (%ct for “commit time”). So some suitable checks might be :⁠—

# This check ensures a file named `.grass` exists.
if [ ! -f .grass ]
then
  echo 'Error: The website has not been built yet!'
  echo 'Run `make` before syncing.'
  exit 1
fi

# This check ensures the commit time isn’t greater than the modified
# time.
if [ "$(git log -1 --format='%ct')" -gt "$(stat -f '%m' .grass)" ]
then
  echo 'Error: A commit was made after the last build!'
  echo 'Run `make` before syncing.'
  exit 1
fi

Once again, when we move these checks into our Makefile, we need to collapse them into single lines and escape their dollar signs :⁠—

ensure-build:
	@if [ ! -f .grass ]; then echo 'Error: The website has not been built yet!'; echo 'Run `make` before syncing.'; exit 1; fi
	@if [ "$$(git log -1 --format='%ct')" -gt "$$(stat -f '%m' .grass)" ]; then echo 'Error: A commit was made after the last build!'; echo 'Run `make` before syncing.'; exit 1; fi

Putting it all together

Our final Makefile might look something like this :⁠—

# Makefile
# =====================================================================
#
# © 2023 Lady [@ Lady’s Computer].
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at https://mozilla.org/MPL/2.0/.

build:
	deno run --allow-read --allow-write ./build.js
	touch .grass

ensure-branch-up-to-date:
	git fetch
	@if ! git merge-base --is-ancestor @{u} HEAD; then echo 'Error: This branch is currently out‐of‐date!'; echo 'Pull in changes with `git pull` before syncing.'; exit 1; fi

ensure-clean:
	@if [ ! -z "$$(git status --porcelain)" ]; then echo 'Error: There are uncommitted changes!'; echo 'Commit changes and run `make` before syncing.'; exit 1; fi

ensure-build:
	@if [ ! -f .grass ]; then echo 'Error: The website has not been built yet!'; echo 'Run `make` before syncing.'; exit 1; fi
	@if [ "$$(git log -1 --format='%ct')" -gt "$$(stat -f '%m' .grass)" ]; then echo 'Error: A commit was made after the last build!'; echo 'Run `make` before syncing.'; exit 1; fi

dry-sync: ensure-clean ensure-branch-up-to-date ensure-build
	rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync: ensure-clean ensure-branch-up-to-date ensure-build
	rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public
	git push

To recap, this Makefile :⁠—

  • Builds the site with make (or make build) and pushes it to your webserver with make sync.

  • Refuses to sync if your current branch isn’t up‐to‐date.

  • Refuses to sync if you have uncommitted changes in your working directory.

  • Refuses to sync if you haven’t built your site since your latest commit.

It still isn’t perfect: It can’t protect you against incorrect builds which you make after your latest commit, for example builds based on files you have since deleted or stashed away. But it should catch most common mistakes. For a fuller example, see the Makefile for this blog, which is a little more verbose and includes some (small) additional functionality.

It’s rare that I see people working in website technologies (Javascript; static site generators) talking about Makefiles, which makes sense because they kind of have an association with old compiled languages and application programming. But they’re dead‐simple to write, and can really simplify automation of a lot of tasks! Mostly I worry that people write themselves into either overly‐complicated technical solutions or a lot of error‐prone manual work for something which could be as easy as typing a few words on a command line.

Don’t let this be you! Do the easy thing! make sync!