Lady’s Computer: MY computer over YOUR internets

Lady’s Weblog

Running a Wiki with a Server and Git

Lady

Published: .

In the past few days I’ve set up a new domain here at Lady’s Computer: wiki.ladys.computer. The goal of this site is to be a public, easily‐editable forum (a wiki) for myself and friends to document our thoughts and feelings. This blogpost exists to formally announce the project, describe its underpinnings, and describe how you can set one up on your own.

What Is a Wiki?

In order for this post to make any sense at all, it’s probably first necessary for me to digress into a bit of wiki philosophizing. Because wiki is not Wikipedia, and a lot of people don’t really understand what a wiki is for.

What I want to emphasize in this post is that a wiki is not a reference material. A wiki is a social space. The etymological root for wiki, wikiwiki, is a Hawaiian word meaning “quick”; the goal of the wiki is to lower the bar for social interaction to be as “quick” as possible. What distinguishes the wiki from other forms of social media is the form this social interaction takes. Where·as on a traditional forum, social behaviours are purely additive, the wiki adds a subtractive component: On a wiki, you engage in the social practice of revision or refinement.

Put simply, you write on a wiki because you want somebody to edit it. Editing means deleting the part that is bad and expanding on the part that is good. Editing can mean taking a page with just a title and filling it with words. Editing can mean adding a dissenting opinion to make a strong claim appear more qualified. Editing can mean adding hyperlinks, both internally (to other wiki pages) and out to the broader web.

In all this, there is nothing preventing a wiki from taking on the shape of more traditional social media, writing in what is called “thread mode”. Discourse on a wiki can and probably should begin, in many cases, as a thread of members posting their thoughts and opinions to each other on a page. It is common to sign statements which have a personal component or which one feels particularly strongly about. But threads also don’t have to remain threads. On a wiki, there is nothing stopping someone from going through and structuring the conversation into a more coherent, and readable, argument at a later date.

All wikis keep a log of edits, so nothing is ever lost.

My hope is that the above points make clear the practical utility of a wiki space for dialectics, for critical conversations and discourse, for evolving development of communal thought, and for a kind of participatory dialogue which it is harder to make manifest on other platforms. This radical potential of the wiki is completely lost in sites like Wikipedia and also in the sorts of reference sites that contemporary big‐name static site generators are typically geared towards. Docusaurus, as one completely arbitrary example, is entirely lacking in social features for editors to communicate with each other, which, after all, is a wiki’s entire point. (Contemporary static site generators also, I think, have failings which limit their utilities as reference texts, but I will save those thoughts for a different blogpost.)

Technological Underpinnings

When I first mentioned to my partner that I was « trying to write a wiki generator », their immediate response was « dont like any of the existing ones? ». I then had to explain the following points :⁠—

  1. The kind of thing I’m trying to do mostly doesn’t exist in any broadly‐acknowledged, existing form, and

  2. If I’m doing this right, the end result should be maybe a thousand total lines of code, so I might as well just write it exactly how I want it anyway.

To be entirely clear about this, I am not the first person to have this idea and I am absolutely certain that, for example, the Emacs folks have been doing this sort of thing with Org Mode for probably decades. But there are a few recent technological developments which I think have made now a very good time for this sort of project, and whose recency also means that, of course, I will have to write the actual code to make use of them.

Git

Well, Git is not new. People have been using Git for a long time, and people have been building wikis on top of Git as well for a long time. John MacFarlane’s Gitit is I think a reasonable example of the sort of thing I am interested in, in terms of its general approach.

What I think is relatively new, or at least growing, is the idea that Git is sufficient, that a commandline git interface is the only editing interface one needs for a website of content, especially one containing zero code. We are in a moment where nontechnical use of Git, Git for writers and not just for programmers, and Git for human text and not just computer code, seems within reach. This drastically lowers the bar for what might be considered a reasonable wiki interface, from a web application to a simple repository which trusted individuals can access over S·S·H.

This is not at all difficult to set up, and can be done without the need for any additional tools on virtually any server that one might control.

Deno

Using a Git repository as a backend means it is desirable to have the wiki generator be something which can be called as a simple shell script in a post‐receive hook. Ideally, this script would be something located and tracked in a different repository, and a single file which can easily be grabbed and run even if that other repository is bare.

There are of course a great many scripting languages out there which one could use for this task, but for most of them a problem immediately emerges in the form of dependency management. Deno is notable, and extremely favourable for this kind of use·case in my opinion, because in Deno the problem of dependency management simply does not exist. Every Deno dependency can be specified as simply a U·R·L, and a single script file with numerous dependencies can be straightforwardly run with no external configuration or prerequisite setup or package installation. And Deno is a fully‐featured Javascript (or, if you prefer, Typescript) runtime, so you still have all the strengths and conveniences of a modern programming language, which you wouldn’t have trying to write this, as some have doubtlessly done, in a Lisp or in Bash.

Djot

Because the goal of a wiki is to be quick, it is a fact generally‐accepted that the markup language it uses must be lightweight. Because we will be taking the approach of static site generation, this markup language needs to be trivially, and quickly, parse·able into an A·S·T which can be easily inspected and transformed by our generation script. The best‐in‐class for lightweight markup transformation today is Pandoc, whose A·S·T I have worked with before and which has a variant of Markdown that is highly feature·ful and welsuited to transformation. Unfortunately, Pandoc is written in Haskell, filters for it are typically written in Python or Lua, and it both more heavyweight and more of a dependency nightmare than would really be ideal for this sort of script.

Fortunately, the creator of Pandoc, the afore·mentioned John MacFarlane, has developed another markup language with all the strengths of Pandoc’s Markdown but none of its weaknesses, and has written a parser implementation for this markup language in Typescript. It’s called Djot. Djot is still extremely beta software, but it meets all of our requirements: It is lightweight, fast to process, and trivially importable in Deno. It has a straightforward A·S·T and was designed with script‐based transformation explicitly in mind.

How to Do It

Having decided on the above, I spent a couple of days and threw together a wiki‐generation script which, while messy and not exactly feature·ful right now, is a usable minimum viable product. I won’t go into the design decisions of that script here, but I do want to enumerate how one might use it to set up their own wiki site. The following explanation assumes you have a box running something relatively unixy and are logged in as a root user.

The first step is to clone the GitWikiWeb repository locally, so that you aren’t waiting for the network every time you want to build your wiki. This also enables you to make modifications to the template and stylesheet, which will almost certainly be necessary. (I recommend maintaining your changes as a set of patches on top of the upstream repository, but do whatever you feel most comfortable with.)

The following clones it into /srv/git/GitWikiWeb :⁠—

git clone --bare https://git.ladys.computer/GitWikiWeb.git /srv/git/GitWikiWeb

Next, you will want to set up a wiki user on your server, which all of the editors to your wiki will have access to. The Pro Git book has a very good explanation of how to create a shared Git user, so I will just summarize it here.

This adds a wiki user and creates its .ssh directory :⁠—

# As the root user…
adduser wiki
su wiki
cd
mkdir .ssh && chmod 700 .ssh
touch .ssh/authorized_keys && chmod 600 .ssh/authorized_keys

This sets up the Wiki repository which will store your wiki content :⁠—

# As the root user…
cd /srv/git
mkdir Wiki
cd Wiki
git init --bare
chown -R wiki:wiki .

To prevent wiki users from changing other aspects of the server, set the login shell for wiki to be git-shell :⁠—

# As the root user…
chsh wiki -s $(which git-shell)

Now you can add the S·S·H public keys to /home/wiki/.ssh/authorized_keys for everybody you want to be able to edit your wiki. It is recommended that you prefix these keys with no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty to further restrict what capabilities wiki users have on your server.

Once you do this, users should be able to clone and contribute to the wiki from wiki@yourcomputer.example:/srv/git/Wiki.

The only thing left to do is to add a post-receive hook to /srv/git/Wiki/hooks which will build the site every time somebody pushes new content. Naturally, this file must be executable. Mine looks like the following :⁠—

#!/usr/bin/env -S sh
export GITWIKIWEB=/srv/git/GitWikiWeb
while read oldrev newrev refname
do
  if [ $(git name-rev --name-only --refs=live $refname) = "live" ]
  then
    git archive --remote=$GITWIKIWEB ladys build.js \
      | tar -xO \
      | deno run -A - ~/public $GITWIKIWEB ladys
  fi
done

To take this line‐by‐line :⁠—

  • #!/usr/bin/env -S sh simply runs the file with sh. I use /usr/bin/env here because sh is in a different place on my local and remote computers.

  • export GITWIKIWEB=/srv/git/GitWikiWeb provides the location to the GitWikiWeb repository that we cloned earlier.

  • while read oldrev newrev refname loops over all the refs which were pushed (there can be multiple) and assigns $refname to the name of the ref.

  • if [ $(git name-rev --name-only --refs=live $refname) = "live" ] checks to see if the pushed ref was equivalent to the live branch. git name-rev tries to provide a name for its argument, and with --refs=live, it will prefer live. Change “live” to whatever the default branch for your wiki is, or edit the file at /srv/git/Wiki/HEAD to be ref: refs/heads/live if you like that as the name for the default branch.

  • git archive --remote=$GITWIKIWEB ladys build.js requests an archive of the file build.js from the ladys branch of the GitWikiWeb repository. You will probably want to specify a different branch name. By default, this is a Tar file.

  • tar -xO untars the Tar to standard output.

  • deno run -A - runs the piped script with full permissions. The GitWikiWeb script takes three arguments: ① the location to which the wiki should be built, ② the location of the GitWikiWeb repository, so that it can request template and configuration files, and ③ the branch in the GitWikiWeb repository which files should be requested from (this should match the branch name you gave to git archive).

…and that’s it! Assuming there is a webserver hosting the content at /home/wiki/public/, the wiki should be live as soon as someone posts content to it.

My hope is that the above steps make this feel reasonable, and bring the idea of small, community‐run wikis into a space where they feel like something one could conceivably do. There has been a lot of noise, with social media these days being what it is, about bringing back forum culture, but forum hosting is often complex and involves a nontrivial amount of sysadminning work. I think bringing back wiki culture in a limited, low‐tech, grassroots sort of way is a viable alternative and can satisfy a lot of the same use·cases, sometimes even better.

It’s only been up for a few days, but so far my wiki at wiki.ladys.computer has proven this to be an idea with potential.