Doubling Build Speed

Background

As a refresher, this site has two components

• The build phase, which generates the HTML, CSS, and JavaScript, as well as OG images, architecture diagrams, and WASM files
• The runtime phase, which serves the pre-generated files to you over the network

This article is focusing on the build phase, and is the story of how I doubled the speed of my build pipeline.

The Old Pipeline

Let's reconstruct my original build pipeline first to fully understand how and why the changes I made increased the build speed.

The first step: naming your job and the triggers.

1name: Build and Deploy
2
3on:
4  push:
5    branches: [ "main" ]

This tells Github to run this pipeline whenever I push a change to the main branch. The main branch is the central location for all of my production code.

Now let's add the actual pipeline logic.

First, we will add tests to the pipeline. We need our automated tests to pass in order for the deployment to succeed. If they fail, it could mean we broke something with our changes.

 1name: Build and Deploy
 2
 3on:
 4  push:
 5    branches: [ "main" ]
 6
 7jobs:
 8  test:
 9    runs-on: ubuntu-latest
10    steps:
11      - uses: actions/checkout@v4
12      - uses: actions/setup-go@v4
13        with:
14          go-version: '1.24.3'
15      - name: Run Tests
16        run: go test -v -race ./...

In this block, we defined the jobs part of the YML, which defines what will actually happen when I push a change to the main branch.

Now, let's move onto the build process. Note that the following is a sub element of jobs, just like test.

 1build:
 2    runs-on: ubuntu-latest
 3    steps:
 4      - uses: actions/checkout@v4
 5      - uses: actions/setup-go@v4
 6        with:
 7          go-version: '1.24.3'
 8
 9      - uses: acifani/setup-tinygo@v1
10        with:
11          tinygo-version: '0.39.0'
12
13      - name: Install Graphviz
14        run: sudo apt-get update && sudo apt-get install -y graphviz
15
16      - run: go run cmd/builder/main.go
17
18      - name: Verify OG Images
19        run: |
20          if [ -z "$(ls -A public/assets/og/*.png 2>/dev/null)" ]; then
21             echo "Error: No OG images generated!"
22             exit 1
23          fi          
24
25      - run: GOOS=linux GOARCH=amd64 go build -o myblog cmd/server/main.go
26
27      - name: Upload Artifacts
28        uses: actions/upload-artifact@v4
29        with:
30          name: release-files
31          path: |
32            myblog
33            public/
34            assets/            
35          retention-days: 1

This block is longer but it performs all of the necessary steps to build our application binary.

• First, it sets up Go.
• Then, it sets up TinyGo, which is needed for the WASM game of life simulation on the homepage.
• It installs Graphviz, which is needed for the architecture diagrams.
• Then, it runs the builder application logic which generates HTML files, WASM, and architecture diagrams.
• It has a verification step to make sure OG images were actually generated by the builder.
• Compiles the Go project into a single binary that can be run on Linux.
• Finally, it uploads the binary and the HTML/WASM files generated by the builder, which will be used by the deploy step.

Note: There is also a Deploy job, which actually takes the uploaded artifacts and uploads them to the server running my blog, but I'm going to exclude that for brevity as it didn't need to be optimized at all. I wanted to mention it, though, to have a holistic view on what the pipeline does.

Let's discuss some optimizations we can make to increase the speed of this pipeline.

Optimizing Installations

Let's recap what the builder is doing in the first few steps.

• First, it sets up Go.
• Then, it sets up TinyGo, which is needed for the WASM game of life simulation on the homepage.
• It installs Graphviz, which is needed for the architecture diagrams.

This can be optimized. We shouldn't need to do this every time we make a change to the code. What if we built a Docker image that comes with all of this pre-installed? This has a few benefits.

• Our build will run in its own container which will not change unless we change it. The Github Action runners' OS or other internals could change otherwise, potentially breaking our build without this container.
• We can pre-package Go, TinyGo, and Graphviz all in the Docker image so that we don't need to re-download and install them every time.

To do this, we create a Dockerfile.build file. That file looks like this:

 1FROM ubuntu:22.04
 2
 3ENV DEBIAN_FRONTEND=noninteractive
 4
 5RUN apt-get update && apt-get install -y \
 6    curl \
 7    git \
 8    build-essential \
 9    graphviz \
10    && rm -rf /var/lib/apt/lists/*
11
12RUN apt-get update && apt-get install -y \
13    curl \
14    wget \
15    git \
16    build-essential \
17    graphviz \
18    && rm -rf /var/lib/apt/lists/*
19
20RUN wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb \
21    && apt-get update \
22    && apt-get install -y ./google-chrome-stable_current_amd64.deb \
23    && rm google-chrome-stable_current_amd64.deb \
24    && rm -rf /var/lib/apt/lists/*
25
26RUN curl -OL https://go.dev/dl/go1.24.3.linux-amd64.tar.gz \
27    && tar -C /usr/local -xzf go1.24.3.linux-amd64.tar.gz \
28    && rm go1.24.3.linux-amd64.tar.gz
29
30ENV PATH=$PATH:/usr/local/go/bin
31
32RUN curl -L -o tinygo_0.39.0_amd64.deb https://github.com/tinygo-org/tinygo/releases/download/v0.39.0/tinygo_0.39.0_amd64.deb \
33    && dpkg -i tinygo_0.39.0_amd64.deb \
34    && rm tinygo_0.39.0_amd64.deb
35
36WORKDIR /app
37
38COPY go.mod go.sum ./
39RUN go mod download

Note that it also installs Chrome, as that's needed for our OG image generation during the app build step. Now that we have this docker image, all our Github Actions pipeline needs to do is download the image from the Docker registry. It no longer needs to re-install all of the dependencies that our app needs to build the static site.

We also need to tell the build job that it will be using the docker image:

1  build:
2      runs-on: ubuntu-latest
3      container:
4        image: ghcr.io/thornhall/my-blog-builder:latest

Optimizing App Builder

Now we need to optimize this step:

1- run: go run cmd/builder/main.go

To optimize this, we need to understand what it's doing:

• It builds the Go application binary for the builder
• It builds the HTML files
• It builds OG images
• It builds architecture diagrams
• It builds WASM files
• It compresses all of them

One thing to notice is that we don't need to redo all of this work for every change to the repo. Often times, we can re-use the HTML files that were generated from the previous step. For example, the articles I write don't change often, so why would we regenerate them every single time? The same is true by extension for the OG images, which are based off the articles.

To optimize this, we will cache files from previous builds and reuse them when we can. We will also cache the first step, which is building the application binary.

 1- name: Cache Go Modules
 2    uses: actions/cache@v4
 3    with:
 4    # Note: be sure to specify the path for GOCACHE and GOMODCACHE in the env to match these
 5    path: |
 6        /root/go/pkg/mod
 7        /root/.cache/go-build        
 8    key: go-deps-${{ runner.os }}-${{ hashFiles('**/go.sum') }}
 9    restore-keys: |
10        go-deps-${{ runner.os }}-
11                
12- name: Restore Previous Build Assets
13    uses: actions/cache@v4
14    with:
15    path: |
16        public/             
17        .build_cache.json        
18    key: build-assets-${{ runner.os }}-${{ hashFiles('templates/**', 'cmd/builder/**') }}
19    restore-keys: |
20                build-assets-${{ runner.os }}-

To do this, we use the cache v4 action. Note that we cache our Go modules separately from our static assets such as HTML files. This is so we don't need to re-download Go modules when we're making simple changes to our app.

Cache Go Modules tells the Github pipeline to only invalidate the cache if the go.sum file changes, which means we introduced a new dependency to be downloaded.

Restore Previous Build Assets tells the pipeline to invalidate the cache if we change the builder itself or any of the HTML templates.

Note: I also needed to add application logic in builder.go to check for already-existing files before generating them, but I'm not including that here as the main focus of this article is the pipeline itself. You could imagine it does something simple like checking if the file path it is trying to generate is already present in the environment it's running in before generating the file.

Optimizing Artifacts

We can perform one last optimization. With the generated artifacts (the Go binary + HTML files + WASM files + images and etc), we can compress them before uploading. This will decrease the time the runner spends sending data over the network, speeding up the pipeline.

 1- name: Package Release
 2    run: |
 3    GOOS=linux GOARCH=amd64 go build -o myblog cmd/server/main.go
 4    tar -czf release.tar.gz myblog public/ assets/    
 5
 6- name: Upload Artifacts
 7    uses: actions/upload-artifact@v4
 8    with:
 9    # Note: be sure to use the same name and to unzip in your deploy step
10    name: release-bundle
11    path: release.tar.gz
12    retention-days: 1

Summary

Here's a list of the optimizations we performed and why we performed them.

• We built our application with a docker image so that the Github pipeline runner doesn't need to download Go, TinyGo, Graphviz, and Chrome on every run. We also get the benefit of reproducible and isolated builds that don't change when the Github runner changes.
• We optimized our app builder to cache its artifacts. If nothing on the critical path changes, we can reuse what we generated on the previous run. This dramatically improves speed for when we make a simple change such as adding a new article.
• We compressed the build artifacts (Go executable, images, HTML files, etc) so that when they're uploaded by the runner, it goes much faster.

The Final Result

My build pipeline went from taking over 4 minutes to run to now taking around 1 minute and 40 seconds. The feedback loop between making a change and seeing the result is now much smaller which improves iteration speed and attention spans.