TypeScript Library Builds with GitHub Actions

Defining goals

At Smartly.io, we already have a working development process, so we’ll formalise and centralise it with github actions. To that end, let’s set out some generic goals for the resultant action:

It should prevent the merging of broken builds (where we can detect them)
It should enforce and simplify versioning
It should automate releases

When I started documenting our approach to GitHub Actions for library builds, I struggled to decide between two quotations to describe the approach:

“Simplify, then add lightness” - Colin Chapman

And:

“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” - Antoine de Saint-Exupéry

The reason for this difficulty is likely that neither actually encapsulates the approach. Instead, an amalgamation of the two is likely more appropriate:

“Remove that which isn’t required for simplicity.”

This might be an odd place to start a blog post about applying GitHub Actions to some of our CI pipelines, but bear with me, this does get a little more concrete in a bit.

Why libraries?

If we want to build the simplest useful action to use as an experiment, then a simple project is a good place to start. We chose to try out GitHub Actions on some of our TypeScript libraries. This would allow us to build a single action for multiple repos and let us gain a little experience managing the same action in multiple repositories.

The simplest case

The most basic pattern we could use looks a little like this:

While this is quite close to typical examples one might find on the web, there is one specific difference worth pointing out. The versions for both the OS we’d be running this on and the version of node we’re using are quite well fixed. We want an environment that is both close to what we run in production and will remain constant until we explicitly tell it to use a different version. Where possible, I recommend not using ‘latest’.

In practice, there may be multiple versions of both OS and node, resulting in a build matrix rather than a single build. That should be tailored to the individual application.

Defining a versioning strategy

Since we’re talking about libraries, we obviously want to use them in other projects. We also want to have a sane versioning strategy so that developers can, at a glance, know when it’s safe to bump versions. Semantic versioning serves our purposes here just fine.

Best practices in semantic versioning could very well be the subject of their own blog post, so we won’t cover that here. Instead, we’ll cover what we decided would work well for our application.

Since we store the code and run the actual release process in GitHub, it seemed logical to control the release from GitHub too. We chose to do this with labels on the PR. We will allow four labels, exactly one of which must be present for the PR to pass tests.

These labels are: “major”, “minor”, “patch”, and “no release”.

Each of the version type labels will do a release of that type. The “no release” label will allow us to explicitly specify that we don’t want a release. This prevents the case of a deployment where a developer forgets to label a PR with a release type.

Hiding complexity in sub-actions

The versioning strategy we described above is a little complicated. We could define it inside the CI action, but that’ll make the action less simple. We want the functionality but not the complexity. GitHub Actions provides a tool to achieve this, in fact we’ve already used it.

In the simplest example case, we used: actions/checkout@v2 which calls an action called ‘checkout’ in an organization called “actions”. We can write our own actions to implement the functionality we want. Fortunately, in the tradition of cooking shows, I have prepared a few useful actions already. We’ll use these in the following examples.

Version Checker

Credential Setup for Publish

Protecting master and versioning commits

The astute reader may have noticed that we have somewhat conflicting requirements above. We want to prevent merges to master (broken and unversioned PRs), which would imply a protected master branch. Still, we also want to use semantic versioning (requiring a push back into master).

Fortunately, github allows us to have branch protection but exclude some users from those branch protection rules. We’ve created a user specifically for this purpose and, for security reasons, it’s a good plan to create a separate user for each repository.

Security

Although we haven’t put this section right at the top of the document, security cannot be an afterthought. This being a relatively new environment, this means that there isn’t a great deal available in terms of security precautions. I’ll cover some of our findings and thinking in terms of remaining secure with GitHub Actions.

Secrets, trust and common sense

When using github actions to build code in a non-public repo, we’ll consider the code itself to be a ‘secret’, since there’s a reason it’s not public. Hopefully there are no actual secrets in your repo, but there may well be GitHub secrets that have been added to use during the build.

The rules here are similar to any other third party software that is making use of your secrets. Apply common sense!

Don’t use any secrets you don’t absolutely have to, use credentials limited to only the required permissions, don’t dump the contents of secrets to logs, etc.

Calling actions

After discussing the advantages of importing actions into your workflow, it’s time to address some of the security implications involved in doing this. When calling an action with the format: “orgname/reponame@version”, we’re implicitly trusting “orgname”.

Why is that? There’s an interesting analysis of the issues here. Essentially, if the “version” refers to a branch or tag, the code you’re actually executing can change without warning. Sure, you did a security audit a week ago, but are you still feeding your secrets to the same code?

There are two approaches to solving this: Using a fixed commit SHA or forking the repo. If you’re using a fixed commit hash, make sure to use the full commit hash, even where a shortened hash would work as the shortened hash could be used as a branch/tag name after deleting the original commit. Forking the repo is safe as you then control the code that you’re pulling into your workflow.

Selecting build jobs and logic

When we know what we want this action to achieve, we’ll split that into jobs. Jobs are individually selectable as branch protection merge requirements and show up as individual items in the Merge Checks list.

The simplest approach is just to split the jobs into the goals we want to achieve:

Version/label Check
Build and test
Release

The next thing to consider is if we should run this in parallel or if we want a chain. The idea of chaining the builds is tempting. First we check the versioning, then we build and test and then we release. There are, however, several problems with this approach. The most obvious one: the environment we build in is not available across jobs, which means that if we want to have the publishing in a separate job, we’ll have to build anyway.

There’s also a more subtle issue. If we chain the jobs, then the first failing job will end the build, we’ll make it more difficult to understand the state of the build. For example, if we put the version label checking step first, then a failure here with chained jobs will not tell us anything about the quality of the code, test status, build status, and so on. We’d need to correct the version labels and re-run the build.

Having the builds in parallel avoids these problems. The builds start simultaneously, the failure of one job doesn’t stop the others from running and the status of individual jobs will be visible in the PR. We’ll run the build and test job when the PR status is not closed and we’ll run the release job when the PR is both closed and merged (more discussion about the triggering in a little bit.

What about splitting it into multiple workflows?

We tried that. It didn’t “feel” right. We gained no extra visibility (separate jobs are already individually viewable in both branch protection checks and PR checks) but split the process into multiple files. In practice, this meant that when we wanted to change anything, we likely ended up needing to remember to make the same changes in various files.

We ended up just moving back to a single CI workflow.

Triggering the workflow

Our process revolves around PRs, so it’s only natural that we’ll trigger our action on PR events. For a start, we’re interested in 3 events related to PRs, opening a PR, pushing to the PR branch and merging the PR. These events are “opened”, “synchronize”, “closed” respectively. For the closed event, we’ll have to make sure that when the PR is closed, it’s also merged. This information is available in the GitHub event payload.

One might ask, since we depend heavily on labels for versioning, why aren’t we also triggering on label add and remove events? We’re not doing this because multiple simultaneous events trigger multiple builds. This means that if we create a PR and label it at the same time, we’ll run two concurrent workflows that do the same thing.

There’s always the possibility of a developer forgetting to add a label to a PR and then having to restart the build later manually, that’s ok. We went with a very low-tech mitigation for this issue: We added a PR template that reminds developers to add a versioning label. We haven’t yet had any complaints about this.

The final trigger looks like this:

What does the final action look like?

So, we’ve talked about what goes into it, here’s the final action:

Adding the workflow to a repo

Currently, we add the workflow to a repo using these steps:

Talk to the respective development team. Aside from it just being plain nice to let people know you’ll be messing with their repo, the release process has changed and they must be aware of this, attempting to release any other way could result in problems.
Add the required secrets to the repo
Make a PR: 1) Add the workflow file. 2) Make changes to what yarn does to make sure that the respective yarn commands work as intended. 3) Add the semantic versioning label to the PR
Add the branch protection to master, making sure to require the build and check_labels steps. Exclude the user for the SSH key you’ve added from the branch protection rules.
Wait for the PR build to succeed
Merge

The problem with success

So, we’ve got many of our dev teams using this and we’ve noticed some issues brought about by success.

Copy-pasting sucks

Sure, we can copy the workflow to every repo, but what happens when we want to change something? For the moment, it’s not a big issue, there are only a few repos involved and there aren’t many changes in the pipeline. This might change in the future. Some options for solving this issue are discussed in the ‘future work’ section.

Stability gets important

The early stage of developing this workflow was great; do whatever you want and see what works. That stops being an option when teams start depending on said workflow.

To deal with this, we have a spearhead project. Changes are deployed there. This project has one of the most sophisticated structures and is quite complex to build. This ensures that if it works there, it’ll likely be fine everywhere else. Also, while this repo is not unimportant, a failed build or two won’t cause significant problems.

Money, money, money

For the moment, we’re not breaking the bank with GitHub Actions, but it’s important to realise that it isn’t free. We build on Linux, so that helps keep costs down (as well as meeting our needs).

At Smartly.io, we have over 300 repos, and if a significant portion of these starts using GitHub Actions heavily, the cost may become an issue. We’re not concerned at this stage, but we’re keeping an eye on it.

Limitations and Future Work

Having used this for a while, some limitations are quite obvious.

We’re planning on dealing with the copy-paste situation mentioned above. We have a few ideas that we’re likely to pursue in the future, the first of which is likely to be git submodules. We’ve also discussed having the action auto-pull itself on every run. The SSH key is available already, so it might be worth a try too.

At the time of writing this, we were unable to find a reasonable metrics solution to keep track of our workflow runs. We’d like to know how many times actions are called, how long runs take, how long preparation takes, which parts take the most time etc. The GitHub Actions API exposes some of these metrics, but we don’t yet have a way to take advantage of this.

Notifications are another issue. With Jenkins, there’s the option of ‘post’ instructions that run at the end of a build and can send notifications about the status of a build. The ‘post’ structure can be emulated in GitHub Actions with the ‘if: always()’ construction, but it’s not possible to get status from previously run jobs, meaning that the notifications would need to be sent at the end of every job. This is not especially convenient, and we’re looking at alternative solutions, possibly using a side-channel to store the status of the previous jobs.

Another worthwhile limitation to mention is the GitHub hosted runners themselves. At the moment, these have 2 CPUs and 7 GB of RAM. That’s simply insufficient for some builds and will prevent us from moving certain larger projects over. To some extent, self-hosted runners can be used to mitigate this, but we would lose the advantage of not needing to manage our own hardware.

Conclusion

We’ve found quite a bit of success in automating library builds in this way. We’re continuing to improve the automation at Smartly.io and the structure of the actions presented may well change, but the principles will likely stay the same. Hopefully this post was helpful in your journey towards using GitHub Actions.