Choosing the Node.js versions for your CI tests (hint: use LTS)

This blogpost was contributed by Dominykas Blyžė, a developer at NearForm and a member of the Package Maintenance Working Group.

tl;dr: we’d like to encourage package authors to make sure their Continuous Integration (CI) setup includes all supported Node.js “long-term support” (LTS) versions, as well as the Current stable release.

At the start of 2019, npm released a list of top 1000 most downloaded packages. We scanned that list to find that a large number of these packages have not been updated in over a year, but even if they have — they still did not include the latest Node.js LTS version (10.x at the time) in their Continuous Integration (CI) setup.¹

It is important to note, that CI providers, including Travis (the de facto standard² in JavaScript package ecosystem) might not be using LTS as the default. There is work in progress to change this, however it is a good time to double check your testing configuration — Node.js 12.x just became the active LTS version and Node.js 13 just got released. At the end of December 2019 Node.js 8.x will reach end-of-life.³

Why

The release schedule of Node.js is set up to have a new release with breaking changes every 6 months. According to semver, each such release increments the major version number. Even-numbered releases (10, 12, etc) receive “long-term support” (30 months) and because of that only LTS versions are recommended to be used in production applications.

While library authors are free to choose any support policy⁴ they prefer, their libraries are likely to be used in one of up to 3 supported LTS versions. We recommend to include all of these versions in the CI test matrix. At the time of writing this would mean Node.js 8, 10 and 12. By January 2020 this will become 10 and 12.

Odd-numbered (11, 13, etc) Node.js releases typically become unsupported after 6 months when a new even-numbered release is made available. While it is generally recommended that production applications do not use non-LTS Node.js versions, for library authors the recommendation is different — we’d like to encourage you to include the Current version (at the time of writing — 13) in the test matrix, as this can serve as an advance warning for both — library authors and Node.js maintainers⁵.

The above recommendation is for the minimal set of versions that should be supported — if it is an option at all, we’d like to encourage everyone to support older versions as well.

Additionally, while it is recommended to use the latest version of a major release line, you may want to support some specific versions, e.g. 8.10.0 is used by AWS Lambda, even though 8.16.1 is available. If that is one of your support targets you will be unable to use some of the newer features in 8.x and you should be checking for that in CI.

How: setting up your .travis.yml

Travis uses nvm to install Node.js, so you can use any version or keyword it supports in your .travis.yml. This means that your setup should look something like this:

language: node_js
node_js:
- node   # Current stable, i.e. 13 in October, 2019
- lts/*  # Most recent LTS version, i.e. 12 in October, 2019
- 10     # Explicitly include an active LTS version
- 8      # Explicitly include the maintenance LTS version

When you use the keywords ( node,lts/* ), you will automatically start testing in new versions as they come out. This does, however, mean that you will also automatically stop testing in versions that may still need to be supported.

Alternatively, you can use Renovate to automatically open pull requests with new versions to your .travis.yml by adding { “travis”: { “enabled”: true } } in your renovate.json.

When you’re testing with a Current (or nightly) Node.js version before it becomes LTS you can also enable allow_failures for it.

How: other CI tools

• Appveyor, at the time of writing, has Node.js 8.x installed by default. They also allow testing in multiple Node.js versions. You can use the Install-Product node ‘‘ command to install the latest (Current) available release. You should verify your configuration to include all the other LTS versions as well.
• Circle CI requires you to explicitly list the docker images you want to use for testing. The images provided by Circle CI on Docker Hub do have aliases such as lts or current, they even provide a ChakraCore based image. You should make sure your configured list is up to date.
• Github Actions includes all LTS versions and the Current version in their Node.js starter workflow, however you will need to maintain that list to keep it up to date. There is an issue discussing the options for matrix aliases with the setup-node action and a PR to introduce keywords for individual versions.

Please leave a comment with other tips!

How: dropping support for older Node.js versions

Maintaining an Open Source Software library can become a burden and it is unfeasible to expect that all versions of Node.js should be supported by library authors indefinitely. Changing the minimum Node.js version requirement is considered a breaking change and according to semver rules requires a major version of your package to be incremented.

Still, this decision should not be taken lightly, as bumping a major version of your package can have a snowball effect on the package dependents and the rest of the ecosystem. This can also result in extra work to backport security patches to older versions, if the package consumers are unable to upgrade fast enough.

You should also keep the engines field in your package.json up to date to indicate supported versions.

Footnotes

1. Out of top 1000 packages using Travis for CI, 315 had neither a variation of 10.x nor lts/*in their setup; ~150 of those had commits since 10.x was first released, of which ~120 had commits since 10.x became active LTS.
2. Out of top 1000 packages, 877 were pointing to GitHub repos with a .travis.yml file at the root in September 2019.
3. Originally, Node.js 8.x was meant to be supported until March 2020, however the schedule had to be moved forward due to the OpenSSL version that it uses getting to end-of-life in December 2019.
4. The Package Maintenance Working Group is working on a way to define your support levels — we welcome discussions and contributions.
5. The Node.js project also runs smoke tests using The Canary in the Goldmine tool.

Thanks to all the reviewers!