This blog was written by Michael Dawson, IBM Community Lead for Node.js and chair of the Node.js Technical Steering Committee (TSC).
Come join the new package maintenance team at https://github.com/nodejs/package-maintenance either to help out or to share your experience. I’ll talk in more detail about the initial work that this team will focus on, but first let’s introduce the challenge.
While Node.js has been growing rapidly, there are aspects of the module ecosystem which act as a source of friction to adoption, particularly in the enterprise. The highest level of comfort is with packages that are stable and in which there is confidence that there will be ongoing maintenance with predictable releases and easy upgrades to new Node.js versions.
Unfortunately, there are a number of packages which are broadly used in the Node.js ecosystem that may no longer get the basic maintenance needed to keep them up-to-date, reliable for the widespread use that they have, and usable with the latest Node.js releases.
There are many different reasons for this, one being that widespread use does not necessarily result in help with the day-to-day work of keeping the module up to date and moving forward. Without this help, the original maintainer may move on or no longer have time and there is nobody else to step in.
These packages are important to the Node.js ecosystem for many reasons. One reason is the philosophy of a “small core” in Node.js itself. Having a small core means that the baseline components for most (if not all) Node.js developers consists of these packages in addition to Node.js itself. These packages are often feature complete, but they need to be updated from time to time to cater with new Node.js releases and new language features. These packages are often downloaded millions of times per month, however they sit in very deep dependency trees and the community expects that they just work: no one is interested in maintaining them. As we accelerate the pace of change with more up to date V8 versions in Node.js, easier vulnerability reporting, and more automated security auditing this just adds to the burden on the package maintainers.
Looking for some poignant examples? Consider readable-stream, the most downloaded package on npm, which is maintained by a number of individuals countable on one hand. MQTT.js is another good example which is used by AWS, Microsoft, and IBM, however the maintainers have trouble in keeping up with the issues (110 and counting) while implementing new features.
Why are companies depending on these key modules not already helping to solve this problem? One key reason is that many companies, who might be willing to contribute time and effort to maintaining packages that are important to them, don’t have a clear and easily understood path to doing so.
I’ve already talked to a number of individuals and companies who are interested getting involved in this kind of effort (Including NearForm, Microsoft, Google, Telus, GoDaddy, IBM, and Paypal) and I’m hoping this blog will spur discussion and interest in even more to want to get involved.
Earlier this year at Node Summit, IBM raised the concept of long term support (LTS) for packages. While there was good support for this concept from those that I talked to, conversations often quickly led to the broader issue of basic package maintenance as mentioned above.
Having talked to a number of people at from companies who are members of the Foundation, as well as companies that depend on key packages, the consensus seems to be that there is a role for the Node.js project to play in helping these key packages get the minimum support/updates required.
Individuals and companies are already doing some of this work in order to help address security, functional, and currency issues. A good example is the work that Chalker is doing to port buffer fixes. Another is NearForm’s sponsorship over the last few years for https://www.npmjs.com/package/joi. Yet another is the IBM i development team’s performance improvements, N-API support and ongoing help in maintaining Node.js ODBC bindings.
However, we believe we can do better by providing a focal point and an easy path for companies to both develop the business case to join in and identify where/how it makes sense to contribute.
As an example, discussions with Ahmad Nassri from Telus revealed that they have tools to help identify which packages an organization depends on. This kind of information can help form the business justification for helping to maintain those packages. Similarly, discussions with Charlie Robbins from GoDaddy and Matt Edelman from Paypal focused on providing approaches and tools to allow organizations to enable their Node.js developers to contribute to this kind of work.
I also talked with Christopher Hiller, one of the maintainers for Mocha, to get a maintainer’s perspective. It seems that there would also be value in working to provide tools and best practices (like a standardized LTS policy) which make it easier for maintainers who are interested in providing long term support if they wish to do so. Examples include figuring out strategies for managing LTS releases and tooling to support implementing those strategies.
An initiative supported by the Node.js project would be a win all around. For package maintainers it could lead to additional processes and tools to simplify ongoing maintenance and potentially additional help as companies better understand their dependencies and the value of helping to support them (of course only if a maintainer wants this). For end users, including enterprises, it could result in packages which are more up to date and safer to use. It would also provide them with a way to mitigate risk in their dependencies by facilitating their contribution to those dependencies. Finally, for the Node.js project, by enabling enterprise adoption it would help to support the phenomenal growth we have seen so far.
Call to Action
The next step is the formation of the package maintenance team within the Node.js organization at https://github.com/nodejs/package-maintenance. My call to action is for you to join the team either to help out or to share your experiences with the problem outlined above.
While it is up to the team to decide on priorities as a group, some of the initial steps that I have in mind include:
- Define and document how to prioritize which packages are key to the Node.js ecosystem and how/what assistance can/should be provided. One of the key aspects of this is understanding what communication channels we need in order to be able to identify when specific issues are slowing migration from one Node.js version to another or causing friction in the ecosystem in some other way.
- Building and documenting guidance, tools and processes that businesses can use to identify packages on which they depend, and then to use this information to be able to build a business case that supports their organization and developers helping to maintain those packages.
- Documenting a backlog and providing resources that help businesses identify how their developers can ramp up and get engaged to help. Test and validate a workflow by helping with some issues that are slowing migration to Node.js 10.x.
- Building, documenting and evangelizing guidance, tools and processes that make it easier for maintainers to manage multiple streams and accept help from those who depend on their module.
We hope you agree this is an important challenge and look forward to working with you as part of this new team.