How to Approach Security with Node.js

We have added a new category to the Node.js Foundation Enterprise Conversation series called Node.js Foundation Enterprise Tech Conversations. This new branch features conversations with experts working on Node.js features that matter to enterprise users, such as security.

For our first tech conversation, we sat down with Mike Samuel who works on Google’s IFC hardening team, which is a part of Google’s security engineering team. This team focuses on frameworks, libraries and programming languages to try to improve them and make it easier to produce secure and robust software.

The conversation covered:

• What kinds of questions should project managers be asking their architects and developers about security? (1:53)
• How to best manage dependencies from a security lens with Node.js? (3:12)
• What kind of security problems should developers expect frameworks to solve? (12:06)
• What emerging threats might development teams need to start planning for? (29:01)
• What new features might help development teams manage security? (33:20)

A few key quick takeaways:

- Your team should update their dependencies when security problems are found upstream.

- Always use second factor authentication with npm.

Security is always a timely topic with any technology as it is ever evolving. If you want to go deeper on the topic of Node.js ecosystem security, check out this introduction to the Node.js Security Working Group and the Node.js Security Roadmap.

The full transcript of the conversation is below and you can also listen to it on YouTube here.

Additional references Mike mentioned during this conversation, include:

• His talk about Node.js Security at JSConf EU 2018
• Script Gadgets
• Follow Mike on Medium here.

Would you like to propose a topic for our enterprise tech conversations? Please send an email to pr@nodejs.org.

Transcript

Tierney Cyren: 00:00 Hey everyone and thank you for coming to another Node.js Enterprise Tech Conversation with Mike Samuel at Google. Mike, do you want to introduce yourself?

Mike Samuel: 00:15 Yeah. I work on Google's IFC hardening team. Hardening is part of security engineering. We take frameworks, libraries, and programing languages and try to improve them to make it easier to produce secure and robust software. It's a fairly small team of security specialists, all of us software engineers. We support a much larger group of application developers.

Tierney Cyren: 00:44 Awesome. I'm Tierney Cyren. I'm a developer at NodeSource and chair of the Node.js community committee. Let's get in to your team and what that team does a bit more. What is the scope of security and how does that kind of play into what you do day-to-day?

Mike Samuel: 01:08 We're responsible for addressing kind of at a framework level common classes of vulnerabilities. We want to take the burden off application developments. For example, if every application developer has to scrutinize all their code to make sure they don't have an XSS, that's not an efficient use of their time. Whereas if we can improve a framework so that they would have to make unusual kind of errors to be vulnerable to XSS, then we’ve improved developer productivity a lot, and that improves security for the end users

Tierney Cyren: 01:44 Awesome. I think that, kind of, plays a bit nicely into the next topic, which as a developer who has to got to do that stuff, has to go and do that work, what are the kinds of questions that someone who isn't a developer like a project manager should be asking their architects and devs about security?

Mike Samuel: 02:12 A team that has a story for how to address each kind of common security vulnerability that's going to be in a lot better place than a team that doesn't. For example, a team that can say, you know, "We are safe from XSS because we use contextually auto escaping template languages. What that means is that when we produce html, our template language understands html and makes sure things are escaped. We use those tools consistently so we're going to be safe against this XSS," and maybe we have another story for why we're safe against SQL injection and other, yeah. Having those stories, kind of, thinking through how do we avoid each of these problems in a systematic and consisted way, puts you in a lot better place.

Tierney Cyren: 03:11 Awesome.

Mike Samuel: 03:12 Other kinds of problems, a team that pays attention to their dependencies and updates their dependencies when security problems are found upstream is going to be in a lot better place.

Tierney Cyren: 03:26 Yeah.

Mike Samuel: 03:27 A lot has been written about the large number of dependencies that many Node project has. That's not a problem in and of itself. It's just that when a bug in one of those becomes known attackers might move quickly to exploit it. Having a story for how you stay up to date is good.

Tierney Cyren: 03:52 You mentioned having a path or an answer for the common security vulnerabilities, or like the common routes of attack. What about uncommon ones? I'm curious if that's, kind of, something that PM should be asked, if there's a question PM's should be asking their team members about that?

Mike Samuel: 04:12 Well, so yeah. Of course. There can be, depending on the nature of the service you're providing, there might be security vulnerabilities that are specific to that. Starting security reviews often early and consulting with security specialists to understand can relate your projects specifics to attacks that have happened in the past is definitely useful.

Tierney Cyren: 04:47 Awesome.

Mike Samuel: 04:50 Then, engaging with security specialists who keep an eye on emerging threats is important too.

Tierney Cyren: 04:59 Mm-hmm (affirmative). You also mentioned managing dependencies. This is something I've learned a lot about recently in terms of, like... A vulnerability usually isn't just going to be at the top level especially in a dependency tree specific to node. It's going to be three or four modules deep; You know, a dependency of a dependency of a dependency. How do you suggest people manage that or deal with that? You know, what sort of tool link do you suggest, maybe?

Mike Samuel: 05:27 Yeah, this is actually one of the things that I've been working with TC39, the JavaScript committee. I've been putting out some code which - and my goal is to make it possible to give different levels of trust to different modules. For example shell access is, you know, if you can access child process then you can... If an attacker can get a string to child process, then they can do all kinds of bad things.

Tierney Cyren: 06:01 Yeah, yeah.

Mike Samuel: 06:03 Very few of your dependencies actually explicitly want to use child process. Very few of them need to use child process to do their jobs. You can focus your attention on understanding the dependencies that do need it if you can make sure that only a small number of modules can access that at one time. This is an example of, kind of, within Google we use static analysis a lot. Static analysis might let us know that a C++ binary, for example, where it is calling out to, you know, on a shell. Node, because it's JavaScript, static analysis only gets you so far. You need to do dynamic enforcement. I've been doing a lot of work on coming up with mechanisms that let us say, " Hey, I know based on, kind of, running my tests that these modules need to invoke child process. I'm going to focus my attention on making sure that they're doing it well. Then I'm going to make sure that in production, you know, other modules try to invoke child process." That kind of limits the downside.

Tierney Cyren: 07:25 Okay.

Mike Samuel: 07:26 Being able to say there's these different sources of authority that can be abused. Being able to differentiate the many, the few of your many dependencies that need any one of those let's you limit the downside and focus your attention... And view focused reviews.

Tierney Cyren: 07:50 That actually brings up a... It's super interesting for me, or a super interesting thing. One of the attack vectors with a node module is the package.json. I'm curious because that's not really the first party JavaScript. That's more third party. I'm curious how that could or does, or theoretically would, integrate into this kind of story if there is integration at that level at all.

Mike Samuel: 08:21 Yes. I think somebody, I forget the researchers name, showed that it was possible in principle to come up with a worm that ran at post install time and used the fact that someone might be npm logged in to make edits to the package.json and propagate itself. You know, you hear something post install and post install basically invokes dash, that dash could have patched the package.json in your project route and then npm committed. Definitely always use second factor authentication with npm.

Tierney Cyren: 09:08 Yeah.

Mike Samuel: 09:11 Yeah, I talk about that a little bit in the... Oh, I should mention I put out this, I and some colleagues from Google put out this document called "Node.js Security Roadmpa. fyi." A security road map kind of explains where we are security wise and how to get to a better place. This abuse of post install scripts is documented there. Ideally, you wind up in a sandbox. There are npm flags that allow you to skip post install scripts. You want to separate the privilege to modify stuff under node modules, which is kind of the end goal of most post install scripts, from the privilege to change developer code, modify the local get client. Also, separate it from the privilege to commit to the global npm repository.

Mike Samuel: 10:21 I documented it there, one was to do that that involves keeping a local snapshot of npm and off-loading the running of post install scripts. It's not trivially influenceable in a few lines of bash so I think it's something that would be provided as a service and funneling all that into a nice, easily deployed set of tools. You know, the short answer is two-factor authentication in the short term and trying to use those tools when they become available in the long term. One of the nice things about maintaining the local snapshot is it actually allows you to react in a more agile manner to security incidents. If you find that somebody has figured out how to exploit a third party dependency, you might not want to wait for upstream to fix that. You might say, "Hey, some people have come up with a fix. We'll patch our local copy because maybe we only use some of the functionality it provides." We could cut out whatever is exploitable if we're not actually using it. Push to our local repository and not have to wait for upstream to secure our servers againstthe zero day.

Tierney Cyren: 11:50 That's super awesome. That's a very interesting portion. I'm kind of excited to see how you evolve to that space. A lot of the things you said kind of speak to the next topic that we were discussing, or where we're going to discuss, which is there's a bunch of different things that developers and frameworks can do. It's always kind of jumbled, at least to me. I'm curious what are the problems around security that devs expect the frameworks to solve?

Mike Samuel: 12:30 That's an excellent question. The previous thing reminded me of one more thing on, kind of, what your project managers should be asking and that is, for me, a team that has a story for what to do when things go wrong is going to be in a better place.

Tierney Cyren: 12:30 Absolutely.

Mike Samuel: 12:49 A team who, kind of, "Where are our database backups?" You know, "Can whoever is holding the pager, if the pager goes off at midnight, do they know how to restore a new and good version of application from backup? And do they know if it happens to be a relatively new member of the team, do they know how to contact someone who can provide a second pair of eyes."

Tierney Cyren: 13:18 Yeah, totally. Honestly, that actually kind of goes into my thought process around the question of what security problems can devs expect frameworks to solve in that one subset of frameworks is local caching. That is one of the things that I have personally seen not enough individuals and companies doing. That, kind of answers that question of, "Okay, we have a local cache. Here's where the thing I need is." That also kind of goes to this is a frameworks to solve this problem. Although, I don't necessarily know if that's how we're talking about framework in the sense that what security problems should that solve.

Mike Samuel: 14:05 I think, yeah. For example, some application containers provide when you push a new version of your application, they provide caches of the old versions. You can revert back to an old version if you have problems with the new version. Is that, kind of, what we're talking about?

Tierney Cyren: 14:27 Yeah, yeah. I'm thinking of something like either JFrog Artifactory or Verdaccio, which is a node app that is just in local npm cache.

Mike Samuel: 14:40 Yes.

Tierney Cyren: 14:40 Just so you know, JFrog Artifactory is a paid product but it's an enterprise grade cache, basically. That's kind of what I'm thinking about in that term of a framework providing a level of security. It's not really a security feature but it's more of a, "How do we go fix something when there is an issue that we can't solve ourselves." Yeah, it's a weird kind of gray area but I think it goes to solve that point that you were talking about.

Mike Samuel: 15:10 Yeah and I think considering that kind of framework code is fine. I think you're right that there are certain kinds of problems that we can expect to solve in a framework like Express and there are certain problems that might be addressed in something like Artifactory.

Tierney Cyren: 15:31 Yeah and-

Mike Samuel: 15:32 It would definitely be relevant to, kind of, how... Where might be a place to offload running a post install script so that we can separate them from developer privileges.

Tierney Cyren: 15:46 Yep. You mentioned Express, what are the things that Express should be solving? What are the security issues at that level that should be solved?

Mike Samuel: 15:58 I'm horribly ignorant of how Express is used. My understanding of Express is that it is a grab bag of many different features and many different product teams use different subsets of those features. With that said, I think you can address many, you know, frameworks can give you a place to check a security assumption. It can also make it very easy to consistently use good tools. I think Express bundles with a set of biopipe handlers. That's one place that you can check security assumption. They also allow you to bundle 10 foot languages. For example, if the simplest, easiest thing is to use a contextually auto escaping 10 foot language, then you can really get a handle on XSS.

Mike Samuel: 17:06 One other way to address many security problems is to represent the safeness against the safeness of some value that you're going to ship over the wire as having a particular run time type. Frameworks likes Express can provide a place to check that assumption consistently. For example, I may express in the type system a distinction between a plain old string and a string of html that I know is safe to load into our origin. You know, maybe safe html is just the name of a type that wraps a string. We do this throughout half a dozen programming languages and frameworks within Google. I define a type that encapsulates some security property. You know, safe html is safe to load into a DOM in my company's origins. Express can then change the writing so that if you are opting in to this additional security assumption, it will object arbitrary strings that are written to your response body when the content type is html. It will allow and unpack these values. It could integrate template systems as low end producers of safe html values to kind of make this transparent to the user where there... You know, when they use the tools that it provides. They may also bundle an html sanitizer that is another low end, kind of trusted producer of these safe html values.

Mike Samuel: 19:10 If you represent as an object, if you represent your security assumptions in your type system, the framework can check the type, check those assumptions where they need to be checked. The framework, by bundling tools, can make it very easy for developers to produce values to distinguish strings that may come from a chapter from values that have gone through a process that has been known to produce safe outlets. I think I repeated myself and was mumbling a bit there.

Tierney Cyren: 19:44 You're good, you're good. Yeah so a lot of... One of the things I kind of picked up on in that is it seems like have an extensible system is important for this. That is a lot of what Node.js has done. A lot of the node stuff is extensible. You build on other people's work, right? You hook into things. Express very much does that. I'd actually like to call it one specific thing just for people who are listening is Helmet. Helmet is a super good tool for Express, and I think HAPI as well, does a lot of this stuff by default. It kind of goes beyond. Express as a tool should be doing this but then it actually goes into, "Hey, we're doing all these things that Express doesn't do." Outside of Express and that specific ecosystem, I think it's a very valuable thing for people who are interested in security and care about it to actually go look for tools that explicitly go and do these things that a core library or a core framework doesn't do.

Mike Samuel: 20:49 Yeah, and I think some-

Tierney Cyren: 20:51 Do you have any thoughts or feelings on that?

Mike Samuel: 20:52 Well, I think Helmet does a good job, among other things, of security relevant headers. You know, making sure that your html response has the headers that you need. I think since the web started, people have noticed that, "Hey, there's these kind of common problems that we can address but we can't just change the way browsers work and so add a header and allow people to opt into kind of the seen behavior," X content type options is one of those. Helmet allows you to, kind of, makes it easy to make sure you're not missing a lot of those opting in to seen behavior things and generating things like content security policies to lockdown eval and client side JavaScript, and that kind of stuff.

Tierney Cyren: 21:52 Yeah. They do a good job with a lot of the things. They also do a few interesting edge cases. I can't think of them off of the top of my head, I've written a few articles about it. Yeah, I can't think of it off the top of my head. That said, I am curious, is that kind of an abstraction of the responsibility to someone else? That stuff should, theoretically, be included in Express. Is it good that someone took the time to go build that stuff where they recognized the weaknesses, or should that be something that's included in Express? Or, like, whatever end. Express and Helmet are one example here, there's multiple examples of things like this, but that kind of approach of going and fixing things. Curious what your thoughts are on that?

Mike Samuel: 22:43 I haven't talked to the Express maintainers about this but my understanding is that the Express, they have a large deployed base of clients.

Tierney Cyren: 22:52 Yes.

Mike Samuel: 22:55 Kind of changing the bites that are sent over the wires are going to be very hard. I think framework, Express is very much a plugin model. Kind of providing a corset of plugins that do things like what some of what Helmet does with, I don't know the breath of what Helmet does, but some of what Helmet does around things like response headers. Being able to say, "Hey, this is my security posture in a list of well-known plugins." Then making sure that any departures from, you know, like I have these plugins which provide new infant tools, make it easy for my development team. I want my development team to use them consistently and I want to have a small white list of exceptions to those rules. Supporting that model, I think, is something that Express could do.

Tierney Cyren: 24:22 Awesome.

Mike Samuel: 24:23 Again, my horrible ignorance is kind of the breath of what the Express team is doing. They may have already have made great strides in that area, I don't know.

Tierney Cyren: 24:32 Yeah, I'm also not as familiar of theirs as I should be.

Mike Samuel: 24:35 Yeah, I really need to learn about it.

Tierney Cyren: 24:37 It's okay. Yeah, there's definitely a lot happening there constantly. We kind of talked a bit about what problems frameworks should solve. What about problems that are not on the frameworks shoulders? Is that something you have strong feelings about or any thoughts around?

Mike Samuel: 25:01 For Node.js frameworks, almost all of the client side JavaScript is not on their shoulders. I think some frameworks do try to do code motion from a server to the client and so they may stand in a place to also vet and maybe re-write client side code to do dynamic enforcement. Yeah, a lot of the network level attacks are things that are best addressed at the container level. If your Google Cloud or the Amazon equivalent... Yeah, so a lot of network level attacks are best addressed at the container level.

Tierney Cyren: 25:46 When you say-

Mike Samuel: 25:46 A lot of-

Tierney Cyren: 25:48 I was going to say when you say container do you mean like Docker or whatever, kind of container en quotes replacement? Or do you mean even like any kind of server? Like VPS or whatever metal you're deploying to, the distinction there.

Mike Samuel: 26:07 Yeah so there... A system is a group of, kind of, cooperating machines. Containers provide a way to introduce those machines to one another. I may have a number of servers up to a number of database backends and maybe there's some micro services in there, as well. The container is where you might handle all the exchange of secret strings that allow and say, "Hey, you're a server. This is where to go for your database access. Here's where the services are." It provides a lot of that. You can attack a system by trying to get to talk to a different database backend, or trying to intercept that traffic. The container can do a lot to make sure that all those are encrypted. Sometimes you want all your... Ideally, you check, you authorize access at every single level. Your micro surfaces don't just trust a message because it happened to come from your web front end. Your web front end, when it sends a message to a micro surface or to your framework, it carries some credentials.

Mike Samuel: 27:42 Let's say I'm doing this on behalf of a particular user, and each step in the chain reauthorizes. Kind of threading those end user credentials through this distributed system, that can involve both the framework and the container because the container is often the one that kind of generates secrets for each node in that distributed system, or maybe for each process running on a node in that distributed system. This is getting pounded into FDO and that kind of stuff, which that's not my specialty.

Tierney Cyren: 28:30 Yeah, yeah. Same. On that, one of the things that I kind of wonder about pretty often is what are some of the more emerging things that happen in terms of devs and needing to plan for security. Security is kind of an often... It's ever evolving. How do people recognize what's happening now? Then also how do they, you can recognize what's happening now, how do they plan to recognize what's happening in the future as well?

Mike Samuel: 29:11 There are a number of emergent threats right now so these routes or features can provide these sources of threats. For example, there are now custom elements in browsers. There was a paper called the Script Gadgets paper who should how features like html import and some ways that new frameworks like Polymer and React use to register element definitions. I think it doesn't actually register element definitions but I think the html imports, some of the html import stuff made it through React. How these new features can let you work around, bypass content security policies and other mechanisms that you use to control how code loads on a client. There's that. If you happen to be doing blockchains, the new thing, there is an emerging body of security literature on ways not to do blockchain and how common application errors can lead to problems in... So that you're attesting to things that you ought not be attesting to.

Mike Samuel: 30:46 I think another of the big changes in security, it used to be the case that security fuzzers, you could run a security fuzzer and it would point you at some possible leak places in your application codes. Attackers started running security fuzzers so they would automatically generate a bunch of queries to see, "Hey, this site may be using an old version of some content management system which is known vulnerable." Fuzzers are now being built around constraint solving systems. There's been likely to be a bit more of an arms race between the security testers and attackers because constraint solving systems are shareistic, so just because I happen to... My constraint solving system might find nine out of 10 vulnerabilities. If the attackers' constraint solving system finds nine out of 10 vulnerabilities, but they're not the same nine, you've got a problem. I think you also asked about how do you deal with emerging threats.

Tierney Cyren: 32:06 Yep.

Mike Samuel: 32:06 Ideally, you'd have a security specialist who reads through the literature and keeps you apprised, and filters in to see what's relevant and what's not. I think there's some good tooling in this area. Npm Audit, for example, does... You know, it's a great feature that kind of tells you when your dependencies have known problems. Not all security vulnerabilities make it into the source of truth for npm Audit but it's great for cases where those are actually published. Yeah, if you're not a security specialist, you just need to kind of know someone like me who reads the html specification so you don't have to.

Tierney Cyren: 33:14 Awesome. Yeah, I don't have a lot of things to ask about that but I'm going to hold those off for now. Moving on to our last thing, or our last topic, I'd like to ask what new features in JavaScript at a platform level, at a framework level, and application level, could or would help teams actually manage security for dev or production deployments?

Mike Samuel: 33:45 Given that many node applications, you know, the code that I write as an application developer is just the tip of the iceberg. There's this huge amount of third party dependency codes that I depend upon. I think one of the critical things is ways to, as I said before, trust code to do what, you know, trust just the modules that need a particular source of visible authority with access to that. Only the things that really need child process get access to it. Being able to say, "These modules need it and I assume the rest don't," requires us to have some concept of module identity. I've proposed to the TC39, the committee that maintains the language, a particular concept of module identity. I think that in JavaScript, JavaScript has a very dynamic language. Static analyzers like typescripts, type system like ESLint, like Google's Closure Compiler are great but they have to make optimistic assumptions. Representing security assumptions about values in the run time type system I think is important.

Mike Samuel: 35:26 I'm a big fan of contract types and I've published the node sec patterns package to serve as a basis for that. Then, I think, we need a way to lockdown powers that, you know, features that are rarely used and which are known to be problems. Eval is evil, but as long as one of your dependencies really, really needs it, then you have to turn it on for the entire runtime. That is unfortunate. Eval has legitimate uses in, kind of, loading that empty generated code. Again, yeah, differential authority for modules should not just extend to other modules. It should also extend to built-ins like Eval.

Tierney Cyren: 36:35 Awesome. With that, I think we should wrap up but before we do, are there any links you'd like to direct people to? Any places, anything that people should look at? Specifically like if that proposal for the identifiers in JS at TC39, if that's up, that would be awesome. Anything else?

Mike Samuel: 36:57 Yeah, a lot of that I gave a talk at JSCon in Berlin. I think that should be up on YouTube soon. The JS Con, it's an awesome conference and they put all of their talks up. I also have some aspirations of being a YouTube start so I put together a YouTube series on showing a series of web puzzles. If you like security puzzles, you can check it out. If you go to medium.com/@mikesamuel, you should find it. It's just called "Puzzling Towards Security." That's where, I don't publish often, but that's where I publish my thoughts on how to coordinate, make interactions between security teams and application developers more efficient and productive.

Tierney Cyren: 38:10 Awesome. Well, thanks for joining us on the enterprise technology conversations. I really appreciate it.

Mike Samuel: 38:17 Yes, thanks much. It was lovely questions.

Tierney Cyren: 38:17 Awesome, thanks.