Back in 2016, when I started to work as a front-end developer, I remember there was a big battle between NodeJS package managers "npm vs. yarn".
Despite some people's preference, npm was always behind, especially in terms of performance. Using the same project, installing the dependencies with yarn and npm, npm always took longer to install dependencies.
Yarn was also trying to innovate. It brought the workspaces (dealing with multiple packages in the same repository), which lacked features but were a big deal when we combined it with Lerna, for example.
Over the years, npm got better and faster, introduced workspace (very weird approach), and Yarn decided to go in another direction with Yarn 2 by completely switching the dependency installation strategy to something called Plug'n'Play.
Yarn's new way of installing dependencies solves many NodeJS dependency management problems, but this innovation came with a trade-off: lack of compatibility with old projects and tools.
Because of the lack of compatibility, people that got used to Yarn (and like it) still use version 1, which hasn't received any relevant feature since 2020.
Others preferred to switch back to npm since it improved a lot in the latest versions, and it come together with NodeJS when we installed it. It's like a "let's stick with the defaults" movement, and that's valid.
However, there's another solution that has been around since 2015, and people are always reluctant to try it out: pnpm.
I understand the overhead of "new tooling" in the JS world is a big deal, but I believe people are missing a lot in not trying out pnpm.
So, in this post, I aim to explain why I made pnpm my default NodeJS package manager, abandoned Yarn completely, and in some specific cases, I still use npm.
Needless to say, this is an opinionated blog post, and you're not forced to change the tools you're used to using. My intention is to bring you some insights I had a while ago.
Why not Yarn?
As I mentioned, Yarn v2^ went to a completely different path to solve the node_module problems in NodeJS, which isn't widely supported by old and modern applications.
This would force me to stick with Yarn v1, which no longer receives features, only minor and security improvements. It's like using a tool that got frozen in time.
So, if I need to install another package manager, I'd rather use something modern, which is still evolving and trying to solve all the existing problems (we still have many), instead of using a legacy tool.
If one day, Plug'n'Play dependencies become a big deal, pnpm already supports it. Meaning that probably I'd never get back to Yarn again 🥺.
Why not NPM?
Npm has had its problems in the past, but in the recent versions, it's much better but still somehow slow in some cases.
As mentioned, there are some cases I still use npm, and I notice that usually, in the first installation, it takes a lot longer than pnpm, for example, but if we reinstall, it goes crazy faster.
My overall feeling for the npm performance is "inconsistency". Sometimes it is fast, sometimes, it is slow.
One big deal for me nowadays is to use monorepos. Even if I don't publish any package, I love to isolate packages and tools for the same domain and deal with them as third-party tools. And to handle monorepos, we need package managers that support "workspace".
Npm has introduced the concept of "workspace" in v7, but it's very basic and introduces a few problems.
Most applications would expect an adjacent to its packages.json node_modules folder.
root/
└─ apps/
└─ website/
├─ node_modules/
└─ package.jsonBut using npm workspace won't create these adjacent node_modules. Instead, it'll hoist node_modules completely to a root level:
root/
├─ node_modules/ <- all dependencies will be here
└─ app/
└─ website/
└─ package.jsonAnd... this is extremely problematic.
If you have simple and tiny libraries, you won't face this problem. However, if you install NextJS for example, you'll probably have issues running the next * commands.
That is because some CLIs or dependencies expect to find the binary inside the node_modules folder in the root level of a project (in this case, the website).
Maybe we would need to run npx next <command>, instead of trying to call Next's CLI directly, but you already notice some necessary mind-shift. Not everyone can deal with all those changes in peace. People just want to boot a new project and start coding.
I've been using pnpm's workspace for a few years already, and when I tried npm workspace, I felt like: "This feature is not the priority of npm", which honestly, it's ok, but it doesn't fulfill my needs.
Another problem with npm is the disk space.
When we install dependencies with npm, it still copies all dependencies to each project you have when we run the installation command. This means if you have 20 NextJS projects on your machine, you'll have 20 copies of all dependencies.
It seems a minor problem, but trust me, the more you try things out, create, and experiment, the bigger this problem becomes, and the more often you have to do disk space cleanups.
To close up, there are two cases I'd stick with npm: when the platform does not support pnpm at all, and there's no workaround, and when I'm sharing something with a new engineer.
Pnpm has been there for a while, but there are still a bunch of platforms that do not accept it, and new engineers already have a lot of things to learn. I don't want to spend their mental energy explaining a new tool to install dependencies.
Why pnpm, then?
I hope it's clear why I don't use Yarn v1, v2, and npm.
Now, I want to bring more details on why I love using pnpm, along with some caveats and situations where I had problems with it.
Global cache + hard links = less disk space usage
Remember when I mentioned that if you have 20 NextJS problems in your machine and install the dependencies with npm, you would have 20 copies of all dependencies on disk?
Pnpm does something different. It uses a global cache strategy and hard links to reduce disk space usage.
When we install the dependencies, pnpm will first check in its global store if a dependency has already been downloaded in the past. If so, instead of downloading it, "reuse" that instance. Otherwise, it downloads and stores it there for further usage.
This minimizes the overall installation time and avoids adding a duplicated dependency to your machine.
After this step, pnpm will create a hard link between the dependency in the global story and the node_modules in your project.
In other words, instead of having full copies, you'll have files that point to dependencies in a single file in the global store.
Because it's actually files and not only points, there will be still some size, but it's significantly smaller when we compare it with the file copies.
I ran a small test where I installed 3 times a NextJS project with npm and 3 times with pnpm. Using npm, the 3 folders had a total of 685M, while the 3 folders using pnpm had only 408M.
If you scale this to over 60 projects (I easily have more than that), then we start actually to see the difference.
Installation speed
Installing dependencies usually take 3 steps:
- Resolving the dependencies. The package manager identifies the dependencies, the versions, etc.;
- Fetching those dependencies;
- Linking all fetched dependencies.
Yarn and npm do these 3 steps separately:
Pnpm tries to combine those steps in a more performance way:
Creates non-flat node_modules fixing phantom dependencies problem
Another pnpm's selling point is that they address an old problem with the traditional way of installing the modules, which is flat node_modules.
To show the problem with a flat node_modules, let's imagine we start a new project and install a single dependency: express.
Yarn1 and npm will not only create a node_modules folder containing express. Instead, they will copy all dependencies of express inside node_modules, meaning that our modules folder will not only have a single dependency (express) but dozens of dependencies of our dependencies and the dependencies of our dependencies.
You might've been thinking:
"What's the problem with that? Do we need to care about this at all?"
The problem is that from a NodeJS perspective, a flat node_modules allows us to import a dependency we have not listed in our package.json.
For example, express has "parseurl" as a dependency. Because this one is has a folder in our node_modules, the following code will work without any error:
const parseUrl = require("parseurl");
Even if we never installed directly "parseurl".
This problem is called "Phantom Dependency".
To solve this issue, pnpm will create only the express folder inside our node_modules, and the dependencies of express will live inside node_modules/.pnpm.
If we now try to execute the same code as before, NodeJS will throw an error saying it wasn't possible to find "parseurl".
We should never be able to use dependencies not listed in our package.json, but with the flat node_modules, I have seen this happening more than I'd like to, mostly in vanilla JS projects.
Although it is a good fix, it may be a problem for projects that relies on this buggy behavior. Later in the caveats section, I will talk more about that and how to change pnpm behavior to fix that.
Workspaces
That is, for me, the biggest win of pnpm.
To clarify, you do not need to use workspaces (monorepo) to use pnpm. You could have a single repository and use npm to manage its dependencies exactly like npm or yarn.
But if one day you decide you want to turn this repository into a monorepo, even if you're not publishing any package, just for a matter of separation of concerns, all you need would be to group all files in a folder and create a file called pnpm-workspace.yaml.
In this file, you'll need to declare where the packages/apps are located:
packages:
- 'packages/*'
- 'apps/*'After that, you will run pnpm install, and everything will simply work. Pnpm will install all dependencies for the tracked project, create symlinks if needed, create multiple node_modules, allow us to run the same command for multiple packages, etc.
It's incredible because when I worked with Yarn1 workspace, I always had to combine with Lerna. That's because Yarn handled the low-level tasks (node_modules, links, etc), and Lerna handled the high-level tasks, such as allowing to run commands for multiple projects.
Pnpm combined both low and high-level tasks in a single tool and does a really good job on that.
There are some gaps that pnpm cannot handle when the monorepo gets bigger, but this is a discussion for another post.
Update command
One very tiny thing I love about pnpm is the update command.
This command is also present on npm, but pnpm gave a step further and gave us the --interactive flag.
When I'm in a project, and I'd like to know which dependency is available to be updated, I can run pnpm up --interactive --latest, and an interactive prompt will show the current versions I have in my project and which one is available to be updated.
Npm offers the update command but without any interactive option, which means it'll just bump everything to the latest version.
Yarn also offers this through the command yarn upgrade-interactive command --latest, which seems to be the inspiration of pnpm.
Pnpm caveats
As we know, there's no such thing as a perfect solution, and with pnpm, it won't be different.
If you decide to switch, here are some considerations you have to make.
Major versions
In semver, a major version usually means breaking change.
Zoltan (author and maintainer of pnpm) always graps the major version bump opportunity to make major improvements, which is totally fair.
The problem is that if we're in a project that uses pnpm 7, we can't just use pnpm 8 and expect things will work out.
Between this major version, there are also breaking changes regarding the pnpm-lock.yml file, meaning that it'll break once you try to install dependencies with incompatible pnpm versions.
Again, that's ok, but most people (including myself from the past) don't think much about the package manager version, especially while setting up environments.
This becomes a major problem if you use the "latest" tag to install pnpm in your CI.
So, to mitigate this problem, I'd recommend always pin the version to the one your project supports:
-npm i -g pnpm
+npm i -g pnpm@latest-8 # or any other versionNot all platform supports it natively
Even though pnpm was created in 2015, not every platform supports it natively. You'll always find support for npm and yarn, but not always for pnpm.
For example, one company I worked for used Cloudfoundry to provide packed environments (node, java, etc.) through build packs.
Their NodeJS buildpack didn't support pnpm and didn't allow us to specify an installation command, making impossible to use anything but yarn/npm.
If the platform doesn't support pnpm but gives us free access to the install command, we can bypass the limitations of installing pnpm through npm:
npm i -g pnpm@latest-8 && pnpm installAnother example of this lack of support is dependabot (the bot to update dependencies from GitHub) which didn't support repositories using pnpm until June/2022.
Non flat node_modules might be a problem
As mentioned in the flat node_modules section, not all projects will work out of the box with this "non-flat node_modules" strategy.
For example, if you start a React Native project using pnpm, you might have tons of errors, and the project will not work. That's because when they build React Native, they rely on this broken behavior and maybe it's too hard to migrate.
Luckily, pnpm gives us the ability to decide how the node_modules will be generated through the "node-linker" option.
So, if you're starting a React Native project (or any that doesn't support non-flat node_modules), all you need is to create a .npmrc file and specify that the node-linker should be "hoisted":
node-linker=hoistedHow would I know if the project supports or not non-flat node_modules?
Trying it out.
If, for some reason, you start a new project, install dependencies with pnpm, and the project seems to be broken, try the setup the hoisted option.
Tips for migrating
Though the learning curve is low, there are some tips and tricks to keep in mind for switching from npm/yarn to pnpm, and it might be handy to know them in advance.
Installation
On pnpm's installation page, you find many alternative ways of installing pnpm.
If you're using NodeJS 16.9 or higher, I recommend using Corepack. That's because Corepack is a NodeJS native tool that addresses managing multiple package managers.
In this case, you'll need to run the following:
corepack enable && corepack prepare pnpm@latest --activateOne tricky bit is that by doing that, you only will be able to run pnpm via corepack pnpm .... Meaning that every time you see a command like pnpm install, you'll need to run corepack pnpm install, for example.
To fix that, Corepack docs recommend we add bash/shell aliases:
alias yarn="corepack yarn"
alias yarnpkg="corepack yarnpkg"
alias pnpm="corepack pnpm"
alias pnpx="corepack pnpx"
alias npm="corepack npm"
alias npx="corepack npx"Or, on Windows environment, you can add functions using the $PROFILE automatic variable:
echo "function yarn { corepack yarn `$args }" >> $PROFILE
echo "function yarnpkg { corepack yarnpkg `$args }" >> $PROFILE
echo "function pnpm { corepack pnpm `$args }" >> $PROFILE
echo "function pnpx { corepack pnpx `$args }" >> $PROFILE
echo "function npm { corepack npm `$args }" >> $PROFILE
echo "function npx { corepack npx `$args }" >> $PROFILEAdding a dependency
The command to add a dependency is "add" instead of "install":
-npm install express
+pnpm add expressNpx
A lot of time, you'll find documentation saying you can run a global tool using npx for running a global tool without installing it first:
npx create-remix@latestThe pnpm equivalent is the `dlx` command (the same as in Yarn):
-npx create-remix@latest
+pnpm dlx create-remix@latestThere is another use case for npx, which is evoking the binary installed on the node_modules. It would be equivalent to doing ./node_modules/.bin/<binary>.
For those cases, you can change `npx` with `pnpm`:
-npx tsc
+pnpm tscThe "create" command
The create command is also present on pnpm:
-npm create astro@latest
+pnpm create astro@latestRun npm scripts
Pnpm follows the same principles to run npm script, you must use the "run":
-npm run test:unit
+pnpm run test:unitIn some cases, like start, dev, or test you can omit "run":
-npm start
+pnpm startWorkspace + commands
If you decide to use workspaces, you'll have to get used to two flags: --filter (-F) and --recursive (-r).
At the root level, if you want to run the dev server for your website, you'd need to run the following:
pnpm -F website run devNow, if you want to check which dependencies are available to be updated in all managed packages and projects, you'd need to add the recursive flag:
pnpm up --latest --recursiveConclusion
For me, pnpm is already a no-brainer decision.
Pnpm has many other options to fit almost every single type of project and needs.
I'd strongly advise you at least to try it out in a few projects and see what you few. The learning curve is very low, and it definitely worth the shoot.
And, if you have any problem, the website is rich in examples, the document is good, the community is great, and the maintainer is very active and open to questions and suggestions.
Peace!