Socks On Before Shoes – a Smarter Dependency Manager

How do you update and manage your package dependencies if you’re not using GitHub? xMatters UI developer Bugra Firat tried upgreat as a dependency manager, and got good results. He shares his experiences in this article.

As we work in more and more projects in a modular manner, we end up creating a number of packages which all have dependencies of their own. Updating and managing these dependencies becomes a problem after some time. Some factors that contribute to this are slow development cycles (combined with fast publishes in the JS world), avoiding scope creep, and testing burdens.

So we sometimes end up with packages which work just fine, but have some outdated dependencies we want to renovate.

There are a few solutions to this problem of updating package dependencies, but none in the wild were quite right for us. The biggest players in this space are Greenkeeper and Dependabot (now acquired by GitHub!). It’s no mystery why GitHub acquired Dependabot, as it’s very tightly integrated into GitHub itself. Greenkeeper is also a great solution and very tightly integrated with GitHub, but it’s not free for private repositories.

What to do if you’re not using GitHub at all? Or you have private packages whose dependencies you still would like to keep updated?

Well, until now, there weren’t many dependency manager options, which left most of us doing it manually.

Updating packages manually. Ugh….

How many PRs to upgrade all dependencies?
Another issue that we’ve had with previous solutions is that, very cleverly, they do bare minimum of upgrading a dependency, and leave the rest to your CI/CD pipelines. This is great for most cases; but in a world where time means money, you might have to think twice about having to wait for and merge 25 pull-requests that update 25 different dependencies.

Found god on the 26th merged PR

Finally, instead of that evergreen utopia, you might want to make conscious and well-tested upgrades that help keep your sanity in your day-to-day environment.

We developed upgreat to address some of these issues and make it easier for us to manage, monitor and schedule dependency upgrades for our internal projects. Now we’re releasing it to the wild as an open source tool in hopes that it will be useful for other developers and organizations.

Another nuance here is that you can upgrade all of your devDependencies without your tests failing once. Sure, upgrade all my build tooling and run unit tests afterwards, what could go wrong? upgreat instead runs build scripts if you’re updating devDependencies to save you from headaches.

In a way, upgreat embraces the “manual-ness” and the human element of undertaking this task, while making things easier for the maintainer(s) at every step.

Socks on before shoes
On a cursory look you could think that doing such an upgrade is super simple. Get the list of all outdated dependencies, then loop over them and upgrade each one to the latest one. Done!

Or is it?

This was our first approach to the problem, but it proves to be rather fragile and too naive for any sufficiently complex package. Lets take a simple app with Webpack for example:

// package.json
"file-loader": "0.11.1",
"webpack": "2.5.1",

If you upgrade file-loader to 4.2.0 first, and then try to upgrade webpack to 4.41.2 afterwards, you will get into a state where your builds will be broken. Do you see why?

I just pretend I understand it perfectly.

It’s because file-loader versions have peerDependency to webpack versions. You need to upgrade the libraries in the correct order to avoid running into issues. Just like in real life, if you put your shoes on first, and then try to put your socks on, you will likely fail.

upgreat’s solution to this problem is straightforward, just like webpack, make and a host of other projects. upgreat sorts the packages to be upgraded in Topological Order. This makes sure that any packages that others in the list depend on are upgraded first, and avoids any nasty collisions.

Topo-what?
Scary as it might sound, topological ordering is a straightforward concept. When presented with a list of dependencies and a flow, we want to find an ordering that will acknowledge the dependencies (like file-loader@4.2.0 depending on webpack@4.41.2).

Think of it as the order in which you would put on your socks and shoes. The astute among you might realize that there could be many different topological orderings for a given dependency graph. The good thing is, they are all valid and will work just fine.

Here are some ways you can put on your socks and shoes:

Left Sock -> Right Sock -> Left Shoe -> Right Shoe
Left Sock -> Left Shoe -> Right Sock -> Right Shoe

and various permutations. Ultimately, the end state will be that you have worn socks and shoes on both feet. The order in which you do it (despite the heated arguments you might read on the internet about the right way of doing it) does not matter, as long as you don’t break the internal dependency between Sock -> Shoe.

Similarly for upgrading dependencies, the exact order of operations don’t make much difference, as long as the inter-dependencies are addressed.

Silver bullets and panaceas
I don’t think I need to tell you that there is no foolproof and fully-automated way to upgrade dependencies of your projects. Unless you don’t write tests and live on the wild side, but then you could just include package@latest until everything breaks down one day, including your sanity.

After trying a few of these approaches in xMatters, we’ve found this one works okay for our purposes. It automates the most grueling parts of the library upgrade process, but after all is said and done leaves it up to you to follow through and commit the changes. You can pretend you did it all by yourself, I won’t tell.

As a developer it feels great to be able to give back to the community who have basically provided for my livelihood over the years with amazing open source projects. I hope to be involved in more projects as we embrace open-source in xMatters and share some of our ideas and other projects with the community from which we all benefit greatly. Check out upgreat on Github. I would love to hear your thoughts on your experiences doing library upgrades and opinions on upgreat.

xMatters is a digital service availability platform. In other words, it’s an incident management platform that prevents IT issues from becoming business problems, and you can use it free forever!

Request a demo