TL;DR: node 10 => node 18, webpacker 3.5 => shakapacker 7.1, webpack 3 => webpack 5.

For over a year I’ve been working on an old Ruby on Rails application that started around 2009, so it has many in-house Rails features because they were not part of Rails yet.

Several years ago it started using React (and Elm, but thankfully that’s no longer the case) with webpacker 3.5 and the JS instrumentation grew explosively but webpacker was never updated — according to the main branch of git, maybe there ware attempts but weren’t merged.

A mechanic is under a Volkswagen Beetle giving maintenance to the engine.
Software can be beautiful with proper maintenance.

From my first day I had issues with tech debt as I was provided an M1 laptop and couldn’t get the application running, as the dependencies were:

  • Ruby 2.6.6 – not compatible with M1. Maybe compatible with Rosetta? I preferred the slow process of upgrading ruby rather than hoop around.
  • Node 10 – not compatible with M1
  • Elm 19 – not compatible with M1 and no community. Not even with Docker did the compiler work
  • Webpacker 3.5 – abandoned project, replaced by Shakapacker
  • Ruby on Rails 5.2

After a couple of months I upgraded ruby to v2.7 so that was running natively in the hardware, but the huge front-end codebase lacked behind with lots of flavors and dependencies intertwined: jQuery, React, in-house, Elm, SASS, LESS, SCSS. A senior developer on the team removed the Elm portions of the app right before leaving the company and without him I don’t think I would’ve been able to do this, I would’ve tagged this project as abandonware in my mind and moved on.

Hundreds of internal source files, thousands of vendor assets & countless NPM dependencies… it was a dire situation.

Different Experiments, Multiple Attempts

The adage says that we should upgrade one major version at a time. In this case, though, I started by trying to upgrade node 10 to node 12 as it’s the next LTS (Long Term Support) version – both versions were unmaintained at the time.

This didn’t work as some NPM dependencies required node <= 10 and upgrading dependencies broke a bunch of other things.

Next attempt was going to node 14 (unmaintained LTS) which allowed to upgrade a lot of dependencies but there were still a crucial few which crushed this attempt, node-sass among them which is also unmaintained.

I spent about a month installing thousands of dependencies, seeing log errors and clearing busted docker images and cache layers… At this point I felt defeated and abandoned the effort for a few months.

I continued working on other ruby/rails aspects of the application which gave me confidence on how the application worked, how it failed and overall set expectations on what was working and what didn’t, but none the less was expected behavior.

Another Shot With Clear Expectations

So after about a 3 month long rest, I decided to give it another shot and to look at webpacker as a black box and care only for the output assets, i.e., the output might be syntactically different but should otherwise produce “equivalent” JS code. So I tried upgrading to node 18 (LTS and works on the M1).

It’s important to note I was prepared for the thousands of lines of error logs, cryptic error messages and slow change-compile-repeat cycles. I consider morale the most important factor for maintenance upgrades. Seeing so many errors coming from files you didn’t know exist sucks the life out of you.

With this in mind, I set these expectations for myself:

  • JS MUST compile
  • CSS MUST compile
  • MUST work natively on the M1
  • MUST NOT change the test suite expectations — as much as I dislike how cumbersome selenium is, it’s an invaluable investment the previous developers did for this application
  • Minor customer-facing features MIGHT break and I will fix them iteratively after the fact

It became so much easier to chew the problem. It was still boring seeing a compiler error, changing a file and checking again ad nauseam but it wasn’t frustrating anymore. Most of the issues were SCSS syntax used in SASS files and some asset location joggling, e.g., images, but after a few weeks the code was compiling successfully and it became a red-green cycle with the test suite. All in all, it took about a month to get a release candidate branch which was daunting to review with around 650 files changed.

Simplifying the Pull Request

I like fixing things along the way when I work on something (who knows when another opportunity will present itself) but with this PR (Pull Request) I decided to make as few changes as possible AND also slip simple changes to other PRs which were being merged regularly so instead of not fixing issues along the way, I was fixing them earlier, e.g.:

   <%- content_for(:js) do %>
     <script>
-      initFeatureSPA();
+      window.addEventListener('DOMContentLoaded', function() { initFeatureSPA(); });
     </script>
   <% end  -%>

Something changed in the way JS assets were being loaded which meant <script> elements were being executed before scripts finished loading (now they’re deferred?). I don’t care enough to go and understand why but changing these JS snippets to the Correct™ way fixed a lot of the failing tests albeit added a lot of noise to the PR, so I merged them to the main branch before the big upgrade, which made the review process much more manageable.

After all was said and done, the PR went from 656 files down to 140 – not messing too much with webpack configuration being the biggest factor at play.

Conclusions

It’s a daunting process and it may seem like an unworthy effort, but with the right mindset it becomes an almost easy process. It’s just slow as hell. I spent about 6 months with the multiple experiments and keeping the problem in the back burner of my mind when not working on it – it’s an iterative process!

On Management

Development aspects aside, Management™ was never onboard with all of this. They weren’t openly against it but didn’t see the upside. From my perspective, it improved my workflow tremendously as I no longer work with the overhead of Docker’s VM (still use it for DB & Redis) and the battery life of my laptop is longer than a day now.

That’s probably why management doesn’t care. It doesn’t add business value directly. But as part of this, I upgraded a bunch of dependencies, found the root cause of recurring production issues and fixed many long-hanging-fruit problems that would’ve otherwise remained in this codebase on the brink of abandonware.

All this to say, as Developers™ we should try to address tech debt at least indirectly even when lacking support from Management™ — it makes our day to day much more enjoyable and allows us to add business value at a faster rate.

"Beetle Buggy" by Ryan McGuire is marked with CC0 1.0 .