Dual Booting with Engines and Gems
Gems are a central part in a Rails application, they help us add new functionality to our apps so we don’t have to reinvent the wheel, but also allows us to extract code to better organize the codebase and to share logic between multiple apps. In many cases, we have custom made gems, and we need to ensure they will work properly with the two Rails versions we run when we use the Dual Boot technique during upgrades. But… How do you dual boot the gems?
When we think about dual booting an application, we usually target specific versions of a gem in our main
Gemfile when we know which one is compatible with the different Rails versions we are using. For example, in one of our applications we have this:
# Gemfile if next? gem "jekyll", "~> 4.0.1" else gem "jekyll", "~> 3.7.4" end
We know that we need a different version of the
jekyll gem for each case.
The problem comes when the gem is not a public gem but an internal gem that is not yet compatible with the new Rails version.
Engines are a more specific case of a gem, so we’ll talk about gems in this article and the same ideas apply.
Non-Dual Booting Approaches
If the gem lives in a separate repository, one approach we can take is to update the gem’s main branch to only work with the new Rails version and then use the
if next? conditional to use different versions/commits/tags for each Rails version:
if next? gem "custom-gem", github: "my/repo", ref: "some-commit-sha" else gem "custom-gem", github: "my/repo", ref: "some-older-commit-sha" end
This is the simplest approach, but the caveat is that, during the upgrade process, we won’t be able to submit fixes or updates to the gem used by the current Rails version, since the main branch will only be compatible with the new version of Rails.
To prevent that, another slightly different approach is to create a separate branch in the gem’s repository, the main branch being compatible with the current Rails version and the new branch being compatible with the new Rails version and it gets merged at the end of the upgrade. Then, in the Gemfile, we can specify different branches for the source of the gem:
if next? gem "custom-gem", github: "my/repo", branch: "new-rails" else gem "custom-gem", github: "my/repo", branch: "main" end
This allows us to continue adding code to the gem that’s used by the app in production. But the caveat, in this case, is that we have to keep the branches in sync, so the new branch also includes the changes done in the main branch.
Dual Booting a Gem
If you noticed, those are the same two problems Dual Booting aims to solve when upgrading a Rails application, so we can borrow the dual booting idea and apply it to a gem: The gem adds compatibility with the next Rails version, without losing compatibility with the current one, allowing us to modify the gem at any point during the upgrade process ensuring that the latest version of the gem is compatible with both Rails versions.
Loosen the Rails Restrictions
The first limitation we typically see during our upgrades is the
gemspec file of the gem adding a strict dependency on the current Rails version. For example, in a Rails 5.2 application we tend to see custom gems with something like this in its gemspec:
# custom_gem.gemspec s.add_dependency "rails", "~> 5.2.4"
If we try to dual boot the main application with Rails 6.0, Bundler won’t be able to resolve a matching version of
custom_gem that is compatible, so we have to loosen this dependency. We don’t go too far, like with dual booting, we move up to the next minor version:
# custom_gem.gemspec s.add_dependency "rails", ">= 5.2.4", "< 6.1.0"
We allow any patch version newer than Rails 5.2.4 and up to any patch version of Rails 6.0
Now Bundler will be able to resolve this because our gem reports being compatible with both Rails 5.2 and 6.0. But this is not enough, we have to actually make the gem compatible with Rails 5.2 and 6.0.
Note that we DID NOT add conditionals in the
gemspecfile! we will add that in the corresponding
When dual booting, we ideally want to update the code to be compatible with the new Rails version in a way that the same also works with the current Rails version (we call them backwards-compatible changes). This is not always possible, so in many cases we have to add a conditional to execute different code for each Rails version. We can do the same in the gem:
if Rails.version ~= /^6.0/ # some rails 6.0 specific code else # some rails 5.2 specific code end
Test Suite Inside the Engine
Testing With a Dummy App
If the gem includes standalone tests (tests that run independent from the main Rails application), we typically test them using a dummy Rails application (this is really common for Engines). When that’s the case, we can dual boot the dummy application the same way we dual boot the main application, and then run the tests once for each Rails version. That way we can test that our code changes are safe in both Rails versions.
Tests Without a Dummy App
In some cases, the dependency is not with the Rails gem but with some railtie (like
active_support for example). In those cases the gem probably won’t have a dummy Rails application to run the tests since it doesn’t need a complete app, but it can still have tests. In these cases, we also have a
Gemfile and a
Gemfile.lock, and we can apply the dual boot idea here. There’s one detail to keep in mind… in this cases, the
Gemfile may look like this:
# Gemfile source "https://rubygems.org" gemspec
We can still dual boot this, we can create the
Gemfile.next symlink and add conditionals:
# Gemfile source "https://rubygems.org" def next? File.basename(__FILE__) == "Gemfile.next" end gemspec if next? gem "active_support", "~> 6.0.5" else gem "active_support", "~> 5.2.4" end
The strict version specified in the conditionals will tell Bundler which version of the game to use for each
Gemfile.lock file. This file is not used by the main application when using the gem, this is only used for the gem’s standalone tests (or when the gem is meant to also be run as a standalone application).
Note that the versions used there must be within the range specified in the
Then we can generate the new
Gemfile.lock.next (don’t do it from scratch! ) and run the tests with different sets of gems:
bundle exec rspec # and BUNDLE_GEMFILE=Gemfile.next bundle exec rspec
Since the Gemfile changed, we also need running
bundle install to keep the
Gemfile.lock in sync, because the gem’s dependencies restrictions changed. We don’t want to change any other gem except the one we are updating, so the changes in the Gemfile should be minimal and only reflect the new loosen restrictions and nothing more. For example, this is the expected change:
custom_gem (0.1.0) - rails (= 22.214.171.124) + rails (>= 5.2.4, <= 6.1.0)
There should be no update for the Rails version gem in the
Gemfile.lock file, gems versions should only be updated in the
Gemfile.next.lock . We want to make sure the application setup is not modified by our changes during the upgrade for the current Rails version.
Testing the Engine in the Main App Test Suite
In some cases, the gems don’t have standalone tests and the functionality is tested within the main application test suite. When that happens, we use those tests to verify the compatibility.
Custom Gems Inside the Main App Repo
When engines and gems live inside the main application repository, we can only apply the dual boot technique, since we can’t target a gem in the same source using a different commit hash or branch.
Bonus Tip: Support More Rails Versions
In some cases, a gem needs to be updated to support more than one Rails version. We can still apply the same technique, we use the
next naming convention because it fits a dual boot scenario, but we can always use the same idea and use a different name for the
Gemfile.next for each Rails version, and we can loosen the dependency even more:
We can have the main
Gemfile file, then symlink a
Gemfile-6-1 (and so on), and we can use the
BUNDLE_GEMFILE env variable to pick which tests to run and to add conditionals in the Gemfile.
Rails upgrades for applications with custom gems have been more common for us over the last few months, and it is important for us to handle upgrades in a way that is less disruptive to our clients’ workflow. Being able to dual boot the custom gems so our clients can continue working on them during the upgrade process and, at the same time, not having to keep an extra upgrade branch in sync is a win for all. It also gives the client the confidence that the changes we are doing are always compatible, and, if they need to change something on that gem during the upgrade, they can already test their changes against the next Rails version without waiting for the upgrade, helping them adapt their code for the future.