Mocking JavaScript Requests During Tests

When we run tests, we don’t want to hit external services in most cases, so our tests don’t depend on external services and are more stable. We can use gems like VCR opens a new window or WebMock opens a new window to stub the requests that are done by the Rails application, but when the request is initiated by the JavaScript code… that’s a different story.

Two Different Approaches

In both cases we run tests that use the Capybara opens a new window and Selenium-webdriver opens a new window gems. This is the default when creating a new Rails application so you should already have them in your Gemfile. The tests can be of type feature, integration, system, etc, because, as long as the type of test uses Selenium and Capybara, we can proxy or intercept the requests.

Proxy Requests

With this method, the browser is started with a proxy configuration pointing to a local proxy that will catch any request and let us handle the different requests. From the browser’s point of view, the request is done normally and the interception is handled by the proxy server. We won’t explore this approach in this blog post but if you want to go with this solution you can try the capybara-webmock opens a new window gem by Hashrocket.

Intercept Requests

When using this method, we’ll intercept the browser requests directly in the browser using some devtools features, and the driver will execute a callback in our Ruby code any time a request is done. In this case, the browser is intercepting the request and the driver is running our ruby code to generate the response (or let it continue).

Intercepting with Selenium

Selenium-webdriver version 4 introduced a new feature that allows adding a driver extension to intercept network requests with the HasNetworkInterception opens a new window module. So we’ll need to specify the version in the Gemfile:

gem 'selenium-webdriver', '>= 4.0'

Interceptor Module

We created a small Interceptor module opens a new window that can be used with both RSpec and Minitest. We can copy that file in spec/support/interceptor.rb or test/support/interceptor.rb depending on the testing tool being used.

Selenium-devtools

For the HasNetworkInterception extension to work, we need to add the selenium-devtools opens a new window gem too:

# Gemfile
group :test do
  # Adds support for Capybara system testing and selenium driver
  gem 'capybara'
  gem 'selenium-webdriver', '>= 4.0'
  gem 'selenium-devtools'
  # Easy installation and use of web drivers to run system tests with browsers
  gem 'webdrivers'
end

MiniTest Setup

In the test/application_system_test_case.rb file, we have to require the module and include it in the ApplicationSystemTestCase class. Then we have to add some code using the lifecycle hooks:

# test/application_system_test_case.rb
require "test_helper"
require_relative "support/interceptor"

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  include Interceptor

  driven_by :selenium, using: :chrome, screen_size: [1400, 1400]

  def after_setup
    start_intercepting
    super
  end

  def before_teardown
    stop_intercepting
    super
  end
end

RSpec Setup

In the spec/rails_helper.rb file, we have to require the module and include it for specs of type system (or feature if you use those). Then we have to add some code using the lifecycle hooks:

# spec/rails_helper.rb
require "spec_helper"
...
...
require_relative "support/interceptor"

RSpec.configure do |config|
  config.include(Interceptor, type: :system)
  config.before(:each, type: :system) do
    # this `driven_by` call is needed because of this issue in rspec-rails
    # https://github.com/rspec/rspec-rails/issues/2550
    # the `drive_by` method is valid only for system tests, ignore this fix if type is `feature`
    driven_by Capybara.javascript_driver
    start_intercepting
  end

  config.after(:each, type: :system) do
    stop_intercepting
  end

You can do a similar setup if you use other type of tests that may use Capybara and Selenium, like feature or integration tests.

Intercepting Requests

Now we can use the intercept method to set fixed responses for specific urls.

For example, if we have a page with a button that says Make SWAPI request, that does a JavaScript request to The Star Wars API opens a new window , and finally puts the response in a div with id response, we can test it like this:

# test/system/index_test.rb
require "application_system_test_case"

class IndexTest < ApplicationSystemTestCase
  test "The Star Wars API request" do
    visit root_path

    intercept("https://swapi.dev/api/planets/1/", "my mocked response")

    click_button "Make SWAPI request"

    assert_selector "#response", text: "my mocked response"
  end

When the browser does that external request, the selenium extension will execute the callback defined by our Interceptor module and will respond, to this specific url, with the fixed string instead of doing a real request to the external API.

Same method can be used in RSpec specs.

Configuring the Interceptor

Default Interceptions

It’s not uncommon to have some global JavaScript that does external requests in many pages (like analytics code), use some external CDN (for web fonts or icons) or external widget (like Twitter’s or Facebook’s). We can define interceptions that will be used by any test by default so we don’t have to remember to intercept them for each test.

We can do that by overriding the default_interceptions method. Let’s say we want to intercept any request to Google (Maps, WebFonts, Analytics, etc) during tests:

module Interceptor
  def default_interceptions
    [{url: /google.*\.com/, method: :any, response: ""}]
  end
end

Check the comments in the interceptor.rb file for the valid values.

Note that the Interceptor module will intercept any request that it’s not explicitly allowed by default. Overriding the default_interceptions helps make this explicit and allows setting expected responses instead of just an empty string. But just by calling the start_intercepting method we’ll prevent any external request.

Allowed Requests

By default, any external request that’s not explicitly intercepted or allowed will be intercepted with an empty response and logged to the console. If the tests happen to depend on this external endpoint, the test will fail and we can add an explicit interception or allow it. If the test does not depend on the external endpoint, we have a free improvement by not doing that unnecessary request.

We can change this by overriding the allowed_requests method. If we have a CDN server with assets that are important (like a JavaScript library), we can allow that external request with:

module Interceptor
  def allowed_requests
    [%r{http://#{Capybara.server_host}}, {url: my_cdn_domain, method: :get}]
  end
end

Check the comments in the interceptor.rb file for the valid values.

Note that you probably always want the Capybara.server_host url to be allowed.

Conclusion

Just by adding this interceptor to some projects, we found many unnecessary external requests that were being triggered during tests (fonts, icons, widgets, analytics). By inspecting the logs, we were able to identify them and add the proper interception rules to be explicit. This reduces the bandwidth used during tests and the time it takes for the page to be responsive for Capybara to start interacting with it.

The main advantage was that we could find some tests that were relying on external endpoints, tests that can eventually fail if the external endpoint is unreachable or be slow if the external endpoint is working slow at that time. By intercepting those requests we can have a more robust test suite and improve the quality of the tests.

Finally, we created a sample app opens a new window with this setup.