Can You Remove DatabaseCleaner?
DatabaseCleaner was a very popular gem that helped you keep a clean database between tests. However, newer versions of Rails removed the need for this tool for most cases. Even so, many applications that always used it still use it to this day. Let’s talk about what it does, why it was needed, and why you probably don’t need it anymore, and when you still might.
What is it
DatabaseCleaner is a gem that you can use to ensure that every test you run starts with the expected database state. DatabaseCleaner supports different databases: relational databases (MySQL, PostgreSQL, SQLite, MariaDB, for example), MongoDB, and Redis. In this article we’ll focus on relational databases which is the default in general for Rails apps using ActiveRecord and the main use case we see for our clients.
The DatabaseCleaner project provides a specific gem to work with applications using ActiveRecord: database_cleaner-active_record . This provides 3 strategies for database resets: transaction (the default), truncation, and deletion.
The main use case for DatabaseCleaner was to handle tests that required multiple threads or processes (for example, a Capybara test that needs both a Ruby/Rails thread controlling the test, and a second thread running your Rails app for the browser to click around and interact with it).
Transaction Strategy
This is the strategy that most Rails developers should be familiar with already. Rails itself has a use_transactional_fixtures/use_transactional_tests configuration (supported by both Minitest and RSpec).
The main idea for this strategy is to wrap each test in a transaction and roll it back at the end of the test. Note that this is the same thing Rails already does with its native configuration: if you are already using this strategy, you have a clue that you might be able to remove this gem and use the Rails native feature.
Truncation Strategy
This strategy does NOT use transactions. Instead, it will run a TRUNCATE database statement on each table. The main benefit of this strategy is that, since it does not use transactions, multiple threads can interact with the database and see the same state. The main trade-off is that truncation takes extra time to run, especially when it truncates every table and not just the ones involved in a given test.
Another problem with this strategy is that, if the database depends on seed data, the truncation does not know which records changed. It removes all records, so the data needs to be added back, which takes even more time.
Deletion Strategy
This is similar to the truncation strategy, but the main difference is that it does NOT reset the incremental ID tracking for the tables. After a table is truncated, the next new record added will have ID “1” (if we are using an auto-incrementing int/bigint column as the primary key). After the contents of a table are deleted, the next auto-generated ID will be the one that follows the last ID that was generated.
Other than that, it has similar benefits and drawbacks as the truncation strategy: it’s useful when you cannot use transactions, but it adds extra database queries that take more time.
Rails changes, shared database connection
At RailsConf 2017, Eileen Uchitelle gave a talk showing all the improvements made in Rails 5.1 related to system tests (also known as end-to-end or e2e tests). There’s a specific moment of that talk that is relevant here: DatabaseCleaner not required . She explains the main problem that was solved: sharing the same database connection between the test thread and the app server thread.
As you can see, this has been solved since 2017, Rails 5.1, 9 years ago.
So, can I remove it?
The short answer is: most likely, yes. But keep reading because there are still some edge cases where you might want it, or you may need to do a bit of extra work after removing it.
As mentioned before, this is focused only on the
database_cleaner-active_recordgem (it would be similar for thedatabase_cleaner-sequelgem) as it can be replaced with Rails’ built in transaction handling. If you use DatabaseCleaner for other databases like MongoDB or Redis you shouldn’t make a decision based on this.
Removing DatabaseCleaner
So, what are the steps to remove it? First, you need to understand why you are using DatabaseCleaner in the first place. Most applications simply add DatabaseCleaner with the basic setup, but we have seen applications with a more complex setup. Some use custom Rake tasks that run DatabaseCleaner between browser tests, as part of custom integrations with tools that Rails does not support out of the box (like Cypress, Playwright, Protractor, or Testcafe).
Edge cases
If you have a custom setup for system or e2e tests, you may still need DatabaseCleaner or something like it.
For example: one of our clients had a custom setup to run browser tests using Protractor. The tests were started from a Rake task that internally started a Rails application in one process and then, once the server booted, it started a JavaScript process running Protractor tests.
Between each test, the Protractor runner would run a rake task to clear the database.
Things to pay attention to
The easiest way to test if you really need DatabaseCleaner is to remove the gem and any related setup code, configure use_transactional_tests = true (or use_transactional_fixtures), and run your tests.
If everything works, run them a few more times and you are good to go.
But it may not “just work”, the test suite can depend on some state that was always ensured by the extra work done by DatabaseCleaner and we need to address that.
Some tests may depend on specific IDs. When using the truncation strategy, we always get the auto-generated primary-key value starting from 1 (if we are using an auto-incremental type), so tests can end up relying on records having specific IDs. You’ll have to fix them to work with any ID (or you may want to update a record’s ID in the setup step of the test).
Tests can also be order-dependent. If DatabaseCleaner was configured to run in an after(:all) hook instead of an after(:each) hook, there might be some order-dependency in the tests that required the database to only be cleared after all the tests of a given test file were done. In those cases you’ll have to make sure each test has the correct setup to run alone. This is also really important if you plan to add parallel testing or randomized test ordering, and we recommend both!
Finally, check for any non-default DatabaseCleaner configuration. The truncation and deletion strategies support different options . It is important to know why some of those options were configured, as they may be important and may require additional setup for specific tests.
These are not the only possible issues, as every application is different and other problems can show up when you remove DatabaseCleaner.
A Case Study
Recently, we worked for a client that wanted us to see if we could improve their tests. They already had a parallel test setup (using the flatware gem in this case), but, as I mentioned in my article about parallelization , parallel testing is not the only improvement that can be done.
While reviewing logs and analyzing the tests, we noticed many TRUNCATE database statements, each only took milliseconds, but this gets slower the more tables you have, since each table requires one more TRUNCATE, and it multiplies by the amount of tests you have!
The application uses Capybara, RSpec, and MySQL, and we ran a quick experiment: remove anything related to DatabaseCleaner, set config.use_transactional_fixtures = true in the rails_helper.rb file, and run the tests both as a single process and with parallelization. We had some test failures of course, but only around 15 failures in a test suite with 10k+ tests which was easy to fix.
These are the numbers for the whole test suite (10k+ tests), both with and without parallelization, and with and without DatabaseCleaner:
rspecwith DatabaseCleaner: 25m28srspecwith Rails’ transactional tests: 20m15s (20% reduction)flatware rspecwith DatabaseCleaner: 3m34sflatware rspecwith Rails’ transactional tests: 2m15s (37% reduction)
The
rspeccommand runs all tests in a single process. Theflatware rspeccommand runs all tests in multiple parallel processes. All these tests were run on a 14-core M4 machine.
The results show a noticeable speed improvement, both for single-process and parallel processing!
It is important to note that the main reason for these percentages is that the application was using the truncation strategy and a MySQL database: multiply each TRUNCATE statement by the number of tables and by the number of tests, and even a 1ms database query for a single table can easily add up to tens and hundreds of seconds that we are now avoiding. With PostgreSQL, DatabaseCleaner uses a single TRUNCATE statement targeting all tables, which might be faster.
The reason this app was using the truncation strategy was that it was added way before Rails 5.1 solved the shared transactions problem, and it was never reviewed after that.
Conclusion
Removing DatabaseCleaner for this application had many benefits and no negative side-effects.
The tests run faster locally, which is always good, but it’s even more important now with agentic programming, speeding up the feedback for agents that will run our tests multiple times.
Tests run faster in CI. The main benefits are a faster green CI state, which allows PRs to merge sooner, and reduced compute time, which saves money.
We have fewer dependencies and less code. We were able to remove a gem and the extra code used to configure it. Now we know that if Rails changes in a way that makes DatabaseCleaner incompatible (this happened when Rails 7.2 was released ), we don’t need to worry about it!
Even if you don’t get the speed benefits for your particular setup (if you were already using the default transaction strategy), the last benefit alone can be a good reason to check if you can remove it.
Do you think your test suite could run faster? We can help with that, send us a message!