Rails World '26: Rails in the Agentic Era
I was in the audience at Rails World 2026 for Ryan Sherlock’s talk, “Shipping is your heartbeat: Rails in the agentic era.” Ryan is Senior Director of Engineering at Fin , the company formerly known as Intercom, and he shared how a very large Rails monolith gets code from a pull request into production.
And it is large. The app is 15 years old, has about 5 million lines of code, and gets deployed more than 200 times a day, with roughly 5 minutes between merging a pull request and seeing it live. In a post on X after the talk , Ryan added that they run about 230,000 tests (25,000 of them added in August alone) and ship a new backend every two to three minutes.
The traffic is large too. At peak, that one monolith handles 150,000 web requests per second and 100,000 async jobs per second, along with millions of database calls and tens of millions of cache queries per second.
Most of the talk was about unglamorous work: test selection, boot time, build artifacts, and rollbacks.
Shrek, their in-house review agent, only shows up near the end, and it rests on everything that came before it.
In this article, you will learn how Fin selects and packs tests, how they cut boot and build time, how they deploy and roll back without humans, and how Shrek decides when a pull request can merge without a second human.
Test Selection
The first idea was simple: don’t run the whole test suite on every pull request.
Fin records which application files each spec touches and builds a map from source files to specs. Simplified, the example on Ryan’s slide looked something like this:
app/models/conversation.rb:
- spec/a_spec.rb
- spec/b_spec.rb
- spec/c_spec.rb
app/models/billing.rb:
- spec/d_spec.rb
If your pull request only changes conversation.rb, CI runs specs A, B, and C, and skips D.
Ryan was upfront that this is risky, so there are safety nets. A required set of specs always runs, covering core functionality and security.
Changes in certain directories skip the guessing and run the full suite.
The map refreshes every 30 minutes, so any spec added since the last refresh runs too. And if you don’t trust the selection, you can push an empty commit to force a full run.
Selection can still miss something when a few pull requests are green on their own but break when combined. That’s why the suite runs again before anything reaches production. When that run fails, the pipeline gets blocked, and Fin treats it as a “shipping incident.”
The results were worth the risk.
CI test time went from just over 7.5 minutes to about 3, and a selected run uses roughly 5% of the compute of a full one.
Ryan said that if they hadn’t done this, their projected CI bill would have been “the GDP of a small nation.” 🔥
Together with the rest of the work in this post, it brought them back to budget. A slow, expensive test suite costs more than the CI bill, too, which we covered in The Hidden Cost of Your Test Suite .
Running fewer tests also means hitting fewer flaky ones. At Fin, a flaky spec is a P1 that the on-call engineer has to fix right away, so not running into one in the first place helps a lot.
If flaky tests are a problem in your app, we wrote about how to track them down .
Packing CI Workers by Duration
Once you know which specs to run, you still have to split them across workers. Fin splits them by duration, not by file count.
Every time a spec runs, its runtime gets recorded, and the next run uses those timings to pack specs into batches with roughly the same total runtime, so the slowest worker doesn’t set the pace for the whole run.
A full run uses about 1,400 workers. A selected run gets as many workers as it needs to finish in about three minutes.
The idea scales down to smaller teams: parallel_tests can group specs by recorded runtime too.
We covered a local setup in How to Parallelize Your RSpec Test Suite , and how we took a client’s suite from 40 minutes to 4 .
Boot Time
Boot is a cost you pay every time, whether you’re starting a test run or deploying.
Ryan started this part by asking who was running something older than Rails 5.2, the version that made Bootsnap a default gem.
Bootsnap caches compiled bytecode so later boots can reuse it, but by default it fills that cache lazily.
Fin precompiles it when building its images, so every CI worker, deploy, and dev restart starts warm. This alone took their boot time from 40 seconds to 20 seconds! 🚀
Fin has been using schema cache for years, even before Ryan joined Fin.
When a Rails process touches a table for the first time, it asks the database for that table’s columns. That’s fine with a handful of processes, but with hundreds of them restarting during a deploy, it caused a mini outage every time they deployed.
Now they dump the schema cache (bin/rails db:schema:cache:dump), upload it to S3, and every process downloads it at boot instead of asking the database.
The last idea was to stop doing work when the inputs didn’t change. Fin reuses the post-migration schema cache until schema.rb or db/migrate changes, and skips bundle install unless the
Gemfile or Gemfile.lock changed.
Each of those saves five or ten seconds, which adds up over thousands of runs.
Build and Artifacts
Fin used to build six deploy artifacts: Two CPU architectures times three regions.
The second architecture only existed for a PDF generation feature. A new engineer on the Berlin team took it as an onboarding project, found another way to generate those PDFs, and got the build down to three artifacts.
There wasn’t a real reason to build one per region either, so now there’s one build. Fewer builds means less compute and fewer flakes, since a flaky step run six times fails more often than one run once.
Then they looked at what was inside the artifact. For years, they had been shipping about 500MB of specs, node_modules, and other files production never needed to every instance: compressed, sent over the wire,
and uncompressed on the other side.
Cutting that dead weight made the artifact 26% smaller.
My favorite story in this section was about assets. A sensible-looking config change had disabled asset caching, and nobody noticed for five years.
That added about two minutes to every build the whole time.
If your assets:precompile step feels slow, it’s worth taking a closer look at it .
All together, the build went from about 4.5 minutes to 53 seconds. Ryan was clear that there was no magic involved.
In his words, the biggest lesson was to “measure everything. Know where every second goes,” and then cut, cache, or optimize whatever you find.
Deploying with Muster
While the test suite runs, the build also goes through smoke tests, connection checks against critical infrastructure, and various security checks .
Then Fin uploads the artifact to S3 and uses AWS Systems Manager (SSM) to tell thousands of EC2 boxes there’s a new version to download. Each box rotates its processes, which takes 3 or 4 minutes, and keeps a few previous versions around for rollbacks.
All of this shows up in Muster, Fin’s internal deploy dashboard. Their CTO started building it almost 15 years ago. It helps their engineering team keep metrics front and center, so engineers can see how quickly their change gets to production.
Monthly deploys of the monolith went from about 150 in 2012 to 3,380 in August 2026, with the steepest growth in 2026.
Heartbeat Metrics and Automatic Rollback
What customers do with a change is the input for the next one. So Fin monitors what they call heartbeat metrics: customer outcomes in addition to system health. Are their customers’ customers sending messages? Are tickets moving through the inbox? Is Fin resolving conversations? A deviation in any of those means something is off.
Ryan walked us through an incident from August 2025. An engineer shipped a change with a bad query, and one of those metrics dropped. The alarm waits 2 minutes before firing (30 seconds caused too many false positives).
When it fires, it creates an incident, pages the incident manager and engineering, and rolls back production automatically. Because every change is small, rolling back is considered safe, but only up to five versions or about 20 minutes back.
No human was involved in that recovery, and it was a 6- or 7-minute incident. Four or five years ago, Ryan said, engineers would have shown up, tried to fix forward, and eventually rolled back, for about 15 minutes total.
Today it would take about 4 minutes.
Treating Every Change as High Risk
An auditor once asked Ryan which deployment categories Fin had for low, medium, and high risk changes. The answer was that they don’t have any.
Every change is treated as high risk until proven otherwise, whether it changes the copy of a button or points the application to a new database.
Every change gets the same baseline controls: review, testing, pre-production smoke tests, a fast staged rollout, and automated rollbacks.
Code got cheap. About 95% of Fin’s pull requests have been written by agents for about 6 months, which puts a lot of pressure on code review.
Their first attempt at AI review, about 2 years ago, didn’t go well.
The reviews came back as very long comments that Ryan described as “a love letter to your code,” with maybe a few real problems buried at the end.
Nobody reads 2,000 words, so engineers skimmed them and went looking for a human reviewer.
Fin wanted to raise the floor of review quality, especially as the growing number of pull requests made it more tempting to type LGTM and move on.
Shrek: An Agent that Approves Pull Requests
So they built Shrek (short for Safe Heuristic Review Engine for Kode), a review agent, using the same techniques Fin uses to build its AI product.
The Five Stages
Shrek reviews every pull request in 5 independent stages, mostly running on Codex, while most of Fin’s code is written with Claude.
(1) The first stage is the problem statement: Does the pull request description explain an understandable problem and the hypothesis behind the change?
If it doesn’t, neither a human nor an agent can tell whether the code does what it’s supposed to do.
(2) The second is intent. If you’ve ever asked an agent to do A and watched it do A, B, C, and D, you know why this stage exists.
Shrek checks that the code does what the problem statement says, and nothing more.
(3) The third is safety, a set of rules per repository.
A change to a core model, to security code, or a destructive action gets a different review than a change to a button’s copy.
(4) The fourth is logic, where Shrek looks for bugs.
It’s the most expensive stage and where most of the time goes.
(5) The fifth is best practices.
A large app accumulates its own rules over the years, and most teams are bad at documenting those.
So Fin took the top 10 or 20 reviewers in each repository, mined the feedback they had left on pull requests over the years, and had those engineers curate the result.
Giving agents that kind of structure is something we’ve looked at from the Ruby side too, in What Happens to Your Ruby Tools With AI? .
Backtesting
Writing good reviews is one thing. Approving code is another.
Fin built a dataset of past changes that went to production and caused an issue the human reviewer missed. Shrek has to catch those, so that any approval it gives is better than the human approvals that came before it.
Every prompt change and every model change gets backtested against it, because a newer frontier model once made some stages worse, finding ghost issues that weren’t there.
What it looks like in a pull request
Ryan showed Shrek’s comment on one of his own pull requests, a performance improvement.
Shrek found a race condition, so it wouldn’t approve the change. It also flagged that the change had more than 150 meaningful lines, the auto-approval limit for that repository, so it needed a human either way.
There’s a loop on the author’s side too. When the code was written with Claude, Claude watches the pull request, sees Shrek’s comment, and starts fixing the race condition without a human stepping in.
Ryan summarized the guardrails in 4 points.
Shrek is independent:
- There’s no case where an agent creates a pull request and an agent approves it without a human involved, which is what makes Shrek’s approval different from a fully automated pipeline.
- It’s evidence-gated, since all the baseline controls still apply and everything is auditable.
- Its authority is bounded to the repositories where they’ve done the backtesting.
- And the human driving Claude or Codex is still accountable for the change.
The last one is a best practice our team cares about deeply: If a human reviews your changes, you need to be able to explain the changes without asking Claude or Codex why they happened. No one at FastRuby.io can delegate that responsibility to an AI agent.
Results
The median time from opening a pull request to merging it went from about 160 minutes with human reviewers to about 36 minutes. Ryan pointed out that this is hard to compare, since Shrek currently approves somewhat safer changes than humans do.
About 40% of Fin’s pull requests are now approved by Shrek, with only one human in the loop.
Their revert rate is about 0.5%, roughly 1 in 200, compared to over 5% for human-approved pull requests.
Again, not apples to apples, because humans are approving the riskier changes.
The result I liked the most was about pull request size. People expected agents to create massive pull requests, and Fin saw the opposite.
The 150-line limit works as a sociotechnical forcing function: if you know Shrek won’t approve anything bigger, you split your work.
Their median pull request is now about 60 lines.
Fin also wrote about their PR review agent in AI Is Approving Our Pull Requests back in April 2026, when just over 19% of their pull requests were auto-approved. Going from that to about 40% in a few months shows how quickly it has grown.
Conclusion
Ryan was honest about what isn’t solved yet. Flaky tests are still a problem, and he wondered whether they’re a smell of architectural drift. Rollbacks still take 3 to 4 minutes, and they want to kill processes faster instead of rotating them gracefully.
I believe the advent of AI-DLC (AI-Driven Development Lifecycle) workflows is going to force engineering teams to get serious about mutation testing. Some of our customers are using this practice to test the quality of AI-generated test code and seeing promising results (e.g. fewer flaky tests introduced to the
mainbranch).
The caveat for the rest of us is that Shrek sits on top of years of pipeline work. Auto-approval is only safe because of the test selection, the small changes, the heartbeat metrics, and the automatic rollbacks underneath it. For most Rails teams, the earlier sections of this post are the place to start.
When I attended Rails World, this is the kind of talk I was interested in: What does the future of Rails look like at scale? How are big corporations using AI agents in a way that keeps humans in the loop? How do the Ruby and Rails ecosystems support AI-driven development workflows?
Ryan’s talk clearly showed us the future of Rails at scale. If you’re curious about where DHH thinks Rails is going, I wrote about his opening keynote in DHH, AI, and the Future of Rails .
If you have half an hour, I recommend watching the full talk:
Interested in how AI-driven development workflows could work in your Rails app? Check out our AI Readiness Assessment for Engineering Teams .