Making AI Agents Safe to Retry
Over in the OmbuLabs.ai blog, our post on the ReAct pattern in Ruby walked through building an agent that reasons about a task, calls tools, and acts on the results, all inside a single Ruby script. That’s fine for a command-line experiment. It’s a different story once that same agent needs to run inside a Rails app, triggered by a web request, where a multi-step reasoning loop can easily take longer than any request should stay open.
The obvious fix is to move the agent into a background job. Sidekiq already handles work that takes a while, so an agent’s loop of thinking, searching, and acting should fit right in. It does, mostly, but a job’s retry semantics were built around retrying a database write, not a chain of API calls that has real, sometimes irreversible, side effects.
In this article, you’ll learn why an agent’s reasoning loop doesn’t survive a naive move into a background job, how to make its side effects idempotent against Sidekiq retries, and how to stream its progress to the user while it runs.
Why Agents Don’t Belong in the Request Cycle
Picture a support ticket coming into a Rails app. A reasonable first instinct is to trigger an agent right there in the controller: it reads the ticket, searches internal docs for anything relevant, drafts a reply, and tags the ticket before the response ever renders. Each of those is a step in the ReAct loop, thought, action, observation, repeated until the agent decides it’s done.
The trouble is that each step is also a network call to an LLM provider, plus whatever tool it decides to use. One search, one draft, one tag isn’t so bad. A ticket that needs two rounds of searching before the agent has enough context to draft a reply is a different story, and there’s no way to know in advance how many rounds a given ticket will need. On Heroku, a web request has 30 seconds before the router gives up and returns an H12 timeout, and that’s an optimistic budget for a loop whose length depends on what the model decides to do next.
Holding a web worker open for that long is also just wasteful. Puma has a fixed number of worker threads, and every request stuck waiting on an agent is a thread that isn’t serving anyone else. This is the same argument that’s applied to slow jobs since long before agents existed: a 10,000+ row CSV upload was enough to time out a browser . It’s just that an agent’s loop is often slower and less predictable than a fixed-size file.
Sidekiq is the obvious place to put this kind of work: queue a job, return a response immediately, let the ticket update once the job finishes. The rest of this article is about what happens once you actually do that.
Moving the Agent into a Sidekiq Job
We’ll build on the same shape as the ReAct pattern post: an agent with a few tools, looping through thought, action, and observation until it’s done. Here, the tools are specific to ticket triage, searching internal docs, drafting a reply, and tagging the ticket:
# app/services/ticket_triage/tools.rb
module TicketTriage
class Tools
def initialize(ticket)
@ticket = ticket
end
def search_docs(query)
InternalDocs.search(query).map(&:summary).join("\n\n")
rescue InternalDocs::Error => e
"Search failed: #{e.message}"
end
def draft_reply(content)
@ticket.update!(draft_reply: content, status: "drafted")
"Draft saved."
end
def tag_ticket(tags)
@ticket.update!(tags: tags)
"Tagged with #{tags.join(", ")}."
end
end
end
The function definitions follow the same JSON shape Claude expects, one per tool:
FUNCTION_DEFINITIONS = [
{
name: "search_docs",
description: "Search internal documentation for relevant context",
input_schema: {
type: "object",
properties: { query: { type: "string", description: "What to search for" } },
required: ["query"]
}
},
{
name: "draft_reply",
description: "Save a draft reply to the ticket",
input_schema: {
type: "object",
properties: { content: { type: "string", description: "The drafted reply text" } },
required: ["content"]
}
},
{
name: "tag_ticket",
description: "Apply tags to the ticket for routing",
input_schema: {
type: "object",
properties: { tags: { type: "array", items: { type: "string" } } },
required: ["tags"]
}
}
].freeze
The agent itself is close to the ReActAgent class from the earlier post, adapted to work on a single ticket instead of a blog topic:
# app/services/ticket_triage/agent.rb
module TicketTriage
class Agent
def initialize(ticket)
@ticket = ticket
@tools = Tools.new(ticket)
@messages = []
end
def run
@messages << {
role: "user",
content: [{ type: "text", text: "Triage this ticket:\n\n#{@ticket.body}" }]
}
loop do
response = complete
break if response.content.all? { |block| block[:type] == :text }
@messages << { role: "assistant", content: response.content }
response.content.each do |block|
next unless block[:type] == :tool_use
result = dispatch(block[:name], block[:input])
@messages << {
role: "user",
content: [{ type: "tool_result", tool_use_id: block[:id], content: result.to_s }]
}
end
end
end
private
def dispatch(tool_name, params)
case tool_name
when "search_docs" then @tools.search_docs(params[:query])
when "draft_reply" then @tools.draft_reply(params[:content])
when "tag_ticket" then @tools.tag_ticket(params[:tags])
else "Unknown tool"
end
end
def complete
client.messages.create(
model: "claude-sonnet-5",
max_tokens: 1024,
system: SYSTEM_PROMPT,
messages: @messages,
tools: FUNCTION_DEFINITIONS
)
end
def client
@client ||= Anthropic::Client.new(api_key: ENV.fetch("ANTHROPIC_API_KEY"))
end
end
end
Wrapping this in a job is, at first, almost anticlimactic:
# app/jobs/ticket_triage_job.rb
class TicketTriageJob < ApplicationJob
queue_as :default
def perform(ticket_id)
ticket = Ticket.find(ticket_id)
TicketTriage::Agent.new(ticket).run
end
end
And the call site, wherever a ticket gets created:
def create
@ticket = Ticket.create!(ticket_params)
TicketTriageJob.perform_later(@ticket.id)
redirect_to @ticket
end
The controller returns right away, the agent does its work on a Sidekiq worker, and the ticket’s draft_reply and tags show up whenever the job finishes. This is the version that works when everything goes right. The next section is about what happens when it doesn’t.
Retries Replay the Whole Loop
Sidekiq’s default behavior on an unhandled exception is to re-enqueue the job and try again later, with a backoff, up to 25 times unless you configure otherwise. This works well for the kind of job Sidekiq was originally built around: something like resizing an uploaded image or writing a row, where the whole method is safe to run again from the top, or already guards itself with a find_or_create_by or a unique index.
TicketTriageJob#perform doesn’t have that property, because Sidekiq has no idea what happened inside the agent’s loop. It only knows whether perform raised or returned. If the database connection drops while tagging the ticket, after the agent has already searched the docs and drafted a reply, the job as a whole raises, and Sidekiq schedules a full retry of perform, not a retry of “just that last step.”
Here’s roughly what that looks like across the two attempts:
Attempt 1:
💬 Thought: Let me search for relevant documentation first.
🔧 Action: search_docs({query: "refund policy digital goods"})
📝 Observation: Refunds are issued within 14 days...
💬 Thought: I have enough context to draft a reply.
🔧 Action: draft_reply({content: "Thanks for reaching out! Per our..."})
📝 Observation: Draft saved.
💬 Thought: This looks like a billing issue, let me tag it.
🔧 Action: tag_ticket({tags: ["billing", "refund"]})
📝 Observation: ActiveRecord::ConnectionTimeoutError: could not obtain a database connection within 5.000 seconds
[job raises, Sidekiq schedules a retry]
Attempt 2:
💬 Thought: Let me search for relevant documentation first.
🔧 Action: search_docs({query: "refund policy digital goods"})
📝 Observation: Refunds are issued within 14 days...
💬 Thought: I have enough context to draft a reply.
🔧 Action: draft_reply({content: "Hi there, thank you for contacting us..."})
📝 Observation: Draft saved.
Two things went wrong here that would be easy to miss in a quick look at the logs. The search happened twice, which is a wasted API call and a few cents, not a disaster on its own. The draft happened twice, and because the model isn’t deterministic between calls, the second draft isn’t the same text as the first, it just quietly replaced it. If draft_reply had gone one step further and actually sent the reply to the customer instead of just saving it, attempt two would have sent a second, different email for the same ticket.
None of this is a Sidekiq bug. Sidekiq is doing exactly what it promises: retrying the job. The mismatch is that “the job” here is a loop with several external, stateful side effects glued together, and retrying it as one unit assumes every step inside is either free to repeat or already protects itself. Neither is true yet in this version.
Making the Agent’s Side Effects Idempotent
The fix isn’t to stop Sidekiq from retrying, it’s to make each side-effecting tool check whether it already did its job before doing it again. search_docs doesn’t need this, it only reads, so repeating it wastes a little time and a little API budget but changes nothing. draft_reply and tag_ticket do need it, because each one writes something that a second, different write would silently clobber or duplicate:
# app/services/ticket_triage/tools.rb
module TicketTriage
class Tools
def initialize(ticket)
@ticket = ticket
end
def search_docs(query)
InternalDocs.search(query).map(&:summary).join("\n\n")
rescue InternalDocs::Error => e
"Search failed: #{e.message}"
end
def draft_reply(content)
return "Already drafted, skipping." if @ticket.draft_reply.present?
@ticket.update!(draft_reply: content, status: "drafted")
"Draft saved."
end
def tag_ticket(tags)
return "Already tagged, skipping." if @ticket.tags.present?
@ticket.update!(tags: tags)
"Tagged with #{tags.join(", ")}."
end
end
end
The guard clause in each method reads the ticket’s own state as the record of what already happened, rather than adding a separate idempotency-key table. This works here because the ticket’s fields double as their own completion markers: a draft_reply isn’t just an input to the next step, it’s proof the step already ran. A job whose side effects don’t leave that kind of trace on an existing record, sending a Slack message with no corresponding column to check, say, would need something more explicit, like a processed_steps array on the ticket, or a Redis key set right before the external call.
The short string each guard returns matters as much as the guard itself. It goes back into the agent’s message history as a normal tool result, so the model reads “Already drafted, skipping.” the same way it would read “Draft saved.”, and moves on to the next step instead of getting stuck or trying the same action again. Replaying attempt two from the earlier trace with these guards in place, the search still runs twice (harmless), the draft short-circuits on the second attempt (no second, different email going out), and the tag, which is what actually failed the first time, gets a real chance to succeed.
Streaming Progress to the User
The agent is now safe to retry, but from the ticket page it’s invisible. Someone looking at the ticket while the job runs sees nothing change for however long the loop takes, then the draft reply and tags appear all at once. That’s a worse experience than the synchronous version was, even though it’s more robust underneath.
Turbo Streams can close that gap without adding a polling endpoint or a websocket layer of our own. The usual Hotwire pattern broadcasts from an ActiveRecord callback, and broadcasting directly from a job used to be the trickier path when we first tried it. What’s different here is that TicketTriageJob runs once and finishes, it isn’t a self-rescheduling poller broadcasting on every tick indefinitely, so calling Turbo::StreamsChannel.broadcast_append_to directly from the agent, the way we do below, stays simple for this bounded sequence of steps. We broadcast a small update at each step of the loop:
# app/services/ticket_triage/agent.rb
module TicketTriage
class Agent
def run
broadcast_step("Starting triage...")
@messages << {
role: "user",
content: [{ type: "text", text: "Triage this ticket:\n\n#{@ticket.body}" }]
}
loop do
response = complete
break if response.content.all? { |block| block[:type] == :text }
@messages << { role: "assistant", content: response.content }
response.content.each do |block|
next unless block[:type] == :tool_use
broadcast_step("Calling #{block[:name]}...")
result = dispatch(block[:name], block[:input])
broadcast_step(result.to_s)
@messages << {
role: "user",
content: [{ type: "tool_result", tool_use_id: block[:id], content: result.to_s }]
}
end
end
broadcast_step("Triage complete.")
end
private
def broadcast_step(text)
Turbo::StreamsChannel.broadcast_append_to(
@ticket,
target: "agent_activity",
partial: "tickets/agent_step",
locals: { text: text }
)
end
# dispatch, complete, and client stay as they were
end
end
The partial being appended is about as plain as a Turbo Stream target gets:
<%# app/views/tickets/_agent_step.html.erb %>
<p class="agent-step"><%= text %></p>
And the ticket’s show page just needs to subscribe to the stream and give the broadcasts somewhere to land:
<%= turbo_stream_from @ticket %>
<div id="agent_activity">
<!-- agent steps get appended here as the job runs -->
</div>
Whoever has the ticket open watches the activity fill in step by step instead of staring at an unchanged page. One thing worth deciding deliberately rather than by default: result.to_s broadcasts whatever the tool returned, including the full text of a drafted reply. That’s fine for an internal support queue, but if this pattern gets reused somewhere a customer might see the stream, broadcast a short status (“Draft ready for review”) instead of the tool’s raw output.
That covers live progress for whoever has the ticket open, which is a different need from knowing what each step cost after the fact. If you also want to track token usage, latency, and dollar cost per call, see Tracking LLM Latency & Cost with Rails Events , which wires that up with the Rails 8.1 Event Reporter.
Conclusion
Getting an agent out of the request cycle and into Sidekiq was the easy part, and it’s also the part that looks finished before it actually is. The TicketTriageJob from the second section runs, and runs correctly, right up until something fails partway through and Sidekiq does exactly what it’s supposed to do: retry the whole thing. Making that safe meant treating the ticket’s own fields as a record of which steps already happened, so a retried loop skips what’s done instead of repeating it. Making it visible meant broadcasting each step over Turbo Streams as it happens, since a job that’s safe to retry silently is still a bad experience if it looks frozen for a minute at a time.
None of this is free complexity. A tool that only reads, or an agent that makes a single classification call and returns, doesn’t need idempotency guards because there’s nothing stateful to duplicate. These patterns earn their keep specifically when an agent’s loop can run more than once, touches something outside the database, and takes long enough that a user is watching it happen, which describes more production agent work than not, but not all of it. It’s also worth confirming an agent is the right feature for the job in the first place, before any of this gets built, and we’ve covered how to prioritize which AI features to build in a Rails app elsewhere. Streaming progress to the user, as covered here, is also a different problem from tracing what an agent did for your own debugging after the fact, we’ve written separately about why LLM applications need that kind of observability too.
Running agents in production and hitting failure modes like these? Let’s talk about it .