Fix Sneaky ArgumentErrors When Upgrading Ruby
Upgrading from Ruby 2 to Ruby 3 can be a challenging task, especially when your
Rails application relies on ActiveJob with Sidekiq. In the process, you may encounter
cryptic ArgumentError exceptions that make the upgrade seem daunting. By following along
you’ll be well-equipped to avoid some of the hurdles that come your way.
TLDR;
- Fix
unknown keywordby making sure you correctly separate positional and keyword args. - Always pass hashes with only string or only symbolic keys, avoid mixing string and symbolic keys in hash arguments.
Issue: unknown keyword: :id
We recently undertook the task of upgrading a Rails application that used ActiveJob with Sidekiq as the backend adapter. Our goal was to transition the application from Ruby 2 to Ruby 3.
We are aware that Ruby 3 introduces a requirement to explicitly separate positional arguments from keyword arguments. This stands in contrast to Ruby 2, where we had the flexibility to pass a hash as the last argument, and Ruby would interpret it as keyword arguments. Read more about this change .
Ruby 2 interprets the last argument hash as keyword arguments.
Ruby 3 interprets the last argument hash as a positional argument.
This change in Ruby 3 does not require that all last argument hashes be changed into keyword arguments. It is still valid to pass a hash as a positional argument. However, this flexibility can pose a challenge when identifying which arguments need to be updated.
Maintaining comprehensive test coverage with high-quality tests proves invaluable in finding the arguments that require updating. Additionally, utilizing a bug tracking system that promptly alerts you to any mismatches between positional and keyword arguments is crucial.
The application we worked on had notably low test coverage. To compensate, we conducted thorough keyword searches across the application‘s files, targeting potentially problematic areas that may not have been captured by our tests.
To mitigate the risks associated with the low test coverage, we implemented several measures.
One such measure was proactively updating keyword arguments in Ruby 2. We introduced the (**)
double splat operator to clearly separate keyword arguments, ensuring a smoother transition
when deploying in Ruby 3.
During our QA process, while still on Ruby 2.7.6, we encountered an exception that was captured by our bug tracking system. The exception message provided valuable information, revealing the following details:
ArgumentError
unknown keyword: :id
We followed the stack trace to pinpoint the exact location where the exception was raised. This enabled us to identify the specific section of the code. A simplified example of the code snippet is as follows:
# inside a sidekiq job file
def perform(params)
args = ["some-value", params]
Service.run(*args)
end
# inside service.rb
def self.run(*args, **kwargs)
new.run(*args, **kwargs)
end
def run(uuid, params)
# do work with the uuid and params
end
Notice that we have a very simple Sidekiq job that calls Service‘ class-level run method. The class method
accepts any positional or keyword arguments and delegates them to the instance-level run method.
The error message appeared cryptic since we were not calling run with an :id keyword argument. In
the example above, the run method expects two positional arguments named uuid and params.
The params hash included an attribute named id. This led us to question: “Why would a key
within a hash cause this error?” This was particularly intriguing as we were passing the hash
as a positional argument rather than a keyword argument.
Serialize/Deserialize: ActiveJob vs Sidekiq
To unravel this issue, it is beneficial to gain a deeper understanding of the internal mechanics of ActiveJob and Sidekiq.
Sidekiq serializes job arguments into JSON and then deserializes the JSON string when the job is ready for execution. One notable aspect is that during the serialization/deserialization process, symbolic hash keys are lost.
x = { id: "asdf"}
y = JSON.dump(x) # y == "{ 'id' : 'asdf' }"
JSON.parse(y) # {"id"=>"asdf"}
ActiveJob (AJ) performs more advanced serialization and deserialization. One of the key differences is that AJ can serialize a hash with a mix of string and symbol keys without losing track of the different key types. This means that when we deserialize to execute the job, we still have these symbol and string keys in our hash.
If AJ serialized { id: "asdf" } and you then looked at the serialized data you would see
this:
"{ 'id': 'asdf', '_aj_symbol_keys' => ['id'] }"
To keep track of keys that should be deserialized as symbolic keys, ActiveJob introduced the
_aj_symbol_keys attribute.
Hashes with string and symbolic keys
Now, if we revisit the previously mentioned code snippet, provided below, we would be wise
to examine the params hash more closely. Upon investigation, we discovered that the params
hash did indeed include a symbolic key. As you might have guessed, the symbolic key in question
was :id.
# inside sidekiq job file
# params == { "message" => "asdf", :id => 1}
def perform(params)
args = ["some-value", params]
Service.run(*args)
end
Since we were passing a hash with a combination of string and symbol keys, Ruby 2 interpreted
the symbolic keys as keyword arguments.
You can verify this behavior by following these steps:
- ensure you have Ruby 2.7.6 installed,
- then open an
irbsession, - define 3 methods by executing the following in
irb.
def test(id, params = {}); pp({ id: id, param: params}); end
def args_and_kwargs(*args, **kwargs); test(*args, **kwargs); end
def args_only(*args); test(*args); end
Now, let’s define an argument that includes a hash with only string keys. Once we have the argument defined, we can confidently call the previously defined methods without encountering any issues.
args = ["asdf", {"message"=>"asdf", "id"=>"asdf"}]
args_only(*args)
args_and_kwargs(*args)
# => {:id=>"asdf", :param=>{"message"=>"asdf", "id"=>"asdf"}}
The code above runs, as we anticipated, without raising any exceptions. However, let’s observe what happens when we update one of the keys to be a symbolic key.
["asdf", {"message"=>"asdf", :id=>"asdf"}]
args_only(*args)
args_and_kwargs(*args)
# => in `test': unknown keyword: :id (ArgumentError)
Now we see the same error caught by our bug tracker.
The Solution
By manually updating the hash construction, you can ensure that all keys are strings from
the beginning. Alternatively, you can use the stringify_keys method, which is available
in Rails and can be called on the hash to convert any symbol keys to string keys.
In this case the solution was to make sure that we pass only string keyed hashes. You can
manually update how the hash is constructed or you can call stringify_keys
on the hash before you pass it around.
What did we learn?
Delegating arguments in Ruby can be done in various ways, but it’s important to note that not all methods are compatible between versions 3 and 2. Ruby 3 requires explicit separation of positional and keyword arguments, which may impact certain argument delegation techniques used in Ruby 2.
When upgrading from Ruby 2 to Ruby 3, it is essential to review and update argument delegation methods to ensure compatibility.
Next, it is advisable to maintain consistency in the types of hash keys whenever possible. This simple practice can save you hours of debugging code, as you may encounter situations where the symbol part of a hash inadvertently becomes a keyword argument when used in a function call.
Background jobs can become more complex when using keyword arguments. In particular, Sidekiq 7 provides options to warn or raise exceptions when passing keyword arguments or symbols to a job.
It is recommended to follow best practices by keeping job parameters small and simple.
Thank you for reading this post! I hope you found the information helpful in your Ruby or Rails upgrade project. If you have any upgrade projects and need assistance or advice, feel free to reach out . We are here to help and provide guidance to make your upgrade process pain free.
Stay tuned for more helpful content, and don’t hesitate to ask if you have any further questions or requests.
Further reading