Table of Contents:
Problem
A. Drop the query string
B. Fix the query string
Final word
Problem
ActionController::BadRequest
Invalid query parameters: invalid %-encoding (miconazole%202%%20cr)
Rack::QueryParser::InvalidParameterError
invalid %-encoding (miconazole%202%%20cr)
ActionController::BadRequest
ActionView::Template::Error: Invalid query parameters: invalid %-encoding (%2Fsearch%2Fall%Forder%3Ddescending%26page%3D5%26sort%3Dcreated_at)
In a full stack apps, sometimes users manually edit query strings in the address bar, and make requests that have invalid encodings.
In a api apps, sometimes front end or mobile engineer forget to encoding the query strings, so they just give raw query strings to your apps, then they left it for years until it became very hard to fixed on the client side, and the management decide the server side should fix it.
These corrupted query strings are not that important, but users are receiving 500 server error pages.
What can we do?
I can offer two choice: Drop the query string OR Fix the query string.
A. Drop the query string
I got this from this post right here.
First, make a middleware:
# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb
class HandleBadEncodingMiddleware
def initialize(app)
@app = app
end
def call(env)
begin
Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
rescue Rack::Utils::InvalidParameterError
env['QUERY_STRING'] = ''
end
@app.call(env)
end
end
Second, use this middleware before Rack::Runtime:
# config/application.rb
require_relative 'boot'
require 'rails/all'
# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'
Bundler.require(*Rails.groups)
module YourAppsName
class Application < Rails::Application
...
# add this:
config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
...
end
end
The result is, you drop the query string:
# from this
"https://your-url.com?query=alcohol%2070%"
# to this
"https://your-url.com"
B. Fix the query string
I did some research and modify the middleware. This is the step:
First (optional), I use gem rack-utf8_sanitizer to remove invalid UTF-8 characters from the query strings. This prevents errors like "invalid byte sequence in UTF-8".
I personally prefer rack-utf8_sanitizer than utf8-cleaner because it is more robust, more option, and less bugs.
Second, make a middleware:
# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb
class HandleBadEncodingMiddleware
def initialize(app)
@app = app
end
def call(env)
begin
Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
rescue Rack::Utils::InvalidParameterError
params = URI.decode_www_form(env['QUERY_STRING'])
query_string = URI.encode_www_form(params)
env['QUERY_STRING'] = query_string
end
@app.call(env)
end
end
There are many ways to encode and decode url:
URI.escape
URI.unescape
URI.encode_www_form
URI.decode_www_form
URI.encode_www_form_component
URI.decode_www_form_component
CGI.escape
CGI.unescape
I've tried them all, and I picked the best for me, which are URI.encode_www_form and URI.decode_www_form. Probably you have to pick which is best for you.
Third, use this middleware before Rack::Runtime:
# config/application.rb
require_relative 'boot'
require 'rails/all'
# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'
Bundler.require(*Rails.groups)
module YourAppsName
class Application < Rails::Application
...
# you add this for the rack-utf8_sanitizer,
# as described in the gem's readme:
config.middleware.insert 0, Rack::UTF8Sanitizer
# add this:
config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
...
end
end
The result is, you fix the query string:
# from this
"https://your-url.com?query=alcohol%2070%"
# to this
"https://your-url.com?query=alcohol%2070%25"
Final Word
If you have any problems or questions, let me know in the comment.
If you are helped, I am glad I can help you.
Top comments (2)
Thank you for your article. It didn't quite help me, but that's probably my fault.
I'm trying to learn to use Hanami which is quite a project and have got to the point where I am trying to upload images. I'm using the shrine gem and have a very basic problem. When I try to upload a file I get the error which is the subject of your article. Since your article applies to rails I don't quite see how to apply it to a Hanami app. The problem seems to be at the request level before any logic is performed.
If you have any insight into this I would appreciate it. If not, thanks for the article which may eventually help me get past this hurdle.
Joe Radkte
Helpful thanks