DEV Community

loading...

Rails: Fixing ‘Invalid query parameters: invalid %-encoding’

K Putra
Fullstack developer. Code from 2018. Rubyist. Sorry for bad english.
Updated on ・3 min read

Table of Contents:
Problem
A. Drop the query string
B. Fix the query string
Final word

Problem

ActionController::BadRequest
Invalid query parameters: invalid %-encoding (miconazole%202%%20cr)

Rack::QueryParser::InvalidParameterError
invalid %-encoding (miconazole%202%%20cr)

ActionController::BadRequest
ActionView::Template::Error: Invalid query parameters: invalid %-encoding (%2Fsearch%2Fall%Forder%3Ddescending%26page%3D5%26sort%3Dcreated_at)
Enter fullscreen mode Exit fullscreen mode

In a full stack apps, sometimes users manually edit query strings in the address bar, and make requests that have invalid encodings.

In a api apps, sometimes front end or mobile engineer forget to encoding the query strings, so they just give raw query strings to your apps, then they left it for years until it became very hard to fixed on the client side, and the management decide the server side should fix it.

These corrupted query strings are not that important, but users are receiving 500 server error pages.

What can we do?

I can offer two choice: Drop the query string OR Fix the query string.

A. Drop the query string

I got this from this post right here.

First, make a middleware:

# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb

class HandleBadEncodingMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    begin
      Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
    rescue Rack::Utils::InvalidParameterError
      env['QUERY_STRING'] = ''
    end

    @app.call(env)
  end
end
Enter fullscreen mode Exit fullscreen mode

Second, use this middleware before Rack::Runtime:

# config/application.rb

require_relative 'boot'

require 'rails/all'

# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'

Bundler.require(*Rails.groups)

module YourAppsName
  class Application < Rails::Application

    ...
    # add this:
    config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
    ...

  end
end
Enter fullscreen mode Exit fullscreen mode

The result is, you drop the query string:

# from this
"https://your-url.com?query=alcohol%2070%"

# to this
"https://your-url.com"
Enter fullscreen mode Exit fullscreen mode

B. Fix the query string

I did some research and modify the middleware. This is the step:

First (optional), I use gem rack-utf8_sanitizer to remove invalid UTF-8 characters from the query strings. This prevents errors like "invalid byte sequence in UTF-8".

I personally prefer rack-utf8_sanitizer than utf8-cleaner because it is more robust, more option, and less bugs.

Second, make a middleware:

# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb

class HandleBadEncodingMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    begin
      Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
    rescue Rack::Utils::InvalidParameterError
      params = URI.decode_www_form(env['QUERY_STRING'])
      query_string = URI.encode_www_form(params)
      env['QUERY_STRING'] = query_string
    end

    @app.call(env)
  end
end
Enter fullscreen mode Exit fullscreen mode

There are many ways to encode and decode url:

URI.escape
URI.unescape
URI.encode_www_form
URI.decode_www_form
URI.encode_www_form_component
URI.decode_www_form_component
CGI.escape
CGI.unescape
Enter fullscreen mode Exit fullscreen mode

I've tried them all, and I picked the best for me, which are URI.encode_www_form and URI.decode_www_form. Probably you have to pick which is best for you.

Third, use this middleware before Rack::Runtime:

# config/application.rb


require_relative 'boot'

require 'rails/all'

# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'

Bundler.require(*Rails.groups)

module YourAppsName
  class Application < Rails::Application

    ...
    # you add this for the rack-utf8_sanitizer,
    # as described in the gem's readme:
    config.middleware.insert 0, Rack::UTF8Sanitizer
    # add this:
    config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
    ...

  end
end
Enter fullscreen mode Exit fullscreen mode

The result is, you fix the query string:

# from this
"https://your-url.com?query=alcohol%2070%"

# to this
"https://your-url.com?query=alcohol%2070%25"
Enter fullscreen mode Exit fullscreen mode

Final Word

If you have any problems or questions, let me know in the comment.

If you are helped, I am glad I can help you.

Discussion (0)