Without getting too deep into the weeds, when storing and inserting these large CSV files, we are kind of meta programming the casting and validation aspect of the process, ultimately storing an individual row's errors in a source_file_errors table. Currently we were only storing the message from changeset. Up to this point this was working fine, because the default Ecto errors we were displaying were things like "is not valid", and almost all of the more complex validations were custom, and therefore provided custom error messages.
But now we are starting to utilize more of the available Ecto validations, like validate_number. When using validate_number and something like less_than_or_equal_to, Ecto produces a default message of "must be less than or equal to %{number}". In this message %{number} is intended to be interpolated to the actual number found in the field.
Normally, when dealing with Ecto out of the box, error messages are run through an error_helpers.ex module before being displayed in views. The helper takes in the message and opts from the changeset, determines if any interpolation needs to be performed, and outputs a message.
But, because we are meta programming the error handling for this particular schema, a bit more manual work is needed before we can take advantage of the helper module. We need to store the opts field of the changeset in the source_file_errors table so that we can pass it, along with the stored message, to error_helpers.ex.
Sounded easy enough, spin up a migration to add an opts field to the already existing source_file_errors table. But the catch was what type to cast the new field as? The changeset.opt field is built out like this:
It is a keyword list. Taking a look at the Ecto Primitive Types documentation, there is an option for storing a casting a field as a list there's no explicit option for storing a keyword list. But, a keyword list can be converted to a map with Map.new/0.
So, what needs to happen, is to take the keyword list from changeset.opts:
Convert it to a map, with Map.new/0 and store it in the database:
Once that gets stored in the database, the syntax shifts a bit, it's shaped like this:
And when it gets called from the database, the syntax shifts yet again, and is shaped like this:
Now, the opts field is being stored, but I found out it's not clear sailing after that. The error_helpers.ex has a translate_error function, which takes in message and opts as arguments, and uses Gettext.dgettext to output either a simple or interpolated message, whichever is necessary. What I realized is that the function is expecting the opts field to be in a specific shape, you guessed it, a keyword list with keys as atoms. One might assume that a keyword list that has been converted to a map can just be converted back to a keyword list in the same shape it originally was. Well, this is what happens when using the Map.to_list/1:
And, if you try to run a keyword list of that shape through the translate_error function, it doesn't recognize the number key because it's not an atom, and therefore it outputs the uninterpolated message:
So, that means the map needs to be converted in a way not covered by the built-in functions. I did a little digging, and StackOverflow offered up this little gem:
That converts the map to a keyword list, with the keys as atoms:
And, now that opts[:number] is an atom, it will be recognized by Gettext.dgettext/4, and the message will be successfully interpolated:
Now that all the pieces are there, a custom function can be added to a error_helpers.ex module to gather the data, convert the list, and output the error message utilizing the existing functions in that module.
Although it can be seen as less than ideal, I always find it interesting when forced to step outside of a language's scaffold. One can become a bit complacent when slapping that <%= error_tag => helper around forms and magically seeing the appropriate error message appear. Although it seemed a bit daunting at first, I'm happy I was able to take this deep dive into the guts of the error message handling.
This post is part of an ongoing This Week I Learned series. I welcome any critique, feedback, or suggestions in the comments.
Top comments (0)