loading...

A Rabbit Hole of Decimal Formatting

noelworden profile image Noel Worden Updated on ・3 min read

This week I learned a little something about cleaning up decimals for html. The TL;DR of this post is that the Decimal.to_string/2 function has an option :xsd that converts the decimal to canonical representation, which in effect cleans the number of trailing zeros, without rounding.

The situation was that for a new index page we were showing a record's decimal converted to a percentage. Off the cuff, it was simple logic: example_decimal * 100. But, we are pulling in data from excel sheets and at the moment all data is being cast as strings, so we have to use Decimal to convert and perform the multiplication:

Still, relatively simple logic, except for the way Decimal displays the result, based on the amount of non-zeros to the left of the decimal point of the input:

For me that inconsistency was the deal breaker, and the slide down the slippery slope of how to show the user a consistent percentage on the index page. Maybe it's just me, but it seems lazy to show one result with a single zero after the decimal, and then, for no apparent reason to the user, show another result with double zeros after the decimal. Initially, the solution seemed obvious, just slap a round on it so that everything would have double zeros after the decimal.

Done, mic drop, I'm off to get some coffee. I was confident in this solution because I had seen a lot of this data and hadn't ever seen it not be a whole number. This solution also allowed accuracy out to the hundredth position, which I felt would cover our bases. But, before I could even leave my desk to get my celebratory coffee, a colleague raised the now obvious question "what about a number with a long decimal, do we really want to round that?". Although I had seen the data and was confident about the assumption of whole numbers, she was 100% right to bring up the edge case, and if we wanted to handle this decimal right we had to account for a long decimal.

I'll skip passed the long discussion and get right to the good stuff. Someone suggested Decimal.to_string/2 with :xsd passed as an option, and it's a great solution. In the docs it states that passing :xsd will convert the number to the canonical XSD representation. Welp, I had no idea what that meant, so I dug into it further. The docs link to W3 xml decimal documentation and the takeaway from that is "... Leading and trailing zeros are prohibited...". Basically, a canonical representation will remove any unnecessary leading and trailing zeros, effectively cleaning up a decimal.

And to see some more extreme examples, here's how Decimal.mult/2 handles multiple zeros after the decimal:

And the cleanup with :xsd:

And, if the digits to the right aren't zeros:

Note the two unnecessary zeros at the end. Now that same decimal as a canonical representation:

For the purpose of a consistent decimal conversion, canonical representation is the best of both worlds. The decimal has unnecessary zeros dropped, but also displays the decimal as accurately as possible. It is definitely a bit odd to have to convert a string to a decimal, perform the math, and ultimately change it back into a string, but life isn't perfect and this is the data we are working with.


This post is part of an ongoing This Week I Learned series. I welcome any critique, feedback, or suggestions in the comments.

Posted on by:

noelworden profile

Noel Worden

@noelworden

Software Engineer in Boulder, CO - Writing code and getting strategically lost in the mountains

Discussion

pic
Editor guide