DEV Community

Kelvin Chu
Kelvin Chu

Posted on

Rounding decimal numbers in Javascript - an interesting problem.

It is not uncommon that we have to deal with decimal numbers in programming - and we often have to round them off for display or other purposes in many circumstances.

In my current work situation, I have encountered one of such problems myself, and here is how I deal with this. Hope this helps people who are faced with similar problems.

The Problem

The problem: I have to round off some decimal numbers to a designated decimal points.

For example, if I have to round off to 3 decimal points, that means:

0.1234 --> 0.123
1.2345 --> 1.235 (note the 5 in the end)
Enter fullscreen mode Exit fullscreen mode

The Search For Solution

Now, as a true javascript developer, the first step I take of course is to google it.

At a first glance, .toPrecision() seems to be the solution, but actually isn't:

Number(0.1234).toPrecision(3) --> 0.123
Number(1.2345).toPrecision(4) --> 1.234
Enter fullscreen mode Exit fullscreen mode

You can see two problems here:
[1] it doesn't round off the answer like I needed, but rather rounding it down by simply removing the extra digits at the end.
[2] I need to know how many significant digits are there in the integer part in order to determine the precision to use.

So I continue the search. Then, I found this post.

The solution is elegant - multiply the number by a certain power of 10 (depending on the decimal points you want), then use Math.round() to round to the nearest integer. In the end, simply divide the number by the same power of 10 to get the correct answer.

A Deeper Look Into This Solution

If you look into its best answer, you might notice something interesting - there is a Number.EPSILON.

This is what I wanted to explain in this post.

First of all, I am not going into full detail into floating point arithmetic. If you really do want to go all-in, here is a post for your reference.

To understand why, let's look into how number is dealt with in Javascript.

Understanding Binary Representation

The simplest of them all is an integer. When doing calculations, it is in its binary format, for example:

13 can be represented as 1101 in binary because

1101 (in binary)
= 1 * 2^3 + 1 * 2^2 + 0 * 2^1 + 1 * 2^0
= 8 + 4 + 1
= 13
Enter fullscreen mode Exit fullscreen mode

How about decimal number? They are stored similarly, but the powers of 2 used are negative powers.

0.875 can be represented as 0.111 in binary because

0.111 (in binary)
= 1 * 2^-1 + 1 * 2^-2 + 1 * 2^-3
= 0.5 + 0.25 + 0.125
= 0.875
Enter fullscreen mode Exit fullscreen mode

Now, you may see a problem with this system. Not all decimal numbers can be precisely represented in the binary format. And this is the reason why we have this weird result:

Alt Text

This is due to the fact that neither 0.1 nor 0.2 can be precisely represented in binary, and their sum is therefore not exact. Javascript tries its best to get the closest answer as it could, and ended up with something very very close to 0.3.

Coming back to the previous observation, why do we need the Number.EPSILON ? This is because of the existence of some edge cases in our previous algorithm. They suggested the problem with the number 1.005 - it being rounded to 1 instead of 1.01 when rounding to 2 decimal points.

Alt Text

The Weird Case

Here, we investigate an even more interesting number to understand why: assume we have to round this number 4.975 to 2 decimal points. Here's the algorithm:

1. First, we have 4.975.
2. We multiply it by 100 to get 497.5
3. We perform Math.round(497.5) to get 498
4. We divide it by 100 to get the answer 4.98
Enter fullscreen mode Exit fullscreen mode

It seems all logical and perfect, right? Javascript says otherwise:

Alt Text

Why so? If you test step-by-step, you will see that:

Alt Text

You can see that 4.975 can not be precisely represented in binary, so Javascript tries to approximate its value but it ended up being under-represented after multiplying by 100.

This is why the original post added Number.EPSILON to the original number - it is so small that it does not really affect the actual value yet it helps the approximation of Javascript to get the correct rounding.

However...

Alt Text

I can now safely say the the stackoverflow answer is WRONG! Haha! Right in your face!

Ok, joking aside, how do we deal with this problem now?

The Real Solution

The ingenious solution can be found here. The rough idea is to make the number an integer before doing any operation. This is because integer can be precisely represented in Javascript. Here's how:

1. Starting with 4.975 again.
2. We multiply 1000 to 4.975 to get 4975, an integer.
3. We now divide it by 10 to get 497.5 for rounding.
4. We perform Math.round(497.5) to get 498.
5. We now divide it by 100 to get 4.98, our final answer.
Enter fullscreen mode Exit fullscreen mode

Does this work? Yes.

Alt Text

Why? This is because in step 2, we convert 4.975 into a precisely represented integer 4975. When it was divided by 10, 497.5 is now precisely represented because its decimal part 0.5 can be precisely represented in binary.

Note that this technique only works on a reasonable range of number. Although integer can be precisely represented to avoid errors, there is still limit on how many digits Javascript can hold for an integer - Number.MAX_SAFE_INTEGER. If after converting to integer your number exceeds this limit, it introduces error into the representation and this technique no longer works. You might want to resort to other means in that case.

That's all for now! Hope you enjoy this post!

TL;DR

Convert the number into integer first before doing any operation or Math.round(). Refer to this post for the algorithm.

Discussion (2)

Collapse
cytim profile image
Tim Wong • Edited on

If you use Lodash, the problem is resolved right away.

_.round(4.975, 2)
// 4.98
Enter fullscreen mode Exit fullscreen mode

The Lodash implementation is also very clean.
github.com/lodash/lodash/blob/mast...

Collapse
scroung720 profile image
scroung720 • Edited on

Thank you for sharing.
Super important topic for anyone doing serious 2D, 3D graphics using javascript. In fact, I have a folder of favorites pointing to resources addressing this problem. This topic appears very often related with subpixel operations in DOM and Canvas manipulations.
en.wikipedia.org/wiki/Subpixel_ren...
johnresig.com/blog/sub-pixel-probl...
Saying That was awesome