DEV Community

Caleb Zhao
Caleb Zhao

Posted on

Incorrect calculations by the computer: cos(x) near x=k*π+π/2

In this blog post, we explore incorrect calculations of the cosine function cos(x) in MATLAB near numbers around kπ+π2. k\,\pi+\frac{\pi}{2}.

Example 1. Given that x1=0.392699082e212π+π2 x_1=0.392699082\textup{e}2\approx 12\,\pi+\frac{\pi}{2} and
x2=0.1115265392024e335π+π2,x_2=0.1115265392024\textup{e}3\approx 35\,\pi+\frac{\pi}{2}\,, calculate cos(x1) \cos(x_1) and cos(x2) \cos(x_2) in MATLAB.

Let's just post the figure directly.

MATLAB outputFrom the above screenshot, it can be seen that the two outputs from MATLAB are
3.012758778139255e08 -3.012758\red{778139255}e-08 and 3.765778342090408e11. -3.765\red{778342090408}e-11.

However, the correct values with 16 significant digits are -0.3012758451921695e-7 and -0.3765996542384011e-10, respectively (as provided by ISRealsoft).

Thus, the outputs of MATLAB contain 9 and 12 incorrect digits, with error rates of 9/16 = 56.25% and 12/16 = 75% respectively for the significant figures.

Top comments (0)