GGshow reloaded GGshow reloaded

December 26, 2012

Floating Point Precision

When ASP CInt( ( 0.3 - 0.2 ) * 10 ) gives you 1, but PHP (int) ( ( 0.3 - 0.2 ) * 10 ) gives you 0;

When MySQL SELECT CAST( 3 * ( 1.0 / 3 ) AS SIGNED ) gives you 1, but MSSQL SELECT CAST( 3 * ( 1.0 / 3 ) AS INT) gives you 0;

How accurate the result you can expect from your web application?

floating point

When your web application is built on top of the myth of the floating point, you can’t even expect 0.1 from JavaScript (0.3-0.2), it gives 0.09999999999999998

Filed under: Web — Tags: , , , , , , , , — GG @ 7:30 pm

No Comments »

No comments yet.

RSS feed for comments on this post.

Leave a comment

© 2017 GGSHOW | Powered by WordPress