Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
Database Servers
DB2InformixIngresMS SQLOraclePervasive.SQLPostgreSQLProgressSybase
Desktop Databases
FileMakerFoxProMS AccessParadox
General
General DB TopicsDatabase Theory
Related Topics
Java Development.NET DevelopmentVB DevelopmentMore Topics ...

Database Forum / FoxPro / Setup / October 2004

Tip: Looking for answers? Try searching our database.

Round()

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Stasys - 12 Oct 2004 13:19 GMT
Such an example:

lnx=1279.17500
lny=1.0000
?ROUND(lnx*lny,2)
The result is 1279.18 - OK

lnx=8279.17500
lny=1.0000
?ROUND(lnx*lny,2)
The result is 8279.17 - not OK ?????
How can you explain such a result?

lnx=8279.17500
lny=1.000
?ROUND(lnx*lny,2)

The result is 8279.18 - OK

Stasys
VFP6, VFP8
Fred Taylor - 12 Oct 2004 16:18 GMT
Such are the vagaries of floating point mathematics on a digital computer.
There is no "exact" with floating point representation.

Signature

Fred
Microsoft Visual FoxPro MVP

> Such an example:
>
[quoted text clipped - 17 lines]
> Stasys
> VFP6, VFP8
Olaf Doschke - 12 Oct 2004 18:02 GMT
> Such are the vagaries of floating point mathematics on a digital computer. There is no "exact" with floating point representation.
yes, and here is why:

? 8279.17500 * 1.000 * 1024
8477875.20000000
? 8279.17500 * 1.0000 * 1024
8477875.199999999

The factor 1024 exactly shifts the value 10 bits.
Now you can see that there is a difference
in the results of  8279.17500 * 1.000 and
8279.17500 * 1.0000, although I cannot
say, how exact numbers are represented
and why a multiplication with 1 has different
results, as 1 can be represented exactly.

But 0.175 has three decimals in base 10 and
binary it's 0.001011.... It's never ending and
can't be exactly represented, so different re-
presentations with a different exactness can be
above or below 0.175 and so rounding can
work or fail...

There are binary coded decimals especially for
monetary calculations. In motorola 68000 assem-
bler there was a command for adding two binary
coded decimals called ABCD (Add Binary
Coded Decimals). In a binary coded decimal
each digit is represented by 4 bits, so a byte
can represent 00 to 99. It's something like hexa-
decimal without letters ;-). With this kind of re-
presentation you can represent decimal numbers
exactly (or you'd have the same problems as
with decimal numbers), so these kind of rounding
errors don't occur with BCDs.

There are two interesting downloads at uni-
versalthread: "decimal COM library" and "Math
Unlimited", which are download ids 9987 and
10044. I think decimal COM library works with
BCDs and Math unlimited works with strings and
can therefore calculate with very large numbers...

Bye, Olaf.
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.