 |
|
List Archives >
Mathcad List Archive >
Archive by date >
This Month By Date >
This Month By Topic
[mathcad] RE: [mathcad]Re: Rounding up/down error
| [mathcad] RE: [mathcad]Re: Rounding up/down error |
|
Author: Chris Whitford
Posted: Tue, 15 Jul 2003 11:19:57 +0100
|
This can lead to unexpected behaviour. Consider if the internal
representation is 0.0442499999... Then to 4 places it will become 0.0442
while for 5 or more places it will become 0.0442500... and Mathcad drops
the trailing zeroes. Try it!
Chris
At 10:50 15/07/2003 +0100, you wrote:
>If it were to be true then any instance of 0.04425 will round incorrectly,
>which is not the case. Just write x:=0.0425 and then x= and format the
>results to 5 digits and you will get exactly x=0.04425.
>
>
>
>52-bit normalised mantissa should be quite enough.
+------------------------------------------------------------------+
+ Chris Whitford University of Leicester +
+ Research Fellow Space Research Centre +
+ Tel: +44 (0)116 252 3496 Physics and Astronomy Department +
+ Fax: +44 (0)116 252 2464 University Road +
+ email: LEICESTER LE1 7RH +
+ http://www.star.le.ac.uk/ UK +
+------------------------------------------------------------------+
---
The Mathcad List - Discussion, Support & News
Contributions: />
Hosted by: Adept Scientific http://www.adeptscience.com
List Archive: http://lists.adeptscience.co.uk/
---
Would you like this to come to a different email address?
---
Simply leave the mailing list (see below) and re-join by
sending a blank email from the new address to:
or
---
If you do not want to be on the Mathcad list, simply send a
blank email (no subject or message needed) to:
/>
|
[View Complete Thread]
Previous by date: [mathcad] RE: [mathcad]Re: Rounding up/down error, Dan Khoushy Ph D
Next by date: [mathcad] RE: [mathcad]Re: Rounding up/down error, Markos E Psarros
Previous thread: [mathcad] [mathcad]Re: Rounding up/down error, Markos E Psarros
Next thread: [mathcad] RE: [mathcad]Re: Rounding up/down error, Sander Lablans
|
|
|
RSS Feeds
and Mathcad Prime 1.0