 |
|
List Archives >
Mathcad List Archive >
Archive by date >
This Month By Date >
This Month By Topic
[mathcad] Re: 6 decimal place precision addendum
| [mathcad] Re: 6 decimal place precision addendum |
|
Author: Jean Rosenfeld
Posted: Mon, 28 Jul 2003 19:42:23 +0100
|
Further to my previous note with attachment to truncate to 6 decimal places.
You might consider rounding to six instead. This depends on how the system
you are trying to simulate handles numbers less than 10^-6.
Also, for floating numbers less than 1, if the system you are trying to
simulate always has numbers in the form 1.123456 x10^n, you should extend as
shown under dround in the attached. There is probably a more elegant way of
programming that (I had to set the trap for x = 0 or y = 0).
I just want to make the point that it really depends on how the system you
are simulating works:
Fixed decimal point numbers to 6 decimal places, 1.123456 x 10^-7 =
0.0000001123456 carries through as 0
Floating point decimal number to 6 decimal places 1.123456 x 10^-7 is
preserved, etc.
Dtrunc would be analogous to dround, but I did not do it.
I hope this carries things a little further..it needs checking out more
thoroughly for other operands.
--------------------------------------------------------
Jean L.J. Rosenfeld
Argoed Hall
Bryn-y-Baal
Mold
Flintshire CH7 6SQ
Wales, UK
Tel +44 1352 750566
E-mail />
-----Original Message-----
From: Bill Dumke
Sent: 28 July 2003 01:05
To: Mathcad Discussion List
Subject: [mathcad] Re: 6 decimal place precision
Sven,
I tried your method and thought it might work if I truncated the result of
every piece of the whole function. But I found it will not work for
floating point complex numbers. Any ideas? See attached file.
Bill
|
|
Attachments:
ctrunc or round.mcd
|
[View Complete Thread]
Previous by date: [mathcad] RE: Minimizing a Function, Gray, Steven G
Next by date: [mathcad] Re: 6 decimal place precision, Sven Lindhardt
Previous thread: [mathcad] Minimizing a Function, Chris Wilkinson
Next thread: [mathcad] Re: 6 decimal place precision, Sven Lindhardt
|
|
|
RSS Feeds
and Mathcad Prime 1.0