List Archives >
Mathcad List Archive >
Archive by date >
This Month By Date >
This Month By Topic
[mathcad] Re: 6 decimal place precision
| [mathcad] Re: 6 decimal place precision |
|
Author: Richard Jackson
Posted: Mon, 28 Jul 2003 10:18:46 -0400
|
What you really need it significant digits, not decimal places.
83333333333333.33333333 to 2 decimal places is
83333333333333.33, whereas to 6 significant digits it is
83333300000000.0
The attached file contains a couple of functions to round to a specified
number of significant digits. I never wrote the one to always round
down, but it shouldn't be hard to adapt the existing ones if this is
needed.
Richard
On 27 Jul 2003 at 19:05, Bill Dumke wrote:
> Sven,
>
> I tried your method and thought it might work if I truncated the result
> of every piece of the whole function. But I found it will not work for
> floating point complex numbers. Any ideas? See attached file.
>
> Bill
>
|
|
Attachments:
Significant Digits 2000.mcd
|
[View Complete Thread]
Previous by date: [mathcad] Re: Multiprocessor system, Colin G Clarke
Next by date: [mathcad] Re: Multiprocessor system, Quang-Viet Nguyen
Previous thread: [mathcad] Multiprocessor system, Zdenek Sedlacek
Next thread: [mathcad] Re: Multiprocessor system, Colin G Clarke
|
RSS Feeds
and Mathcad Prime 1.0