|
Post by tsh73 on Apr 2, 2022 14:08:05 GMT -5
problem #1 big floats print as loooong integers (with invented digits after 16 significant ones Double actually supposed to have)
print 1e309 produces error window with text
- nice. But
print 1e308 gives
100000000000000001097906362944045541740492309677311846336810682903157585404911491537163328978494688899061249669721172515611590283743140088328307009198146046031271664502933027185697489699588559043338384466165001178426897626212945177628091195786707458122783970171784415105291802893207873272974885715430223118336 If it is Double, shouldn't it show only 16 digits (or less) that really there? Probably in exponential form?
problem #2 adding small floats to big ints give weird results
p1=100000000000000000000000 q=p1+0.001 print q p0=99999999999999991611392 'q prints like this
print "p0-q",p0-q 'actually gives 0 print "p1-q",p1-q '8388608
so adding .001 get number smaller, and difference is 8388608
And it really looks like result q is indeed integer - then I subtract integer it printed as (p0), I have 0 difference
LB 4.x obvouulsy stores q as double, and math with double makes all double too producing difference 0.0
|
|
|
Post by Carl Gundel on Apr 2, 2022 15:46:13 GMT -5
problem #1big floats print as loooong integers (with invented digits after 16 significant ones Double actually supposed to have) print 1e309 produces error window with text - nice. But print 1e308 gives 100000000000000001097906362944045541740492309677311846336810682903157585404911491537163328978494688899061249669721172515611590283743140088328307009198146046031271664502933027185697489699588559043338384466165001178426897626212945177628091195786707458122783970171784415105291802893207873272974885715430223118336 If it is Double, shouldn't it show only 16 digits (or less) that really there? Probably in exponential form? problem #2adding small floats to big ints give weird results p1=100000000000000000000000 q=p1+0.001 print q p0=99999999999999991611392 'q prints like this
print "p0-q",p0-q 'actually gives 0 print "p1-q",p1-q '8388608
so adding .001 get number smaller, and difference is 8388608 And it really looks like result q is indeed integer - then I subtract integer it printed as (p0), I have 0 difference LB 4.x obvouulsy stores q as double, and math with double makes all double too producing difference 0.0 Hmm. Don't do that. Thanks for the bug report. You're the right kind of torture chamber to be an alpha tester.
|
|
|
Post by Carl Gundel on Apr 5, 2022 19:22:06 GMT -5
problem #1big floats print as loooong integers (with invented digits after 16 significant ones Double actually supposed to have) so adding .001 get number smaller, and difference is 8388608 And it really looks like result q is indeed integer - then I subtract integer it printed as (p0), I have 0 difference LB 4.x obvouulsy stores q as double, and math with double makes all double too producing difference 0.0 Hmm. Don't do that. Thanks for the bug report. You're the right kind of torture chamber to be an alpha tester. Okay, I've got this worked out. Fixed for build 354.
|
|