|
Post by Walt Decker on Oct 7, 2022 16:48:29 GMT -5
My testing with LB indicates that the problem still exists and has to do with an embedded nul character in the return string. It appears that when LB sends the number string it is truncating the string at the first nul character found, therefore the dll cannot properly translate it and returns garbage. i. e. some number * 10^-44 plus or minus.
I will continue working on it. Perhaps I can come up with a solution.
|
|
|
Post by Walt Decker on Oct 9, 2022 13:28:01 GMT -5
I found the cause for and a solution of the float denormalized problem. The cause is the definition of the struct element. When one defines a struct element as CHAR[n] and then fills that element with a string containing a null character (CHR$(0)) the C language standard is to truncate the string to the first null found. For example: STRUCT tDip, _ North$ AS CHAR[5]
tDip.North$.struct = "anl#G" '<--- where "n" = null character
PRINT tDip.North$.struct '<--- produces "a"
When that is passed to NUMBERMANDLL function FN_GetSingleStr() it sees "annn" (where "n" is a nul character) and does its best to translate it to a single precision value. This results in some number * 10-44 plus or minus 1, a number well beyond single precision scope. GIGO. The solution: define the struct element as CHAR[9] then parse the four bytes returned from FN_SetSingleStr() to their hexidecimal equivalent to produce an eight byte string and store this in the element: FUNCTION FN.SetSingle$(Value)
AnsiChar = 0
ValOut$ = "" A$ = ""
SngOut$ = STR$(Value) + CHR$(0) SngIn$ = SPACE$(16) + CHR$(0)
CALLDLL #NUM, "FN_SetSingleStr", SngOut$ AS PTR, SngIn$ AS STRUCT, _ RetVal AS LONG
SngIn$ = LEFT$(SngIn$, RetVal) FOR I = 1 TO RetVal AnsiChar = ASC(MID$(SngIn$, I, 1)) A$ = DECHEX$(AnsiChar) IF LEN(A$) < 2 THEN A$ = " " + A$ ValOut$ = ValOut$ + A$ NEXT I
FN.SetSingleStr$ = ValOut$ END FUNCTION
I have made some changes in FN.GetSingleStr() to accomodate the eight-byte string, but it is really pointless. If one is going to set the struct to CHAR[9] one might as well use it to store the actual string value instead. A better solution is just to declare the element AS DOUBLE (8-BYTES) and forgo the use of NUMBERMANDLL.
The dll in the zip file HAS NOT been updated.
|
|
|
Post by pierre on Oct 10, 2022 7:10:42 GMT -5
The new function, as it is written, returns nothing. The last line should be:
FN.GetSingle$ = SngIn$ + ValOut$
then it works.
Now, I understand the reasoning, but I don't believe it to be the solution of the whole problem. I know i can forgo the use of the dll, but I precisely want to use it.
Let me explain:
The runtime error 'Float denormalized operand' disappeared completely when I applied the 'USING(...VAL(.....))' correction in the GetSingle$ function. Actually, this replaces a ROUND() function which is missing in Liberty BASIC.
However, even after the last modification, I continue finding randomly "near zero" output values in the range of 1 to 99,999, where it should not be a problem for single precision.
I have some clues, but I need to investigate a little more. As soon as possible I will come up with a short example.
pierre
|
|
|
Post by Rod on Oct 10, 2022 10:06:34 GMT -5
I have not run the code. However if the using() function is fed a number it cannot accommodate in the width requested it prepends a % character. This might mislead the val() function.
|
|
|
Post by Walt Decker on Oct 10, 2022 10:54:19 GMT -5
Pierre, NUMBERMANDLL.DLL in the zip IS NOT UPDATED to accomodate an 8-byte string. Here is my test code. You can see at which value the function fails because of an embedded null character. ' STRUCT tDip, _ '<--- RECORD DEFINITION North$ AS CHAR[5], _'<--- RECORD FIELDS; CHAR[5] INDICATES CSTR East$ AS CHAR[5] 'BY CONVENTION CSTRS ARE NUL-TERMINATED SO USEABLE
DIM NE$(199, 3)
OPEN "NUMBERMANDLL" FOR DLL AS #NUM '<--- NUMBER CONVERSION LIBRARY '85945.2939 33005.1122 RndNum = 0 K = -1 FOR I = 1 TO 20 FOR J = 1 TO 10 K = K + 1 RndNum = 100000 - RND(0) * 100000 tDip.North$.struct = FN.SetSingle$(RndNum, K) PRINT "Row = "; K; " tDip.North$ = "; tDip.North$.struct
NE$(K, 0) = FN.SetSingle$(RndNum, K) 'tDip.North$.struct NE$(K, 1) = STR$(RndNum)
RndNum = 100000 - RND(0) * 100000 tDip.East$.struct = FN.SetSingle$(RndNum, K) PRINT "Row = "; K; " tDip.East$ = "; tDip.East$.struct
NE$(K, 2) = FN.SetSingle$(RndNum, K) 'tDip.East$.struct NE$(K, 3) = STR$(RndNum) NEXT J NEXT I
FOR I = 0 TO K North$ = LEFT$(NE$(I, 0), 4) East$ = LEFT$(NE$(I, 2), 4) PRINT "Row = "; I; " "; North$; " "; NE$(I, 1); " "; East$; " "; NE$(I, 3) tDip.North$.struct = North$ tDip.East$.struct = East$ North$ = FN.GetSingle$(tDip.North$.struct) '<--- comment out East$ = FN.GetSingle$(tDip.East$.struct) '<--- comment out North$ = LEFT$(NE$(I, 0), 4) East$ = LEFT$(NE$(I, 2), 4) North$ = FN.GetSingle$(North$) '<--- GET FLOAT VALUES East$ = FN.GetSingle$(East$) NEXT I CLOSE #NUM END
'-------------------------------------------- '--------------------------------------------
FUNCTION FN.SetSingle$(Value, Row) ' TRANSLATE FLOAT VALUE TO 4-BYTE STRING
SngOut$ = "" SngIn$ = ""
RetVal = 0
SngOut$ = STR$(Value) + CHR$(0) SngIn$ = SPACE$(16) + CHR$(0)
CALLDLL #NUM, "FN_SetSingleStr", SngOut$ AS PTR, SngIn$ AS STRUCT, _ RetVal AS LONG I = INSTR(SngIn$, CHR$(0)) IF I <= 4 THEN PRINT "NULL AT ROW "; Row; " in Posn "; I; " Num = "; LEFT$(SngIn$, 4); " "; Value END IF
FN.SetSingle$ = LEFT$(SngIn$, RetVal) END FUNCTION
'--------------------------------------------------------------------- '---------------------------------------------------------------------
FUNCTION FN.GetSingle$(Bytes4$) ' TRANSLATE 4-BYTE FLOAT STRING TO MULTI-BYTE FLOAT STRING
Bytes4$ = Bytes4$ + CHR$(0) ValIn$ = SPACE$(16) CALLDLL #NUM, "FN_GetSingleStr", Bytes4$ AS PTR, ValIn$ AS STRUCT, NumChrs AS LONG PRINT "VALIN$ = "; ValIn$ FN.GetSingle$ = USING("#####.####", VAL(LEFT$(ValIn$, NumChrs)))'LEFT$(ValIn$, NumChrs) END FUNCTION
'
If you really want to use the NUMBERMANDLL I will update it in the zip. EDIT: after failure, comment these lines in the last FOR loop:
North$ = FN.GetSingle$(tDip.North$.struct) East$ = FN.GetSingle$(tDip.East$.struct)
Run it again and see what happens.
|
|
|
Post by pierre on Oct 11, 2022 4:55:35 GMT -5
I have not run the code. However if the using() function is fed a number it cannot accommodate in the width requested it prepends a % character. This might mislead the val() function. Thank you,Rod. This was inspired by Brandon's recent observations about double precision in structs. As the program works on a range from 1 to 99,999 plus a couple of decimal places there is no danger of exceeding the maximum number of possible displayed characters. But the formula is perhaps not 'water-proof', I agree. As far as I know, there is no ROUND() function in LB. The 'val(using(...))' trick seems to accomplish that, at least to some extent. A simple example: '--------------------------------------------------------------- a = 1.23456789 print "a = ";a print "print using with 4 decimal places : ";using("#.####",a) print "a has not changed : " print "a = ";a print "rounded to 4 decimal places" a = val(using("#.####",a)) print "a has changed : " print "a = ";a print "now displayed with 20 decimal places : " print "a = ";using("#.####################",a) print print "a = 123456789.123456789" a = 123456789.123456789 print "print a : " print a print "exponent:";int(a) print "fractional part : "; a - int(a) print "printed with using : ";using("#########.#########", a) print "rounded to 4 decimal places : " a = val(using("#########.####", a)) print using("#########.####", a) print "now displayed with 9 decimal places : " print using("#########.#########", a) '----------------------------------------------------------------
|
|
|
Post by pierre on Oct 11, 2022 5:11:56 GMT -5
Walt:
<< NUMBERMANDLL.DLL in the zip IS NOT UPDATED to accomodate an 8-byte string >>
Yes, you are right. I had seen your observation, but I forgot about it.
Now, if we forgo the use of the dll, coming back to a string representation of the numbers, one could say : why not stay simply with the standard LB random access files ?
My interest in the dll is not specially for space-saving, it is above all for the fact that the numbers are crypted and cannot be read with a simple notepad program. That can be important for databases with sensitive numbers, such as accountancy, personal finance and so on. In that case we don't have random numbers, but 'real-life' numbers, usually with 2 decimal places. Then, the numbers we store in the database MUST come back with exactly the same precision. We cannot allow us a single failure....
From this perspective, I would say yes I am interested in using the dll. So if you can update it without much effort, I would be very obliged.
As far as the testing is concerned I did not limit myself to a simple random approach, I tested the whole range from 1 to 99,999 for the exponents, with a couple of assumptions for the fractional parts. I don't know much about IEEE-754 so my findings and - very cautious - conclusions may be completely 'out of the box'. But strange things happen.... and all those things have something to do with a power of the number 2...... Thus, conversion of decimal to binary values and vice versa.
Is it due to LB or to the dll, I absolutely don't know and I am unable to point to one or another.
However, if you are willing to update the dll, I'd better wait and perform my testing again. That will impeed me from publishing nonsense.
pierre
|
|
|
Post by Walt Decker on Oct 11, 2022 13:37:59 GMT -5
Pierre, I updated NUMBERMANDLL.DLL in the zip file. If you want to encrpt a number you can do so by getting the ASCI code of the each digit, transforming it to its hex value, then offset that by a certain amount. If you plan to use the dll for currency you might consider using one of these functions: "FN_SetDouble", tDbl AS STRUCT, ValInStr$ AS STRUCT, NumChrs AS LONG "FN_GetDouble", ValOutStr$ AS PTR, ValInStr$ AS STRUCT, NumChrs AS LONG "FN_SetCurVal", tDbl AS STRUCT, CurOutStr$ AS STRUCT, NumChrs AS LONG "FN_GetCurVal", CurValStr$ AS PTR, CurInStr$ AS STRUCT, NumChrs AS LONG "FN_SetCurValEX", tDbl AS STRUCT, CurInStr$ AS STRUCT, NumChrs AS LONG "FN_GetCurValEX", CurValStr$ AS PTR, CurInStr AS STRUCT, NumChrs AS LONG Each of the above return an 8-byte string. Be aware that the string MAY HAVE an embedded null (CHR$(0)) character. "FN_SetDouble" has a precision from +4.19*10^-307 to -1.79*10^308 "FN_SetCurVal" has a precision from -9.22*10^14 to +9.22*10^14 with 4-digit decimal "FN_GetCurValEX" has a precision from -9.22*10^16 to +9.22*10^16 with 2-digit decimal tDbl PROTOTYPE: STRUCT tDbl, _ Dbl AS DOUBLE
In each use the Dbl element of the structure is sized using the appropriate format string, e. g. VAL(USING("#######.####", Value)) A function to use one of the above: FUNCTION FN.SetDouble(Value)
STRUCT tDbl Dbl AS DOUBLE
OutStr$ = "" A$ = ""
NumChrs = 0 I = 0 J = 0
tDbl.Dbl.struct = VAL(USING("#######.###", Value))
ValInStr$ = SPACE$(10) + CHR$(0)
CALLDLL #NUM, "FN_SetDouble", tDbl AS STRUCT, ValInStr$ AS STRUCT, NumChrs AS LONG
'<=== translate ascii values to hex to produce a 16-byte string ====> ValInStr$ = LEFT$(ValInStr$, NumChrs) FOR I = 1 TO NumChrs J = ASC(MID$(ValInStr$, I, 1) A$ = DECHEX$(J) IF LEN(A$) < 2 THEN A$ = " " + A$ OutStr$ = OutStr$ + A$ NEXT I
FN.SetDouble = OutStr$ END FUNCTION
You can obtain a reasonable format string for the above by using the functions I recently placed in the "Libery BASIC code" thread topic "Share Your Snipped II". The file output structure might look something like this: STRUCT tOut, _ CurVal AS CHAR[17] <--- large enought to contain 16 characters
The reason I posted the original is to demonstrate an alternative method for random access files. If one wants to use native LB random access, go ahead. You can also use the methods I used in the PVS application or those I used in the topic "BINARY RA files" or you can just output the data as a text file. Your choice.
EDIT:
There is a round function in NUMBERMANDLL: "FN_RoundNum", DecNum$ AS STRUCT, NumDec AS LONG, RetVal AS LONG
DecNum$ = string representation of a decimal value; INPUT AND RETURN
NumDec = Number of decimal places to round
On return DecNum$ contains the rounded value
|
|
|
Post by pierre on Oct 11, 2022 14:38:27 GMT -5
Thank you very much Walt !
I will replace the dll and perform a new test. If I find the same strange behaviour, I will certainly post a short example. I will also try your test code for values from 1.xxx to 99,999.xxx and see what happens.
Many thanks also for your detailed explanations.I will study them the best I can.
I was not aware of the ROUND() function in the dll, it will come in handy.
pierre
|
|
|
Post by pierre on Oct 12, 2022 14:05:43 GMT -5
Walt, I am facing a huge problem. I tested the opdated dll with a range of values from 1 to, for the moment, 65536 for the exponent and some assumptions .for the fractional part. At first sight, it is all OK. Ihe program runs without errors, no more 'nearby zero' values, that is true, but there are many, many retrieved values that do not match their original counterpart. IMHO this can only be due to a conversion problem.... So, unfortunately it seems to point at the dll......Could there be a flaw in the conversion code? Is it because of the translation in Hexadecimal? I don't know. I compared the INPUT and OUTPUT values and determined the differences. The results are printed in the file '54 test.txt'. There are errors in the exponent, in the fractional part or in both. One can detect a certain logic in the different patterns. They often change when a power of the number 2 is reached: 2, 8, 16, 32, 64, 128, 256, and so on and so forth. I chose a simple fractional part: 0.54 in order to show the differences in a clear and readable manner. Other fractional numbers may have even more errors because of rounding problems. One can take values at random in the text file and input them manually for conversion, instead of using a random generation. It would be great if you could have a quick look at it, for 'at the doorstep' of the dll, I am totally unable to find a solution. Thanks in advance. pierre 54 test.txt (265.54 KB)
|
|
|
Post by Walt Decker on Oct 12, 2022 16:39:05 GMT -5
Pierre, let me see your test code.
|
|
|
Post by pierre on Oct 13, 2022 4:04:08 GMT -5
Attached test file 'random access TEST.bas' data file used : 'RndFileTEST.RND' dll used : 'NUMBERMANDLL.DLL' Oct 11,2022 21.50 Kb FN.SetSingle$ adapted to 8 Bytes strings Attachments:random access TEST.bas (8.59 KB)
|
|
|
Post by Walt Decker on Oct 14, 2022 14:07:20 GMT -5
Pierre, I have made a few improvements in the dll for tracing potential bugs so you might want to download it again. In your test code do this: This will reduce the file size and make it a bit faster ' STRUCT tDip, _ North$ AS CHAR[9], _ East$ AS CHAR[9], _ CRLF AS CHAR[3] '
' fract = 0.54 '<--- no need to define this more than once
FOR I = 1 TO 65536 '<--- WRITE THE FILE SEQUENTIALLY 'first number must be 1 !!! RndNum = I + fract tDip.North$.struct = FN.SetSingle$(RndNum) tDip.East$.struct = STR$(RndNum) '<--- store the real value '
Compare strored values
' for I = 0 to NumRecs - 1 '<--- compare stored values RetVal = FN.GetRecord(FlHndl, I, StructSize) '<--- READ THE RECORD HexN$ = tDip.North$.struct East$ = tDip.East$.struct North$ = FN.GetSingle$(HexN$) Nval = VAL(North$) Eval = VAL(East$) Diff = Eval - Nval IF Diff <> 0.0 THEN PRINT "RECORD = "; I; " HEXN$ = "; HexN$; " VALUE = "; Nval; " EAST$ = "; HexE$; PRINT " VALUE = "; Eval; " DIFF = "; Diff
z = z + 1 END IF
Next I print " # errors : ";z
BytesWritten = FN.CloseFile(FlHndl) '<--- RELEASE THE FILE
CLOSE #KERN CLOSE #NUM END [div]'
[/div]
' FUNCTION FN.SetSingle$(Value) '############################################[/div][div]' SetSingle() is the same as FN_SetSingleStr() but with special tracing code[/div][div]'############################################[/div][div] [/div][div]AnsiChar = 0
ValOut$ = "" A$ = ""
SngOut$ = STR$(Value) + CHR$(0) SngIn$ = SPACE$(16) + CHR$(0)
CALLDLL #NUM, "SetSingle", SngOut$ AS PTR, SngIn$ AS STRUCT, _ RetVal AS LONG
SngIn$ = LEFT$(SngIn$, RetVal)
FOR I = 1 TO RetVal AnsiChar = ASC(MID$(SngIn$, I, 1)) A$ = DECHEX$(AnsiChar) IF LEN(A$) < 2 THEN A$ = "0" + A$ ValOut$ = ValOut$ + A$ NEXT I
FN.SetSingle$ = ValOut$
END FUNCTION
'--------------------------------------------------------------------- '---------------------------------------------------------------------
FUNCTION FN.GetSingle$(Bytes4$) ' TRANSLATE 4-BYTE FLOAT STRING TO MULTI-BYTE FLOAT STRING '############################################[/div][div]' GetSingle() is the same as FN_GetSingleStr() with special tracing code[/div][div]'############################################[/div][div] ValIn$ = "" NumChrs = 0
Bytes4$ = Bytes4$ + CHR$(0) ValIn$ = SPACE$(16) + CHR$(0) CALLDLL #NUM, "GetSingle", Bytes4$ AS PTR, ValIn$ AS STRUCT, _ NumChrs AS LONG ValIn$ = LEFT$(ValIn$, NumChrs) FN.GetSingle$ = USING("######.####", VAL(ValIn$)) 'FN.GetSingle$ = LEFT$(ValIn$, NumChrs)
END FUNCTION '
I think it will run without error.
EDIT: You might want the change the size of tDip.East$ to CHAR[13] to account for negative numbers.
|
|
|
Post by pierre on Oct 15, 2022 5:52:35 GMT -5
Walt,
Thank you very much for taking the time to solve the problem and simplify my test code.
I now understand why it didn't work:
It is in the FN.SetSingle$ function in the middle of the FOR -- NEXT loop that sets up the 8 Byte Hexadecimal string.
When you first published it on Oct 11, it read:
IF LEN(A$) < 2 THEN A$ = " " + A$
Actually, this formula sent the wrong 8 Byte string to the file...but not being an expert, I didn't understand it.
You solved the problem by changing this in your last version:
IF LEN(A$) < 2 THEN A$ = "0" + A$ '<----------
Now it works !
So, thanks again, and have a nice weekend !
pierre
|
|
|
Post by Walt Decker on Oct 15, 2022 9:11:06 GMT -5
It also took me a bit to find the problem. The hex conversion in the dll looks for a two digit hex number hex number not a space-diget number, i. e. " A" is not the same as "0A". When I wrote the first LB version I was asleep at the switch.
If you have more problems, please let me know.
Walt
|
|