System Primitive Failed

[dodgycoder]

Hi All

I am working with large arrays and I suspect this is a BASIC or memory limitation.

When I define ten two-dimensional array of (1000000,5) dimensions at try to execute the code, the "System Primitive Failed" error comes up.

With one smaller array of (1000000,5) the error does not occur.


Any ideas?

Cheers 

[Carl Gundel]

May 5, 2022 3:47:30 GMT -5 dodgycoder said:

I am working with large arrays and I suspect this is a BASIC or memory limitation.

When I define ten two-dimensional array of (1000000,5) dimensions at try to execute the code, the "System Primitive Failed" error comes up.

With one smaller array of (1000000,5) the error does not occur.

Any ideas?

I have not trouble with 1000000,5 dimensions, or even 2000000,10.

Perhaps you can give us a little more code?  A smallest example program that fails?

-Carl

[Rod]

Also what version of Liberty are you using. Older versions have less access to memory. And is it a numeric array or a string array. As Carl says code would be good.

[tenochtitlanuk]

Following code 'works' on LB4.5 on my Wine/Linux laptop with 4G RAM. It took about 5min to allocate the memory- I haven't tried USING the arrays. Definitely 'big data' as they say nowadays... Do you need all this in memory- can you refer to it say in RAMdisk??

   dim a( 1000000, 5)
   dim b( 1000000, 5)
   dim c( 1000000, 5)
   dim d( 1000000, 5)
   dim e( 1000000, 5)
   dim f( 1000000, 5)
   dim g( 1000000, 5)
   dim h( 1000000, 5)
   dim i( 1000000, 5)
   dim j( 1000000, 5)


[tsh73]

really interesting stuff
As John said, 2-dimensional arrays take lot's of time
But - is
dim a5(1000000,5)
supposed to use same amount of RAM as
dim a1(5000000)
?
Indeed (I decreased number of arrays to 4 because it really took very long for one try to finish)
single dimensional arrays compile and runs almost instantly
859 ms
t0=time$("ms")
dim a1(5000000)
print 1
dim a2(5000000)
print 2
dim a3(5000000)
print 3
dim a4(5000000)
print 4
'dim a5(1000000,5)
'print 5
'dim a6(1000000,5)
'print 6
'dim a7(1000000,5)
'print 7
'dim a8(1000000,5)
'print 8
'dim a9(1000000,5)
'print 9
'dim a10(1000000,5)
'print 10
print "Ok"
t1=time$("ms")
print t1-t0;" ms"

While two-dimensional take way longer - reports 61078 ms
70x slower, just for DIMs of same amount of memory
t0=time$("ms")
dim a1(1000000,5)
print 1
dim a2(1000000,5)
print 2
dim a3(1000000,5)
print 3
dim a4(1000000,5)
print 4
'dim a5(1000000,5)
'print 5
'dim a6(1000000,5)
'print 6
'dim a7(1000000,5)
'print 7
'dim a8(1000000,5)
'print 8
'dim a9(1000000,5)
'print 9
'dim a10(1000000,5)
'print 10
print "Ok"
t1=time$("ms")
print t1-t0;" ms"

and to my surprise *compile* part takes long time
Here is memory usage chart from Process explorer
big slope is compiling - it allocates memory!
"sawtooth" things is there program runs and executes DIM statements.


Also interesting then I changed index order to
dim a1(5,1000000)
it runs much faster then
dim a5(1000000,5)
- reported 3172 ms

Of course one need to check if order of indexes affects speed of working with array... but I have no meaningful task for that.

[Carl Gundel]

May 5, 2022 16:25:25 GMT -5 tsh73 said:

really interesting stuff
As John said, 2-dimensional arrays take lot's of time
But - is
dim a5(1000000,5)
supposed to ust same amount of RAM as
dim a1(5000000)
?
Indeed (I decreased number of arrays to 4 because it really took very long for one try to finish)
single dimensional arrays compile and runs almost instantly
859 ms

Also interesting then I changed index order to
dim a1(5,1000000)
it runs much faster then
dim a5(1000000,5)
- reported 3172 ms

Of course one need to check if order of indexes affects speed of working with array... but I have no meaningful task for that.

This is because double dimensioned arrays are implemented as an array of array objects.  It takes time to build them out.  Maybe I can improve on this with LB5.

This is not usually a problem because people don't usually try to use such large arrays.  On the plus side, it used to be impossible to do this because our computers didn't have enough memory to have such large data structures.  However I will try to make it faster.

[Rod]

From memory an array$ takes two bytes un filled and an array takes four. So while we can create such vast arrays the problem then becomes actual filled size an processing time to skip records.

A description of the task in hand might get better, specific processing help.

[James Grubbs (Jimmy)]

In case anyone else has this error, check to make sure an exclamation point isn't being printed as a leading character in a text box or text editor. LB gives this error message when you do that.

[Carl Gundel]

Jun 9, 2022 0:33:38 GMT -5 James Grubbs (Jimmy) said:

In case anyone else has this error, check to make sure an exclamation point isn't being printed as a leading character in a text box or text editor. LB gives this error message when you do that.

Care to share a short example program?

[Carl Gundel]

Jun 9, 2022 9:00:32 GMT -5 Carl Gundel said:

Jun 9, 2022 0:33:38 GMT -5 James Grubbs (Jimmy) said:

In case anyone else has this error, check to make sure an exclamation point isn't being printed as a leading character in a text box or text editor. LB gives this error message when you do that.

Care to share a short example program?

The following short example does not throw any error.

 textbox #w.tb, 10, 10, 200, 25
 button #w.bttn, "OK", [ok], UL, 10, 40
 open "test" for window as #w
 #w.tb "!setfocus"
 wait


[James Grubbs (Jimmy)]

Jun 9, 2022 9:00:32 GMT -5 Carl Gundel said:

Jun 9, 2022 0:33:38 GMT -5 James Grubbs (Jimmy) said:

In case anyone else has this error, check to make sure an exclamation point isn't being printed as a leading character in a text box or text editor. LB gives this error message when you do that.

Care to share a short example program?

Sorry for the late response Carl, The error I was getting when simply printing 1 character at a time into a texteditor was indeed giving me the 'system primitive failed' error when it tried to print an exclamation point, however when I try to duplicate it with the exact program as before, I now get the proper error 'the collection is empty' which is what is supposed to happen. The only explanation I have come up with thus far as I have upgraded linux mint to 20.3 recently and am running a different version of wine. It was originally 19.x and not sure what version of wine. I've loved using this software since the early 90's. I'm certain of the error I received and like you, I was confused as to how.