using global vars slows down the program

[tsh73]

Trying to speed up rotating donut program
libertybasiccom.proboards.com/thread/2482/ascii-3d-spining-donut

I tried to move local parameters in nested functions to global variable
But it made program slower!

So mere
*using global variable*
is SLOWER then
* assigning local variable
and using *that* *

I really expected using global variable SAVE assignment time (done once, up front).
Especially the assignment time in LB4 is about 10x time of math operator, try this program and see
justbasiccom.proboards.com/thread/901/benchmarking-fast-different-stuff-jb

This behavior (using global is slower) is same in LB5.

I would say it is weird and counter intuitive as for trying to optimize program.
I really wonder if there is something in LB code that could be fixed regarding to this.
May be LB5 could benefit from it.

Here is the demo, number is function calls per second (more == faster).
Variant with globals is slowest.
I really expected it to be about as fast as last line (not using assignment, instead use preassigned (local) variable (again))
pass data as param
k= 43187
pass data as const
k= 42962
using local variable to use 0.001
k= 43494
using constant to use 0.001
k= 43389
using global variable to use 0.001
k= 42167
not using 0.001
k= 43966


'time testing
global p4
p4=0.001

gosub [waitSecondEdge]
'so how many function calls per second?
t0=time$("seconds")
k=0
p1=0.001
while t0=time$("seconds")
   'scan
   y = f1(p1)
   k=k+1
wend
print "pass data as param"
print "k= ";k

gosub [waitSecondEdge]
t0=time$("seconds")
k=0
while t0=time$("seconds")
   'scan
   y = f1(0.001)
   k=k+1
wend
print "pass data as const"
print "k= ";k

gosub [waitSecondEdge]
t0=time$("seconds")
k=0
while t0=time$("seconds")
   'scan
   y = f2()
   k=k+1
wend
print "using local variable to use 0.001"
print "k= ";k

gosub [waitSecondEdge]
t0=time$("seconds")
k=0
while t0=time$("seconds")
   'scan
   y = f3()
   k=k+1
wend
print "using constant to use 0.001"
print "k= ";k

gosub [waitSecondEdge]
t0=time$("seconds")
k=0
while t0=time$("seconds")
   'scan
   y = f4()
   k=k+1
wend
print "using global variable to use 0.001"
print "k= ";k

gosub [waitSecondEdge]
t0=time$("seconds")
k=0
while t0=time$("seconds")
   'scan
   y = f5()
   k=k+1
wend
print "not using 0.001"
print "k= ";k

end '---------------------------------------
'set time on the end of second
[waitSecondEdge]
   t0=time$("seconds")
   while t0=time$("seconds")
       scan
   wend
return

'---------------------------------------

function f1(p1) 'using param to pass value 0.001
   x=1.2
   f1=p1+x
end function

function f2() 'using local variable to use 0.001
   p2=0.001
   x=1.2
   f2=p2+x
end function

function f3() 'using constant to use 0.001
   x=1.2
   f3=0.001+x
end function

function f4() 'using global variable to use 0.001
   x=1.2
   f4=p4+x
end function

function f5() 'NOT using 0.001
   x=1.2
   f5=x+x
end function


[Chris Iverson]

I don't know for sure, but I would guess that the problem is probably memory access.

Even counting the assignment, everything in the local scope is fairly close by, and likely to be in the lowest level cache. The values can be manipulated directly there.
The global variable, on the other hand, requires reaching out to another point of memory, possibly two times over(depending on how global variables are kept track of, it might need to access through a couple of pointers.)

If that memory isn't immediately available in the lowest cache, it'll have to be pulled down. That slows things down.
You wouldn't likely notice it for just one access, but if you're making thousands, hundreds of thousands, millions of accesses... that time adds up.


I posted a comparison a while back of the various speeds of different access methods: libertybasiccom.proboards.com/thread/1812/fast-cache-memory-compared-drive

These three in particular:

L1 cache reference ......................... 0.5 ns
---
L2 cache reference ........................... 7 ns
---
Main memory reference ...................... 100 ns

Now, those are in nanoseconds, which are 1.0 x 10^-9 seconds. There are 1000 milliseconds in a second, and there are a MILLION nanoseconds in a millisecond. Differences of magnitude at that scale are kind of hard to grasp, so that comparison I linked to scaled it up to be understandable on a human timescale, by adjusting it so that 1 nanosecond = 1 second. That results in the following:

L1 cache reference                  0.5 s        
---
L2 cache reference                  7 s          
---
Main memory reference               100 s

Using the human-scale versions of those speeds, if the values we need are in L1 cache, then the total access times for this test would be ~40,000 x 0.5s, or 20,000 seconds(333.33~ minutes, or 5.5~ hours).

If the values we need are in L2 cache, then those access times are ~40,000 x 7s, or 280,000 seconds. This is still in cache, and yet we've hit access times equivalent to 77 hours.

If the values aren't in either cache, and we need to go all the way to memory(there's actually an L3 cache nowadays, those example values are outdated, but the point still stands), those access times become 40,000 x 100s, or 4 million seconds. That's 46 days.

Now, again, this is operating at human timescale, but even in the examples given, the first total would be 20 microseconds(or 0.00002 seconds), and the last would be 4 milliseconds(or 0.0004 seconds). That's a vast difference, literally an order of magnitude.

[Carl Gundel]

Guys, any conventional analysis of performance for variable access is probably not really fruitful here because LB (both LB4 and LB5) are compiled to an object model (meaning object oriented model) that is then executed by just-in-time native code compiler.

In any case, I do care about performance and I hope that the improvements in performance of LB5 over LB4 are a convincing proof of that.  But... I will probably not spend much more time trying to optimize Liberty BASIC 5 until after it is finally released.  Liberty BASIC v5.1 will probably contain several performance enhancements.

So I have to ask, is it fast enough to do what you need right now?

Thanks for your patience,

-Carl

[Chris Iverson]

Honestly, I'm just having fun speculating. You aren't wrong that the different compilation layers make specific analysis a bit of a wash.

As it is, I've got no problems with the speed shown so far of LB5.

[tsh73]

Hello Carl
First, thank you for the music ((C) ABBA)

So I have to ask, is it fast enough to do what you need right now?

LB/JB is not about speed. It's about fun.
Of cource programmer always have thing or two he could add with more speed available
But if really pressed, one could take another tool.
(LB5/LBB/VB.NET/any other language that happened to fit better)

This issue is not quite about speed
It's about tool (LB) conforming to a mental model
(of cource it is MY mental model, so problem is actually in my head. Others could feel different.)
LB is not only language I use/used, so is for many LB users
So I expect it to behave like I learned from other languages. Including Old BASICs as far as ZX SPECTRUM one.
Like this:
Q: I need some speed?
A: Get common subexpression out, assign to a var and use it several times.
(OOPS assignment is costly. Will it worth? have to measure...)
A: Use a variable assigned far above (* happened to be global)
(OOPS global use happens to be slower then assigning local and use it)
(
Anecdotal evidence.
(that is, I remember fretting about it, but I already forgot what program it was)
I measured that to pass parameter into nested functions, 4 levels deep, is faster then using global variable.
I thought it was due to assignment cost. Now I think it about using global variable.
)

I tell students that BASIC is compact language, it "fits" into programmer's head entirely.
That makes programmer productive.
But these "OOPS" moments - not.

[atomose]

I'm not going to say that speed isn't important.
I mainly use LB to create games and simulations. I even created a video game studio... (atomose.fr)

so yes it's important but what made me choose LB was not its performance but the pleasure of coding with it (the single window, the syntax (goto , wait, [], etc)). It's a whole and the performance is a bonus ^^.

I really hope to see an LB 5 as stable and functional as the LB 4.51 with the added possibilities.

[Chris Iverson]

Dec 10, 2023 3:59:03 GMT -5 tsh73 said:

It's about tool (LB) conforming to a mental model
(of cource it is MY mental model, so problem is actually in my head. Others could feel different.)
LB is not only language I use/used, so is for many LB users
So I expect it to behave like I learned from other languages. Including Old BASICs as far as ZX SPECTRUM one.
Like this:
Q: I need some speed?
A: Get common subexpression out, assign to a var and use it several times.
(OOPS assignment is costly. Will it worth? have to measure...)
A: Use a variable assigned far above (* happened to be global)
(OOPS global use happens to be slower then assigning local and use it)
(
Anecdotal evidence.
(that is, I remember fretting about it, but I already forgot what program it was)
I measured that to pass parameter into nested functions, 4 levels deep, is faster then using global variable.
I thought it was due to assignment cost. Now I think it about using global variable.
)

Thing is, when it comes to optimization, with everything that happens on modern computers, having any kind of "mental model" for premature optimization is basically moot.

There are so many levels of things that affect how fast things run in seemingly strange ways even if you're coding at the level of C. Adding two more languages on top of that(Smalltalk and LB), and you have even more in-between stuff that's affecting things.

Heck, modern CPUs even INVISIBLY CHANGE HOW THEY OPERATE¹ to make your programs faster! While it does make your programs faster, if you don't realize that's what's happening, it can be very confusing to understand where the speed difference came from.


That's not to say you shouldn't optimize. You absolutely should. But you should profile your code to see what's actually running slow, and then test alternatives to see what's faster. There's not going to be any hard and fast rules; you just have to test and see what winds up faster for you.


¹A lot of CPU-based speedups come from speculative execution, which is completely invisible to the system, except for code running faster. (Well, in theory.)

There's a fascinating post on StackOverflow about someone trying to understand why sorting an array makes their C++ code execute in literally 10% of the total needed time of the unsorted array.
They had code that was making a sum of all the values in the array that were greater than 128. Sorting the array made the code run much, much faster, even though the same actual calculations are being made. (Each element is compared with 128, and if the element is greater, it's added to the sum. Otherwise, it's ignored.)

People in the answers pointed out that the sorted array likely let the branch prediction in the CPU speed things up immensely by being able to pre-execute code, whereas the randomized, unsorted array would cause branch prediction and speculative execution to fail.

Here's the post in question, which includes a very good high-level explanation of branch prediction: stackoverflow.com/questions/11227809/why-is-processing-a-sorted-array-faster-than-processing-an-unsorted-array

[enzo]

Again, as I said on JustBASIC forums assignment cost as you say does not linearize to a real program. Too many time input variables. Example no ones data (code; variables) is linearized in character length except, this give rule over: to databases.