Share data between processes

[nixa]

Hello! Long time Lyberty BASIC user here.

I'm making program for navigation and route planing (SLAM, actualy) of my robot lawn mower.
Since using a single thread was to slow, I've separated my program to 3 separate threads. First for managing map (map.tkn), second for obsticle and beacon detection (position.tkn), and third for route planing (path.tkn).

As I start map.tkn, it loads bmp file to graphicbox to be used as grid map of limits, fixed obsticles and beacons locations, and static text controlls for receiving data from position and path programs.
Map program starts map.tkn and position.tkn with command line parameters consisting of hwnd of statictext and graphicbox. On first start of position.tkn, it reads data from graphicbox pixel by pixel using 
CallDLL #gdi32, "GetPixel", hDC as uLong,x As Long,y As Long, GetPixel As Long
creates its own texteditor controll, and sends texteditors hwnd to map.tkn statictext controll using
CallDLL #user32, "SetWindowTextA", hWnd as uLong, txt$ As Ptr, result As Void
By receiving that hwnd, map.tkn knows that position.tkn is redy and clears statictext using
CallDLL #user32, "SetWindowTextA", hWnd as uLong, txt$ As Ptr, result As Void
That is basicly the way I'm comunicating between separate processes.


I'm wondering, is there better/faster/safer way to share data between processes???

I think, I should be using "named pipes" for this, but I have no idea how that works.

[Rod]

Sounds like you have had a lot of fun building that! My immediate reaction is to question why three processes? What is too slow? Without seeing the programs I would guess getpixel is the thing that is really slowing you down.

I will search for links but you will find several threads that discuss alternatives to getpixel, these access the bmp file directly and are far faster than getpixel.

I think your solution for sharing info is slick, but it adds to the process, I cant help feel it would all be quicker in one program. So tell us more about what is slowing it all down, are you using getpixel continuously or only at the start?

linklink

[tenochtitlanuk]

LB's 'set' is slow, because of all the background work in updating and displaying an image, and getpixel is indeed much faster with dll or the pixel-file-save methods.

Don't really understand your program and context. For me I've had a lot of fun saving pixel data in a 2D array and saving it as a BMP when needed, so it can be used on-screen via 'loadbmp'.

This image shows an initial image, and versions where I swapped all R and B components; where I grey-scaled all pixels; and where I vignetted all within a circle. Takes a second or two with my method- but ages using getting and setting pixels.

[nixa]

My map is 2700 x 1800 pixels. Each pixel represents 10x10cm. Simply inverting colors with "GetPixel" and "SetPixel" of entire map, takes about 997243 ms (16.6 minutes). Reading map to 2D array takes 203635 ms (3,4 min), and inverting array only takes 12106 ms (12.1 seconds).

Anyway,
by reading pixel with "GetPixel" i get number from 0 to 16777215.
On my map, 1 represents fixed untraversable part of map. Ie, part of map that is out of limits, trees, or buildings. 0 is beacon. 2 is obstacle and 16777215 is traversable.
Each time laser its something at X, Y it substracts 1 from each pixel, if there is no obstacle, it ads 1 to pixel. That is managed by map.tkn and that is how I acheave "SLAM". That is done inside graphicbox so data is available for position.tkn and route.tkn.

Position.tkn is actual program that comunicates with robot. That program reads data from robot, sends it to map.tkn and if needed, to route.tkn. Also, Position.tkn reads map to 2d array and it translates GPS data to my 10 cm grid map and vice versa.

When robot sends "beacon found - I'm at -x:y- GPS -N E- and beacon is at distance -d-, compass angle -a-", position.tkn calculates point that robot has sent and searches for closest beacon on map in circle paterns from the point that robot has sent. That way i can get about <20cm position precision without need to search entire map and compare data.
All sensor data is sent to map.tkn so it can always keep an updated map.

Finding closest beacon takes up to allmost half a second because GPS misses 2.5 up to 20 meters (say, avarage 7 meters). That is radius of 70 pixels, or 440 pixels to calculate and compare data. By my standards, that is slow. But acceptable. (I have to say that robot travels at up to 85 meters per minute or 1.5m per second)

Route.tkn doesent need to be fast so it uses "GetPixel" directly. It generates 1000 random points on map and check wich of generated coordinates have less than 500000. those get discarded couse of high posibility of being untraverseable. Then it calculates shorthest path and tests for passing over points with value under 500000 using remaining points and sends coordinates to position.tkn wich then crunches them back to GPS coordinates and sends them to robot.

If there would be a way for all 3 apps to read from and write to same 2d array, that would be awsome.

Rod
Sounds like you have had a lot of fun building that!
-Yes, I had alot of fun. I still do. My 5 years old son loves it.

My immediate reaction is to question why three processes? What is too slow?
-specifically updating map takes alot of time. Im updating pixels in graphicbox so they can be used by route. I still haven't managed make position.tkn to update its own map in "real time". Puting everything in single app would make primary loop very slow. When I would update map, and calculate robots position, robot might be a few meters away.

For simply navigating robot on existing map, single thread is more then enough. But not for simultaneous navigation and map updating. At least, not in the way I'm trying to do it.

[Rod]

Thinking about it it is the drawing of pixels or the drawing of text that is slowing it down, so too the reading of the screen pixel. If it has to be on screen it has to battle through layers of Windows code before it can be seen and be read.

Now before I ramble on have a browse at this if you have not already seen it. It is about using environment variables to pass info between .tkn files.

Couple of things come into my head. First is just work on the .bmp file directly, you don't need to see it (continuously). File access is super fast these days, not so when other articles were written. But how to share? Well Windows and Liberty can open the same file for reading, its only writing that makes file sharing a no no. But I will need to experiment more as I am interested to see whats possible.

To give you a feel for what I am thinking look at this code, be sure not to trash your real .bmp. This code does not need to load the array, can access 440 pixels in 40ms but as yet cant share.

 WindowWidth  = 700
   WindowHeight = 400
   UpperLeftX   = (DisplayWidth-WindowWidth)/2
   UpperLeftY   = (DisplayHeight-WindowHeight)/2
   graphicbox #1.gb1,10,10,300,300
   graphicbox #1.gb2,330,10,300,300
   open "SET GET Pixel" for graphics_nf_nsb as #1
   #1 "trapclose [quit]"
   global bmpw,bmph,padding,offset,bytes,Blue,Green,Red




   [loadbmp]
       filedialog "Choose an image","*.bmp",file$
       if file$<>"" then
           'display the original bmp in gb1
           loadbmp "original",file$
           #1.gb1 "down ; drawbmp original 0 0"

           'now open the same bmp file for binary read/write access
           open file$ for binary as #bmp

           'analyse the file header bytes
           seek #bmp,10 'picture data offset, where the color data starts 4 bytes of data
           offset=asc(input$(#bmp,1))+asc(input$(#bmp,1))*256+asc(input$(#bmp,1))*65536+asc(input$(#bmp,1))*16777216
           seek #bmp,18 'width 4 bytes of data
           bmpw=asc(input$(#bmp,1))+asc(input$(#bmp,1))*256+asc(input$(#bmp,1))*65536+asc(input$(#bmp,1))*16777216
           seek #bmp,22 'height 4 bytes of data
           bmph=asc(input$(#bmp,1))+asc(input$(#bmp,1))*256+asc(input$(#bmp,1))*65536+asc(input$(#bmp,1))*16777216
           seek #bmp,28 'bits per pixel, ie color depth 2 bytes of data
           bits=asc(input$(#bmp,1))+asc(input$(#bmp,1))*256

           'work out start of picture data and how to move through file
           pointer=offset

           'work out how many bytes in the bits per pixel 24=3 bgr 32=4 abgr
           bytes=bits/8

           'work out padding each raster line must be a 4byte multiple
           mult=bits/8*bmpw/4
           padding = 4*(1-(mult-int(mult))) mod 4
           close #bmp

       end if

   print "note access is immediate, no need for array"
   goto [setpixel]

   [setall]
       'this demo updates every pixel of the image
       'takes 171 seconds on my pc
       s=time$("ms")
       print "started update of all pixels"
       print "note access is immediate, no need for array"
       open file$ for binary as #bmp

       for y=1 to bmph
           for x=1 to bmpw

               'run through binary file data moving the file pointer as we go
               seek #bmp, pointer

               'get pixel color stored as BGR Liberty needs RGB
               b=asc(input$(#bmp,1))
               g=asc(input$(#bmp,1))
               r=asc(input$(#bmp,1))

               'change the color
               r=r-150
               if r<0 then r=0

               'set pixel color
               'because we read three bytes we need to set the file pointer back
               seek #bmp, pointer
               #bmp chr$(b);chr$(g);chr$(r);

               'move on
               pointer=pointer+bytes
           next

           'add padding at end of raster line
           pointer=pointer+padding
       next

       'note we need to close the file to have Windows
       'write the last update
       close #bmp
       print "ending",time$("ms")-s;"ms"
       gosub [show]

   [setpixel]
       'this writes 440 random pixels
       'open temporary file for read and write access
       open file$ for binary as #bmp
       print "started writing"
       s=time$("ms")
       for n=0 to 440 'zero index
           null=setpixel(int(rnd(0)*bmpw),int(rnd(0)*bmph),0,0,0) 'x,y,r,g,b
       next
       print "ending write ";time$("ms")-s
       close #bmp
       gosub [show]

   [getpixel]
       'this reads 440 random pixels
       'open temporary file for read and write access
       open file$ for binary as #bmp
       print "started reading"
       s=time$("ms")
       for n=0 to 440 'zero index
           null=getpixel(int(rnd(0)*bmpw),int(rnd(0)*bmph)) 'x,y
           'the global RGB variables are filled
           'print Red,Green,Blue
       next
       print "ending read ";time$("ms")-s
       close #bmp

       wait


   [show]
       'show what we did, load file as bmp and display
       loadbmp "newfile",file$
       #1.gb2 "down ; drawbmp newfile 0 0"
       return

   [quit]
       close #1
       end


       function setpixel(x,y,r,g,b)
           'raster lines are stored bottom up so we invert y
           y=(bmph-1)-y
           pointer=offset+x*bytes+y*(bmpw*bytes+padding)
           'set pixel color
           seek #bmp, pointer
           'unrem this will trash the .bmp
           '#bmp chr$(b);chr$(g);chr$(r);
       end function

       function getpixel(x,y)
           'raster lines are stored bottom up so we invert y
           y=(bmph-1)-y
           pointer=offset+x*bytes+y*(bmpw*bytes+padding)
           'get pixel color stored as BGR Liberty needs RGB
           seek #bmp, pointer
           Blue=asc(input$(#bmp,1))
           Green=asc(input$(#bmp,1))
           Red=asc(input$(#bmp,1))
       end function








[tenochtitlanuk]

Rod and I both use these ways to handle fast pixel changes. I'd just point out that the need to calculate the pointer offset to the data triplets is removed if any bitmap BMP you wish to LOAD has been saved as 24 bit, has no colour table, and has width a multiple of 4. The pointer to a triplet is then directly to '54 +3 *x +y *width'.

You might also consider using a timer to save the array as a bmp and reload and display on screen. Between times, get on with the other calculations..

This sounds like a fun project...

[nixa]

I dont't have to make map visible to user. Actualy, I want to make program that will run on my server and I can enjoy watching robot crawling around...
So, no, I don't have to make map visible, nor i have to use bitmap. Using graphicbox, bitmap and getpixel/setpixel was in range of my knowledge of how to share data to 3 different programs at the same time.

I actualy don't even need 24 bits for storing data. I think, 16 bits would be just fine.

Envoromental variables seem very appealing. I never used them before. If map.tkn is managing map, i can just send coordinates of "pixel" that have changed traverseable/obstacle status to position.tkn and route.tkn
And i can simply make map.tkn do the path planing and discard route.tkn.

For robot, or any of the apps running, I don't think it is important that they all have EXACT up-to-date data at any given moment. So they can all just dump data to variables, and work with what ever is available at that moment. Another idea is to chop the map into sections. That should speed things up. Allso, I can make complete map resyncing during charging time.

I'll defenetly try variables and I'll share code once i have something worth sharing.

Please, don't get me wrong. I sad I'm long time liberty basic user. I am. But I'm a loooooong way from expert. So expect some dumb questions.

[Rod]

We look forwards to the dumb questions. I hope the environment variables work out.

I have three programs running right now. The main program is reading and writing to the .bmp file using open for binary. The two sub programs constantly loadbmp the .bmp file and following the changes without any conflict. So next step for me is to see if I can get the two subs reading the file directly.

BMP is a bit of a red herring, my interest is for database sharing. I know there is SQL etc but there must be room for a simpler Liberty BASIC sharing technique on a local network or shared drive.

[Brandon Parker]

Here are a couple of functions for using Environment Variables if you need them...

Print SetEnvironmentVariable("TestVariable", "Test Successful!!")
Print GetEnvironmentVariable$("TestVariable")
Wait

Function SetEnvironmentVariable(lpName$, lpValue$)
   CallDLL #kernel32, "SetEnvironmentVariableA", lpName$                   As ptr, _ 'Label of your environment var
                                                 lpValue$                  As ptr, _ 'String value to store in the environment var
                                                 SetEnvironmentVariable    As long   'Non-Zero if successful; Zero if unsuccessful
End Function

Function GetEnvironmentVariable$(lpName$)
   GetEnvironmentVariable$ = Space$(_MAX_PATH) + chr$(0)
   nSize = Len(GetEnvironmentVariable$)
   CallDLL #kernel32, "GetEnvironmentVariableA", lpName$                   As ptr, _   'Label of your environment var
                                                 GetEnvironmentVariable$   As ptr, _   'String buffer to hold value
                                                 nSize                     As long, _  'Length of buffer
                                                 result                    As long     'Length of returned data

   GetEnvironmentVariable$ = Left$(GetEnvironmentVariable$, result)
End Function

You might also be interested in using the Windows message WM_COPYDATA as well. You would have to use one of the available message handling DLL's to capture the message though. I have an example of this although it is within the same process; it just shows how the WM_COPYDATA message works. I will dig it up if there's interest ...


{:0)

Brandon Parker