recursive sub errors on unwind

[blametroi]

I've seen a couple of different errors during my testing, but they all appear to be variations of wrong variable values while returning back up the call chain. I'm writing some code to recursively use FILES to get a list of all files and directories under a starting directory. I've managed to create a reproducible error (subscript out of range) with the following directory structure:

e:\testlb

    backup.db

    new.db

    asubdir\

        asubsubdir\

    bsubdir\

Running the included code gets a Runtime Error: Subscript out of range: 4, filelist$()

Looks as if this is the first unwind.

Here's the code, I'm stumped. 4.5.1 licensed but not pro. There's some notes and debugging output, but not really a lot of code at the moment.

' recursedirs.bas -- try to get a list of all files and directories
'
' files command returns a 2d array of the contents of a -specific directory-
' row 0 describes the data in the rest of the array
' rows 1 -> n contain information on files in the directory
' rows n+1 -> end contain information on subdirectories
'
' row 0:
'   # files, # subdirs, drive spec with slash, directory path with slash
'
' row 1 -> n:
'   file name, file size in bytes, date and time stamp, attributes
'
' row n+1 -> end:
'   full directory path without drive spec, directory name, empty, empty
'
' therefore,
'
' drive spec (without trailing slash) + dir row(0) is full subdir
'
' drive spec + directory path + file name is full file name
'
' a trailing slash on the argument to FILES makes no difference *except*
' when your working directory is a sub directory ...
'
' eg. FILES "E:\" when working directory is "E:\BIN" returns information
' on E:\, whereas FILES "E:" will return information on E:\BIN\
'

global depth, maxdepth, filecount, dircount

depth = 0
maxdepth = 0
filecount = 0
dircount = 0

open "files.txt" for output as #txtout

call recurseDirs "e:\testlb"

close #txtout

print
print "directories: "; dircount
print "files: "; filecount
print "maximum directory nesting depth: "; maxdepth

print "done"

end

'
' read each directory and write info
'
' lots of work to do here
sub recurseDirs currentDir$

   depth = depth + 1
   if depth > maxdepth then maxdepth = depth
   dircount = dircount + 1

   print "depth: "; using("####", depth); "     directory: "; currentDir$

   'print #txtout, filelist$(0,0) ' qty files
   'print #txtout, filelist$(0,1) ' qty subdirs
   'print #txtout, filelist$(0,2) ' drive spec
   'print #txtout, filelist$(0,3) ' dir path

   dim filelist$(10,10)

   FILES currentDir$, filelist$()

' possible nasty situations: empty directory, what else? no sub dirs
   print val(filelist$(0,0)), val(filelist$(0,1))
   for i = 1 to val(filelist$(0,0)) + val(filelist$(0,1))
       print ">>>>> "; i; ", '"; _
           filelist$(i, 0); "', '"; _
           filelist$(i, 1); "', '"; _
           filelist$(i, 2); "', '"; _
           filelist$(i, 3); "'"
       if left$(filelist$(i,0), 1) = "\" then ' this is a directory entry
           call recurseDirs filelist$(i,0)
       else
           filecount = filecount + 1
           ' just ignore beyond counting for now
       end if
   next i

   depth = depth - 1
end sub





[Chris Iverson]

Arrays are always global. They are NOT function-local. The second time through the subroutine(to get the info from the first subdir), you corrupt the state of the array, and when the sub returns, it's still working with the recursed version of the array. The original state, with the original contents, is lost.

You need to save the original state in some way before recursing, into a local variable.


Here's a similar function I made a while back, to deal with the same issue.

folderPath$ = StartupDir$ + "lb4help"
print folderAction(folderPath$)

Function folderAction(path$)
   'If there's a trailing path separator, remove it
   if right$(path$, 1) = "\" then path$ = left$(path$, len(path$) - 1)

   dim folderActionInfo$(1, 1)
   files path$, folderActionInfo$()

   numFiles = val(folderActionInfo$(0, 0))
   numFolders = val(folderActionInfo$(0, 1))

   'Print out the name of the folder we're currently working with.
   'Move this to just before the [end] branch to perform the action AFTER going through
   'subfolders.
   print path$
   x = 0

   'Print the name of each regular file found
   if numFiles > 0 then
       do
           x = x + 1
           print path$;"\";folderActionInfo$(x, 0)
       loop while x < numFiles

   end if

   if numFolders > 0 then
       do
           'Convert the list of folder names in the array to a local string
           'This is done because arrays are global in LB, they are not locally-scoped.
           'If we didn't do this, we would lose all of the folder names when we called
           'the function recursively.
           if folderList$ <> "" then folderList$ = folderList$ + "|"
           x = x + 1
           folderList$ = folderList$ + folderActionInfo$(x, 1)
       loop while (x - numFiles) < numFolders

       'For each folder, call folderAction() again
       '
       'Note that this contains an error reporting mechanism:
       'if something goes wrong with the action you want to take on a file or folder,
       'set the return value of folderAction() to any non-zero value.
       '
       'This will stop the recursion instantly and back all the way out
       'to the code that initially called folderAction().
       For x = 1 to numFolders
           folder$ = word$(folderList$, x, "|")
           a = folderAction(path$ + "\" + folder$)
           if a then
               ERRFLAG = a
               exit for
           end if
       Next x
       if ERRFLAG then
           folderAction = ERRFLAG

           'We do a goto [end] to skip any post-processing actions.
           'This can be removed if you still want to perform them, or you
           'can put custom actions here to handle the case of an error before
           'returning to the calling code.
           goto [end]
       end if
   End If
   'folderAction = rmdir(path$)
   [end]
End Function

[blametroi]

Thanks Chris. Your solution looks good. I was going to end up learning to put all this data in sqlite at some point so I may just skip ahead in my work, but it's nice to have an option to stay all in basic. I'll put a pot of coffee on and start hacking

edit:

And I'm moving forward again, thanks Chris! I was afraid the marshaling into strings would max out my system, but memory usage of the lb process only increased by a couple of MB and CPU utilization was low.

I learned programming in the era when we did all this on tape and 5 MB hard drives were new and HUGE, and that has me worrying about optimization way too soon in a project :/

Chewing through my data drive, I have

directories: 25132
files: 314408
maximum directory nesting depth: 17

And neither lb nor my system broke a sweat.

[Carl Gundel]

Jul 22, 2019 3:58:55 GMT -5 blametroi said:

And I'm moving forward again, thanks Chris! I was afraid the marshaling into strings would max out my system, but memory usage of the lb process only increased by a couple of MB and CPU utilization was low.

I learned programming in the era when we did all this on tape and 5 MB hard drives were new and HUGE, and that has me worrying about optimization way too soon in a project :/

Back in the 80's I worked on a couple of business applications in PC-BASIC & IBM BASIC Compiler.  We had a single 10MB hard drive, and I don't think in the 4 years I was there that we filled it up.

Before long you needed a HD in the hundreds of megabytes, and now we're into the hundreds of gigabytes.  If we ever find it necessary to have 1 terabyte just to install an operating system we should probably just throw in the towel. 

Note: The core stuff in MS-DOS 6.22 fits on a 1.44MB floppy disk.

[Brandon Parker]

Jul 22, 2019 12:55:06 GMT -5 Carl Gundel said:

Before long you needed a HD in the hundreds of megabytes, and now we're into the hundreds of gigabytes.  If we ever find it necessary to have 1 terabyte just to install an operating system we should probably just throw in the towel. 

Note: The core stuff in MS-DOS 6.22 fits on a 1.44MB floppy disk.

Must be all those sloppy programmers out there ...


{:0)

Brandon Parker