Grabbing Data ?

[mark]

Hi, I want to try to create a lottery program using the Virginia Pick 3 lottery game. I thought this might be a good program to learn the Liberty Basic code better by doing. I am just getting started and to begin I wanted to download the previous drawn numbers. I tried using a code I found on this message board but I couldn't get it to work right.

The code I used was:

asxcode$="BHP"

selmonth$="Jun"

year$="2020"

global asxcode$,selmonth$,year$

call Table2CSV asxcode$

WAIT

' --------------------------------------

SUB Table2CSV asxcode$

' change this to your web page

'webPage$    = "https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"

 webPage$="https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"

' change this to your output csv file name

print webPage$

fileName$    = asxcode$



htmlData$    = httpget$(webPage$)



lowData$    = lower$(htmlData$)

open DefaultDir$+"\"+"DATA\"+ fileName$+".txt" for output as #f



[nxtTbl]

tblBeg        = instr(lowData$,"<table",tblEnd)

if tblBeg     = 0 then goto [endTbl]

tblEnd        = instr(lowData$,"</table>",tblBeg)

tblData$    = mid$(htmlData$,tblBeg,(tblEnd - tblBeg) + 8)

lineBeg        = 0

lineEnd        = instr(tblData$,"</tr>",lineBeg)



while lineEnd > 0

   lineData$    = mid$(tblData$,lineBeg - 1,(lineEnd - lineBeg) + 1)

   lineData$    = strip$(lineData$)

   lineData$    = strRep$(lineData$,"</td>",",")

   lineData$    = strRep$(lineData$,"</th>",",")

   lineData$    = tags$(lineData$,"")

   lineData$    = strRep$(lineData$,"&nbsp;"," ")

   lineData$    = strRep$(lineData$,"&amp;","and")



 print "";left$(lineData$,len(lineData$) -2)

   print #f, "";left$(lineData$,len(lineData$) -2)



   lineBeg        = lineEnd + 1

   lineEnd        = instr(tblData$,"</tr>",lineBeg)

wend

print "----------------- end <table> --------------------------"



goto [nxtTbl]



[endTbl]

close #f

END SUB





' -----------------------------------------

' strip junk

' -----------------------------------------

FUNCTION strip$(str$)

   strip$ = ""

   for i = 1 to len(str$)

       a$ = MID$(str$,i,1)

       a = ASC(a$)

       if a > 31 then

                  if a <  127 then

            if a$ <> "'" then

                      if a$ <> "" then

           strip$ = strip$ + a$

              end if

            end if

          end if

       end if

   next i

END FUNCTION



' --------------------------------

' string replace rep str with

' --------------------------------

FUNCTION strRep$(str$,rep$,with$)

ln    = len(rep$)

ln1    = ln - 1

i    = 1

while i <= len(str$)

   if mid$(str$,i,ln) = rep$ then

       strRep$ = strRep$ + with$

       i = i + ln1

   else

       strRep$ = strRep$ + mid$(str$,i,1)

   end if

i = i + 1

WEND

END FUNCTION



' --------------------------------

' Replace HTML <tags> with$

' --------------------------------

FUNCTION tags$(tag$,with$)

   i    = instr(tag$,"<")

   j    = instr(tag$,">",i)

WHILE j > i

   tag$    = left$(tag$,i - 1) + mid$(tag$,j + 1)

   i    = instr(tag$,"<")

   j    = instr(tag$,">")

WEND

tags$    = tag$

END FUNCTION

I tried it with a normal link to a website and it seems to work, But when using this link it will not. This link on the website normally downloads a .txt file with previously drawn numbers. Is there a way to get this or some other code to download and read the .txt file? Or maybe if I downloaded the file be able to read the text file from my desktop or something similar. Any suggestions or ideas would be appreciated.

This is what the txt file looks like after downloading it (partial) from the lottery website. Va Lottery Link

6/9/2020; Day: 9,1,6; Night: 4,1,0

6/8/2020; Day: 9,4,1; Night: 4,2,1

6/7/2020; Day: 7,8,5; Night: 3,9,7

6/6/2020; Day: 2,2,6; Night: 1,7,7

6/5/2020; Day: 9,0,5; Night: 4,3,0

6/4/2020; Day: 3,2,5; Night: 4,3,7

6/3/2020; Day: 3,9,6; Night: 3,5,3

6/2/2020; Day: 0,4,1; Night: 0,9,5

6/1/2020; Day: 9,4,2; Night: 3,9,5

5/31/2020; Day: 1,5,1; Night: 1,5,4


I would like to extract the numbers to write to another text file something like for the program to read:



916

410

941

421

785

397

etc

etc

[Chris Iverson]

HTTPGET$() is getting the data just fine.

webPage$="https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"
print httpget$(webPage$)

The problem is the rest of the code attempting to parse the data like an HTML file instead of a text file.

It's possible the interface for retrieving data from the website has been changed, in which case, there's nothing to do but rewrite completely.


My recommendation: use HTTPGET$(), save the results directly to a file, then go through that file line-by-line with LINE INPUT, pulling out the data you need.

[metro]

There are plenty of options, those that are proficient with strings will manipulate htmlData$ to get the desired result

This may give you some ideas, I have not solved it completely for you cos then you don't learn.....and the learning never stops

EDIT : I see Chris never sleeps Ninja'd again

webPage$="https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"

'goto [JUMP]' USED THIS ONCE I HAD THE DOWNLOAD

htmlData$    = httpget$(webPage$)
open "myluckyNumbers.txt" for output as #1
print htmlData$
#1, htmlData$
close #1

[JUMP]
dim linedata$(20000) ' YOU COULD COUNT THEM FIRST
' THEN OPEN TO STORE INTO AN ARRAY
'*************Count the Records **************
open "myluckyNumbers.txt" for input as #1
i=1
while not(eof(#1))
   line input #1, linedata$(i)
   i=i+1
 wend
 close #1

 for a = 2 to i -4 ' REMEBER THE LINES AT THE END WE DO NOT WANT
                   ' also we do not want the first line
Night$=   WORD$(linedata$(a),2,";")
Night$= AFTER$(Night$,":")
if Night$ <> "" then print Night$
Day$=   WORD$(linedata$(a),1,";")
Day$= AFTER$(Day$,":")
If Day$ <> "" then print Day$
 next a

Print " Now to win some money"
 end





' -----------------------------------------

' strip junk

' -----------------------------------------

FUNCTION strip$(str$)

   strip$ = ""

   for i = 1 to len(str$)

       a$ = MID$(str$,i,1)

       a = ASC(a$)

       if a > 31 then

                  if a <  127 then

            if a$ <> "'" then

                      if a$ <> "" then

           strip$ = strip$ + a$

              end if

            end if

          end if

       end if

   next i

END FUNCTION



' --------------------------------

' string replace rep str with

' --------------------------------

FUNCTION strRep$(str$,rep$,with$)

ln    = len(rep$)

ln1    = ln - 1

i    = 1

while i <= len(str$)

   if mid$(str$,i,ln) = rep$ then

       strRep$ = strRep$ + with$

       i = i + ln1

   else

       strRep$ = strRep$ + mid$(str$,i,1)

   end if

i = i + 1

WEND

END FUNCTION



' --------------------------------

' Replace HTML <tags> with$

' --------------------------------

FUNCTION tags$(tag$,with$)

   i    = instr(tag$,"<")

   j    = instr(tag$,">",i)

WHILE j > i

   tag$    = left$(tag$,i - 1) + mid$(tag$,j + 1)

   i    = instr(tag$,"<")

   j    = instr(tag$,">")

WEND

tags$    = tag$

END FUNCTION


[mark]

Thank you Chris and metro. I will play around with these and see what I can come up with.

[mark]

metro, it looks like this is the code to omit the 1st line and the last line, but when it prints out to the txt page it does not omit the lines. Am I assuming right that the code is for omitting those lines or is it something else.
 for a = 2 to i -4 ' REMEMBER THE LINES AT THE END WE DO NOT WANT
                   ' also we do not want the first line
Never mind on above question. I see it only works for the array section (I think it is an array)






Also when executing the program the part where the numbers are stripped of junk prints out fine on the pop up window but in the text file they do not print. Was this something you left for me to figure out myself? So far I'm stumped but still trying to figure it out. Just checking.


One last thing, at the beginning you have a goto in which you commented you used once you had the download.
'goto [JUMP]' USED THIS ONCE I HAD THE DOWNLOAD
What does that mean? I know what a GOTO is but didn't understand the comment. I tried using it and it would not work. I was just wondering what you meant by that and what does it do or did?

Thanks a lot.

[metro]

Jun 10, 2020 3:54:57 GMT -5 mark said:

metro, it looks like this is the code to omit the 1st line and the last line, but when it prints out to the txt page it does not omit the lines. Am I assuming right that the code is for omitting those lines or is it something else.
 for a = 2 to i -4 ' REMEMBER THE LINES AT THE END WE DO NOT WANT
                   ' also we do not want the first line
Never mind on above question. I see it only works for the array section (I think it is an array)






Also when executing the program the part where the numbers are stripped of junk prints out fine on the pop up window but in the text file they do not print. Was this something you left for me to figure out myself? So far I'm stumped but still trying to figure it out. Just checking.


One last thing, at the beginning you have a goto in which you commented you used once you had the download.
'goto [JUMP]' USED THIS ONCE I HAD THE DOWNLOAD
What does that mean? I know what a GOTO is but didn't understand the comment. I tried using it and it would not work. I was just wondering what you meant by that and what does it do or did?

Thanks a lot.

Correct , The lines are left out when reading the file into the array. Which, if you wanted to you could print the result to another (or the sane file  name)

instead of the mainwin and that file would not have the offending lines.

The GOTO, I placed  the "rem" ( ie single quote) to do the initial download

once I had the data in a file I did not need to keep downloading it so removed the ' and the download

will not happen again the program wil JUMP to [JUMP].

 We can work on methods to get the data into the format you want just from what is in the file.

 Hope that makes sense

[mark]

OK, thanks. I appreciate your help. I'll do my best to figure it out if I can.

[metro]

Also when executing the program the part where the numbers are stripped of junk prints out fine on the pop up window but in the text file they do not print. Was this something you left for me to figure out myself? So far I'm stumped but still trying to figure it out. Just checking.

Sorry I should have left those functions out of the code

not needed ATM they were left over from the original code (which was mine from another post)

Just keep asking questions............

[mark]



' webPage$="https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"



'goto [JUMP]' USED THIS ONCE I HAD THE DOWNLOAD



'htmlData$    = httpget$(webPage$)

'open "myluckyNumbers.txt" for output as #1

'print htmlData$

'#1, htmlData$

'close #1



'[JUMP]

dim linedata$(20000) ' YOU COULD COUNT THEM FIRST

' THEN OPEN TO STORE INTO AN ARRAY

'*************Count the Records **************

open "myluckyNumbers.txt" for input as #1

i=1

while not(eof(#1))

   line input #1, linedata$(i)

   i=i+1

 wend

 close #1



 for a = 2 to i -4 ' REMEBER THE LINES AT THE END WE DO NOT WANT

                   ' also we do not want the first line

Night$=   WORD$(linedata$(a),2,";")

Night$= AFTER$(Night$,":")

if Night$ <> "" then print Night$

Day$=   WORD$(linedata$(a),1,";")

Day$= AFTER$(Day$,":")

If Day$ <> "" then print Day$

 next a



Print " Now to win some money"







open "test.txt" for output as #test

print #test, linedata$

close #test



   end







' -----------------------------------------



' strip junk



' -----------------------------------------



FUNCTION strip$(str$)



   strip$ = ""



   for i = 1 to len(str$)



       a$ = MID$(str$,i,1)



       a = ASC(a$)



       if a > 31 then



                  if a <  127 then



            if a$ <> "'" then



                      if a$ <> "" then



           strip$ = strip$ + a$



              end if



            end if



          end if



       end if



   next i



END FUNCTION







' --------------------------------



' string replace rep str with



' --------------------------------



FUNCTION strRep$(str$,rep$,with$)



ln    = len(rep$)



ln1    = ln - 1



i    = 1



while i <= len(str$)



   if mid$(str$,i,ln) = rep$ then



       strRep$ = strRep$ + with$



       i = i + ln1



   else



       strRep$ = strRep$ + mid$(str$,i,1)



   end if



i = i + 1



WEND



END FUNCTION







' --------------------------------



' Replace HTML <tags> with$



' --------------------------------



FUNCTION tags$(tag$,with$)



   i    = instr(tag$,"<")



   j    = instr(tag$,">",i)



WHILE j > i



   tag$    = left$(tag$,i - 1) + mid$(tag$,j + 1)



   i    = instr(tag$,"<")



   j    = instr(tag$,">")



WEND



tags$    = tag$



END FUNCTION





The above is what I have right now. I am trying to write to an external file called test.txt.




Print " Now to win some money"





open "test.txt" for output as #test

print #test, linedata$

close #test





end


' -----------------------------------------

' strip junk




I am not sure if I am doing something wrong or placing it in the wrong place. I can't get it to work in the program no matter where I place it.
  Is the linedata$
the correct variable I should be using? I'm not good at arrays yet so I am not sure.

[metro]

Nearly there, just need to loop through the data (array) then print it to the file when it meets the criteria
and there was an oops in the code got the wrong WORD$ needed 2 & 3.... see code
HTH

Goodluck
' webPage$="https://www.valottery.com/sitecore/shell/api/sitecore/DrawPastNumbers/DownloadAll?gameId=1050"
'goto [JUMP]' USED THIS ONCE I HAD THE DOWNLOAD
'htmlData$    = httpget$(webPage$)
'open "myluckyNumbers.txt" for output as #1
'print htmlData$
'#1, htmlData$
'close #1



'[JUMP]

dim linedata$(20000) ' YOU COULD COUNT THEM FIRST
' THEN OPEN TO STORE INTO AN ARRAY
'*************Count the Records **************
open "myluckyNumbers.txt" for input as #1
i=1
while not(eof(#1))
   line input #1, linedata$(i)
   i=i+1
 wend
 close #1
' OK WE HAVECLOSED THE FILE WE READ FROM

'NOW OPEN ONE TO SAVE THE INFO

open "test.txt" for output as #test
'NOW LOOP THROUGH THE ARRAY TO GRAB THE DATA WE NEED
' We ARE STARTING AT 2
for a = 2 to i -4 ' REMEBER THE LINES AT THE END WE DO NOT WANT
                   ' also we do not want the first line

Night$=   WORD$(linedata$(a),3,";")
Night$= AFTER$(Night$,":")
if Night$ <> "" then
print #test, Night$
end if

Day$=   WORD$(linedata$(a),2,";")
Day$= AFTER$(Day$,":")
If Day$ <> "" then
print #test, Day$
end if

next a
' AND LOOPING UNTIL 4 ROWS FROM THE END
close #test ' MUST CLOSE THE FILE

Print " Now to win some money"








   end








[mark]

I guess I had better study up on arrays next. I would have never figured that out. Having fun. Thanks a lot.

[mark]

Does it matter if the 20000 is a lot larger than needed? I bumped it up to 100000 and see no difference. I see that dim stands for dimensions to give a count to arrays. So I thought since the program would constantly grow I would enlarge it.

dim linedata$(20000) ' YOU COULD COUNT THEM FIRST

[metro]

Jun 10, 2020 6:52:19 GMT -5 mark said:

Does it matter if the 20000 is a lot larger than needed? I bumped it up to 100000 and see no difference. I see that dim stands for dimensions to give a count to arrays. So I thought since the program would constantly grow I would enlarge it.

dim linedata$(20000) ' YOU COULD COUNT THEM FIRST

I think I mentioned in my first post that you can dimension the array first. There are actually 10,422 lines in the original file

but here is a good read  alycesrestaurant.com/lbpe/TutorialSequentialTextFile.html

Also, I have just noticed back in the 1990's there were only night games . I would use INSTR to test if "Day" is in linedata$() then decide which part of the string gets saved....So lookup INSTR  and SELECT CASE in the help file

 Counting the Data

Sometimes the number of data needs to be known. This is especially true if an array is used to capture the data. If a sequential text file contains an unknown number of data, use a counter to determine that exact number. Increase the counter by one with each data read.

ct = 0
OPEN "TwainNovels.txt" for INPUT as #1
DO
ct = ct + 1
INPUT #1, n$
PRINT ct, n$
LOOP WHILE EOF(#1) = 0
CLOSE #1
PRINT "There are ";ct;" items of data in this text file."
END

[mark]

I get 19058 lines with the result. Looking at the text file all the numbers look (even the 1990's) accurate with no problems that I can see.


Using the Count code above I get this. It counts each digit in the number.

...partial...
57166         5
57167         2
57168         2
57169         9
57170         1
57171         4
There are 57171 items of data in this text file.


I think I mentioned in my first post that you can dimension the array first. There are actually 10,422 lines in the original file
but here is a good read alycesrestaurant.com/lbpe/TutorialSequentialTextFile.html
Also, I have just noticed back in the 1990's there were only night games . I would use INSTR to test if "Day" is in linedata$() then decide which part of the string gets saved....So lookup INSTR and CASE in the help file.

As soon as I get a little sleep I will read the material you suggested. Thanks a lot!