Grabbing a current stock price?

[mikeschn]

Is there an easy way to grab a stock price using LB?



Or do I have to use httpget to grab the entire page and then search that page for the price? 

Suggestions?

Thanks,

Mike...

[ntech]

Hmm, is there an API you are looking at? The most straightforward way would be to grab the page and extract the data you want from there.

[metro]

May 1, 2020 16:44:29 GMT -5 ntech said:

Hmm, is there an API you are looking at? The most straightforward way would be to grab the page and extract the data you want from there.

I did use google and yahoo in the past, read this  for some guidance towardsdatascience.com/best-5-free-stock-market-apis-in-2019-ad91dddec984

or this medium.com/@andy.m9627/the-ultimate-guide-to-stock-market-apis-for-2020-1de6f55adbb

I use the "pain in the parse" method down here on the ASX

If your exchange displays prices in a table somewhere you may find some of this useful (or not)..the data here is for options on the ASX

asxcode$="BHP"
selmonth$="Jun"
year$="2020"
global asxcode$,selmonth$,year$
call Table2CSV asxcode$
WAIT
' --------------------------------------
SUB Table2CSV asxcode$
' change this to your web page
'webPage$    = "https://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode="+asxcode$+"&expiryDate=&optionType=B"
 webPage$="http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=" + asxcode$ + "&expiryDate="+ selmonth$ + "+" +year$+"&optionType=B"
' change this to your output csv file name
print webPage$
fileName$    = asxcode$

htmlData$    = httpget$(webPage$)

lowData$    = lower$(htmlData$)
open DefaultDir$+"\"+"DATA\"+ fileName$+".txt" for output as #f

[nxtTbl]
tblBeg        = instr(lowData$,"<table",tblEnd)
if tblBeg     = 0 then goto [endTbl]
tblEnd        = instr(lowData$,"</table>",tblBeg)
tblData$    = mid$(htmlData$,tblBeg,(tblEnd - tblBeg) + 8)
lineBeg        = 0
lineEnd        = instr(tblData$,"</tr>",lineBeg)

while lineEnd > 0
   lineData$    = mid$(tblData$,lineBeg - 1,(lineEnd - lineBeg) + 1)
   lineData$    = strip$(lineData$)
   lineData$    = strRep$(lineData$,"</td>",",")
   lineData$    = strRep$(lineData$,"</th>",",")
   lineData$    = tags$(lineData$,"")
   lineData$    = strRep$(lineData$,"&nbsp;"," ")
   lineData$    = strRep$(lineData$,"&amp;","and")

 print "";left$(lineData$,len(lineData$) -2)
   print #f, "";left$(lineData$,len(lineData$) -2)

   lineBeg        = lineEnd + 1
   lineEnd        = instr(tblData$,"</tr>",lineBeg)
wend
print "----------------- end <table> --------------------------"

goto [nxtTbl]

[endTbl]
close #f
END SUB


' -----------------------------------------
' strip junk
' -----------------------------------------
FUNCTION strip$(str$)
   strip$ = ""
   for i = 1 to len(str$)
       a$ = MID$(str$,i,1)
       a = ASC(a$)
       if a > 31 then
                  if a <  127 then
            if a$ <> "'" then
                      if a$ <> "" then
           strip$ = strip$ + a$
              end if
            end if
          end if
       end if
   next i
END FUNCTION

' --------------------------------
' string replace rep str with
' --------------------------------
FUNCTION strRep$(str$,rep$,with$)
ln    = len(rep$)
ln1    = ln - 1
i    = 1
while i <= len(str$)
   if mid$(str$,i,ln) = rep$ then
       strRep$ = strRep$ + with$
       i = i + ln1
   else
       strRep$ = strRep$ + mid$(str$,i,1)
   end if
i = i + 1
WEND
END FUNCTION

' --------------------------------
' Replace HTML <tags> with$
' --------------------------------
FUNCTION tags$(tag$,with$)
   i    = instr(tag$,"<")
   j    = instr(tag$,">",i)
WHILE j > i
   tag$    = left$(tag$,i - 1) + mid$(tag$,j + 1)
   i    = instr(tag$,"<")
   j    = instr(tag$,">")
WEND
tags$    = tag$
END FUNCTION

MyStocks.zip (253.53 KB)_  There maybe something useful in this I use it to find  stocks that have moved the most (either up or down) intra day

[mikeschn]

No luck yet! I can't even grab the page to scrape!

page$=httpget$("https://finance.yahoo.com/quote/VIAC?p=VIAC&.tsrc=fin-srch-v1")

for example, just yields a bunch of html code without the data!

Mike...

[svajoklis]

If you can't find data itself in the page, then it is probably loaded in dynamically with JavaScript. PhantomJS is a headless browser (a browser that runs all the code, just doesn't show a window) that you can control programatically, it would be able to download the page, run the required JS code to load the data and then after it completely loads you would be able to scrape off the data. I don't think Liberty BASIC has anything like that though...

A quick look around shows that page GETs some XHR requests, like this query1.finance.yahoo.com/v7/finance/spark?symbols=%5EDJI&range=1d&interval=5m&indicators=close&includeTimestamps=false&includePrePost=false&corsDomain=finance.yahoo.com&.tsrc=finance

If I didn't go the PhantomJS route, then I'd look into those (visible in the Inspect -> Network tab in Chrome, similar tools are available in Firefox and other browsers) and try to decode the URL so that Liberty could retrieve that. It gets back with a JSON file, which, again I don't think Liberty can tackle out of the box, so it would be a case of trying to parse it same as HTML - with plain text functions.

[Rod]

A good few folks have posted working examples. If it isn’t working for you you have the wrong page or have not delved deep enough into the page. Try a different source. Many many servers publish stock prices.

[tenochtitlanuk]

I ran

page$=httpget$("https://finance.yahoo.com/quote/VIAC?p=VIAC&.tsrc=fin-srch-v1")

open "page.txt" for output as #fOut
   #fOut page$;
close #fOut


The saved text included headings and data. Too lazy to write any parsing code, BUT.. searching in a text editor for as an example 'Previous Close' and find this section..
Previous Close</span></td><td class="Ta(end) Fw(600) Lh(14px)" data-test="PREV_CLOSE-value" data-reactid="15"><span class="Trsdu(0.3s) " data-reactid="16">17.26</spa....
The data associated with that heading is correctly found.
I helped another LB poster do something similar downloading from a webpage and parsing out data- in his case to generate barcodes and printing them..


It's MESSY to search like this in LB, but straightforward. Only worth while if you have only one or a few target pages.

[svajoklis]

Well I'll be damned, it seems it does contain the data, should check myself the next time

The "PREV_CLOSE-value" attribute value could be a good anchor to skip forward from since it only seems to occur once in a page and it is closer to the required value. You'd have to skip over the opening <span /> element though. I'd still lean towards the API endpoint I posted - a WHOLE LOT easier to parse than HTML/XML!

query1.finance.yahoo.com/v8/finance/chart/VIAC?region=US&lang=en-US&includePrePost=false&interval=2m&range=1d&corsDomain=finance.yahoo.com&.tsrc=finance

Seems to be right on the money, since it contains an easy '"previousClose":17.26' to parse.

Other data is easy to scrape out too and you could replace "VIAC" in the URL to scrape other symbols.

[svajoklis]

' configuration options
' symbol to look up and field to get
' interesting fields: "symbol", "currency", "previousClose", "regularMarketPrice"
' view https://query1.finance.yahoo.com/v8/finance/chart/VIAC?region=US&lang=en-US to see other examples
symbol$ = "VIAC"
field$ = "previousClose"

' download page
url$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US"
page$ = httpget$(url$)

' print found value
print getNumericalFieldFromJSON(field$, page$)

end

function getNumericalFieldFromJSON (field$, json$)
   key$ = chr$(34) + field$ + chr$(34) + ":"
   keyPosition = instr(json$, key$)
   if keyPosition = 0 then
       getNumericalFieldFromJSON = -1
   else
       valuePosition = keyPosition + len(key$)

       valueString$ = ""
       i = valuePosition
       while 1
           character$ = mid$(json$, i, 1)
           if instr("0123456789.", character$) > 0 then
               valueString$ = valueString$ + character$
               i = i + 1
           else
               exit while
           end if
       wend

       value = val(valueString$)
       getNumericalFieldFromJSON = value
   end if
end function

ez

[Brandon Parker]

I personally would stay far away from "While 1 ... Wend" which relies on a specific condition to break the While Loop.

This is how I would do it given a first little go at it.

symbol$ = "VIAC"
field$ = "previousClose"

URL$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US"
webPage$ = HTTPGet$(URL$)

Print getFieldFromJSON(field$, webPage$, ",";chr$(34))
End

Function getFieldFromJSON(field$, json$, endDelim$)
   key$ = chr$(34) + field$ + chr$(34) + ":"
   keyPosition = Instr(json$, key$)
   If Not(keyPosition) Then getFieldFromJSON = -1 : Exit Function
   getFieldFromJSON = Val(UpTo$(Mid$(json$, (keyPosition + Len(key$))), endDelim$))
End Function


{:0)

Brandon Parker

[svajoklis]

Your solution wouldn't work if the key in question was the last one, since then the delimiter would be a } instead of ," for next key.

' configuration options
' symbol to look up and field to get
' interesting fields: "symbol", "currency", "previousClose", "regularMarketPrice"
' view https://query1.finance.yahoo.com/v8/finance/chart/VIAC?region=US&lang=en-US to see other examples
symbol$ = "VIAC"
field$ = "previousClose"

' download page
url$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US"
page$=httpget$(url$)

' print found value
print getNumericalFieldFromJSON(field$, page$)

end

function getNumericalFieldFromJSON (field$, json$)
   key$ = chr$(34) + field$ + chr$(34) + ":"
   keyPosition = instr(json$, key$)

   ' key not found in file
   if not(keyPosition) then
       getNumericalFieldFromJSON = -1
       exit function
   end if

   ' find value position and string containing value and everything until end of file
   valuePosition = keyPosition + len(key$)
   valueString$ = mid$(json$, valuePosition)

   ' try to find ," delimiter
   valueEndPosition = instr(valueString$, "," + chr$(34))
   ' try to find } delimiter
   if not(valueEndPosition) then
       valueEndPosition = instr(valueString$, "}")
   end if
   ' value end not found
   if not(valueEndPosition) then
       getNumericalFieldFromJSON = -1
       exit function
   end if
   valueString$ = mid$(valueString$, 0, valueEndPosition)

   value = val(valueString$)
   getNumericalFieldFromJSON = value
end function

Sometimes the "while true" really works, it's just that you have to make sure the end condition will happen eventually, which is the hard part (:

Other than that - pretty slick solution, though the nesting is a bit much for readability S:

[Brandon Parker]

In that case, I would change the function to return a string and allow the user to deal with it from there. This will allow for a single function to grab any data for a field. Notice the extra UpTo$() to remove the "}}" if the last item, "error" in this case, is pulled. I'm no JSON expert, nor do I really want to be, so this function could fail in some other way.  As for the nesting, that's nothing....you should check out my Rosetta Code Quine and Narcissist examples from years ago.

Based on the current data, this function allows you to grab any of the fields...at least from what I see...

symbol$ = "VIAC"
field$ = "timestamp"

URL$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US"
webPage$ = HTTPGet$(URL$)

Print getFieldFromJSON$(field$, webPage$, ",";chr$(34))
End

Function getFieldFromJSON$(field$, json$, endDelim$)
   key$ = chr$(34) + field$ + chr$(34) + ":"
   keyPosition = Instr(json$, key$)
   If Not(keyPosition) Then getFieldFromJSON$ = "ERROR" : Exit Function
   getFieldFromJSON$ = UpTo$(UpTo$(Mid$(json$, (keyPosition + Len(key$))), endDelim$), "}")
End Function


{:0)

Brandon Parker

[svajoklis]

Well yeah, there's that with the quine and narcissist, but then again - this is the "beginner" subforum, so at least there is enough different material to learn from (:

Now I'm starting to think it isn't really possible to really parse out a JSON fully, since it would require to set up variables dynamically. A scanner would work, it would go through the whole JSON string looking for a specific key and the return it's value, though the value must then be either a string or a number. It would work something like scanJSON(json$, path$) or scanJSON$(json$, path$) where path$ is let's say "data.symbols[0].value".

[Brandon Parker]

Anything is possible! The aim is to always be improving one's understanding of a programming language and push the boundaries of what is "possible" in that language.

I typically store pretty much everything as strings in larger than simple test programs. Since HTTPGet$() returns a string, it is fairly easy to parse for any field if you know the data layout. If you know which fields need to be treated as numbers then all you have to do is wrap the function call to getFieldFromJSON$() inside of Val() and store it to a numerical variable. The added benefit of treating everything as a string from the function is that you can return a set of numbers and then parse them in a different Sub/Function.

Everyone has their own way of doing things, and we can all learn at least a little bit from everyone who positively contributes.


{:0)

Brandon Parker

[svajoklis]

Yeah, but my point is that every time you need to access a value in the JSON you have to scan for it from a string instead of single-passing it and then accessing it from a data structure. More often than not you want more than a single value from a document like this.

JSON supports dictionaries and arrays, so the scanner should be able to delve into

{ "buses": [{ "name": "Michigan", "license": "A1" }, { "name": "New York", "license": "A2" }] }

Let's say I want to extract all bus names from this. Since arrays are either global or scoped to a function, and you can't even return an array (only integers or strings), how would you specify what array you want to return the bus names to? It would have to write to a global array that is hardcoded into the function or I'm missing something

I know this is a bit off-topic in a way, but at the same time we are getting into JSON parsing here.