|
Post by mikeschn on May 1, 2020 13:46:49 GMT -5
Is there an easy way to grab a stock price using LB?
Or do I have to use httpget to grab the entire page and then search that page for the price?
Suggestions?
Thanks,
Mike...
|
|
ntech
New Member
Posts: 49
|
Post by ntech on May 1, 2020 16:44:29 GMT -5
Hmm, is there an API you are looking at? The most straightforward way would be to grab the page and extract the data you want from there.
|
|
|
Post by metro on May 1, 2020 16:49:28 GMT -5
Hmm, is there an API you are looking at? The most straightforward way would be to grab the page and extract the data you want from there.
I use the "pain in the parse" method down here on the ASX
If your exchange displays prices in a table somewhere you may find some of this useful (or not)..the data here is for options on the ASX
asxcode$="BHP" selmonth$="Jun" year$="2020" global asxcode$,selmonth$,year$ call Table2CSV asxcode$ WAIT ' -------------------------------------- SUB Table2CSV asxcode$ ' change this to your web page 'webPage$ = "https://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode="+asxcode$+"&expiryDate=&optionType=B" webPage$="http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=" + asxcode$ + "&expiryDate="+ selmonth$ + "+" +year$+"&optionType=B" ' change this to your output csv file name print webPage$ fileName$ = asxcode$
htmlData$ = httpget$(webPage$)
lowData$ = lower$(htmlData$) open DefaultDir$+"\"+"DATA\"+ fileName$+".txt" for output as #f
[nxtTbl] tblBeg = instr(lowData$,"<table",tblEnd) if tblBeg = 0 then goto [endTbl] tblEnd = instr(lowData$,"</table>",tblBeg) tblData$ = mid$(htmlData$,tblBeg,(tblEnd - tblBeg) + 8) lineBeg = 0 lineEnd = instr(tblData$,"</tr>",lineBeg)
while lineEnd > 0 lineData$ = mid$(tblData$,lineBeg - 1,(lineEnd - lineBeg) + 1) lineData$ = strip$(lineData$) lineData$ = strRep$(lineData$,"</td>",",") lineData$ = strRep$(lineData$,"</th>",",") lineData$ = tags$(lineData$,"") lineData$ = strRep$(lineData$," "," ") lineData$ = strRep$(lineData$,"&","and")
print "";left$(lineData$,len(lineData$) -2) print #f, "";left$(lineData$,len(lineData$) -2)
lineBeg = lineEnd + 1 lineEnd = instr(tblData$,"</tr>",lineBeg) wend print "----------------- end <table> --------------------------"
goto [nxtTbl]
[endTbl] close #f END SUB
' ----------------------------------------- ' strip junk ' ----------------------------------------- FUNCTION strip$(str$) strip$ = "" for i = 1 to len(str$) a$ = MID$(str$,i,1) a = ASC(a$) if a > 31 then if a < 127 then if a$ <> "'" then if a$ <> "" then strip$ = strip$ + a$ end if end if end if end if next i END FUNCTION
' -------------------------------- ' string replace rep str with ' -------------------------------- FUNCTION strRep$(str$,rep$,with$) ln = len(rep$) ln1 = ln - 1 i = 1 while i <= len(str$) if mid$(str$,i,ln) = rep$ then strRep$ = strRep$ + with$ i = i + ln1 else strRep$ = strRep$ + mid$(str$,i,1) end if i = i + 1 WEND END FUNCTION
' -------------------------------- ' Replace HTML <tags> with$ ' -------------------------------- FUNCTION tags$(tag$,with$) i = instr(tag$,"<") j = instr(tag$,">",i) WHILE j > i tag$ = left$(tag$,i - 1) + mid$(tag$,j + 1) i = instr(tag$,"<") j = instr(tag$,">") WEND tags$ = tag$ END FUNCTION
MyStocks.zip (253.53 KB)_ There maybe something useful in this I use it to find stocks that have moved the most (either up or down) intra day
|
|
|
Post by mikeschn on May 2, 2020 12:06:12 GMT -5
No luck yet! I can't even grab the page to scrape! page$=httpget$("https://finance.yahoo.com/quote/VIAC?p=VIAC&.tsrc=fin-srch-v1") for example, just yields a bunch of html code without the data! Mike...
|
|
|
Post by svajoklis on May 2, 2020 13:02:51 GMT -5
If you can't find data itself in the page, then it is probably loaded in dynamically with JavaScript. PhantomJS is a headless browser (a browser that runs all the code, just doesn't show a window) that you can control programatically, it would be able to download the page, run the required JS code to load the data and then after it completely loads you would be able to scrape off the data. I don't think Liberty BASIC has anything like that though... A quick look around shows that page GETs some XHR requests, like this query1.finance.yahoo.com/v7/finance/spark?symbols=%5EDJI&range=1d&interval=5m&indicators=close&includeTimestamps=false&includePrePost=false&corsDomain=finance.yahoo.com&.tsrc=financeIf I didn't go the PhantomJS route, then I'd look into those (visible in the Inspect -> Network tab in Chrome, similar tools are available in Firefox and other browsers) and try to decode the URL so that Liberty could retrieve that. It gets back with a JSON file, which, again I don't think Liberty can tackle out of the box, so it would be a case of trying to parse it same as HTML - with plain text functions.
|
|
|
Post by Rod on May 2, 2020 13:49:37 GMT -5
A good few folks have posted working examples. If it isn’t working for you you have the wrong page or have not delved deep enough into the page. Try a different source. Many many servers publish stock prices.
|
|
|
Post by tenochtitlanuk on May 2, 2020 14:08:16 GMT -5
I ran
page$=httpget$("https://finance.yahoo.com/quote/VIAC?p=VIAC&.tsrc=fin-srch-v1")
open "page.txt" for output as #fOut #fOut page$; close #fOut
The saved text included headings and data. Too lazy to write any parsing code, BUT.. searching in a text editor for as an example 'Previous Close' and find this section.. Previous Close</span></td><td class="Ta(end) Fw(600) Lh(14px)" data-test="PREV_CLOSE-value" data-reactid="15"><span class="Trsdu(0.3s) " data-reactid="16">17.26</spa.... The data associated with that heading is correctly found. I helped another LB poster do something similar downloading from a webpage and parsing out data- in his case to generate barcodes and printing them..
It's MESSY to search like this in LB, but straightforward. Only worth while if you have only one or a few target pages.
|
|
|
Post by svajoklis on May 2, 2020 14:21:36 GMT -5
|
|
|
Post by svajoklis on May 3, 2020 2:58:26 GMT -5
' configuration options ' symbol to look up and field to get ' interesting fields: "symbol", "currency", "previousClose", "regularMarketPrice" ' view https://query1.finance.yahoo.com/v8/finance/chart/VIAC?region=US&lang=en-US to see other examples symbol$ = "VIAC" field$ = "previousClose"
' download page url$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US" page$ = httpget$(url$)
' print found value print getNumericalFieldFromJSON(field$, page$)
end
function getNumericalFieldFromJSON (field$, json$) key$ = chr$(34) + field$ + chr$(34) + ":" keyPosition = instr(json$, key$) if keyPosition = 0 then getNumericalFieldFromJSON = -1 else valuePosition = keyPosition + len(key$)
valueString$ = "" i = valuePosition while 1 character$ = mid$(json$, i, 1) if instr("0123456789.", character$) > 0 then valueString$ = valueString$ + character$ i = i + 1 else exit while end if wend
value = val(valueString$) getNumericalFieldFromJSON = value end if end function
ez
|
|
|
Post by Brandon Parker on May 3, 2020 9:12:16 GMT -5
I personally would stay far away from "While 1 ... Wend" which relies on a specific condition to break the While Loop.
This is how I would do it given a first little go at it.
symbol$ = "VIAC" field$ = "previousClose"
URL$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US" webPage$ = HTTPGet$(URL$)
Print getFieldFromJSON(field$, webPage$, ",";chr$(34)) End
Function getFieldFromJSON(field$, json$, endDelim$) key$ = chr$(34) + field$ + chr$(34) + ":" keyPosition = Instr(json$, key$) If Not(keyPosition) Then getFieldFromJSON = -1 : Exit Function getFieldFromJSON = Val(UpTo$(Mid$(json$, (keyPosition + Len(key$))), endDelim$)) End Function
{:0)
Brandon Parker
|
|
|
Post by svajoklis on May 3, 2020 10:01:33 GMT -5
Your solution wouldn't work if the key in question was the last one, since then the delimiter would be a } instead of ," for next key.
' configuration options ' symbol to look up and field to get ' interesting fields: "symbol", "currency", "previousClose", "regularMarketPrice" ' view https://query1.finance.yahoo.com/v8/finance/chart/VIAC?region=US&lang=en-US to see other examples symbol$ = "VIAC" field$ = "previousClose"
' download page url$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US" page$=httpget$(url$)
' print found value print getNumericalFieldFromJSON(field$, page$)
end
function getNumericalFieldFromJSON (field$, json$) key$ = chr$(34) + field$ + chr$(34) + ":" keyPosition = instr(json$, key$)
' key not found in file if not(keyPosition) then getNumericalFieldFromJSON = -1 exit function end if
' find value position and string containing value and everything until end of file valuePosition = keyPosition + len(key$) valueString$ = mid$(json$, valuePosition)
' try to find ," delimiter valueEndPosition = instr(valueString$, "," + chr$(34)) ' try to find } delimiter if not(valueEndPosition) then valueEndPosition = instr(valueString$, "}") end if ' value end not found if not(valueEndPosition) then getNumericalFieldFromJSON = -1 exit function end if valueString$ = mid$(valueString$, 0, valueEndPosition)
value = val(valueString$) getNumericalFieldFromJSON = value end function
Sometimes the "while true" really works, it's just that you have to make sure the end condition will happen eventually, which is the hard part (:
Other than that - pretty slick solution, though the nesting is a bit much for readability S:
|
|
|
Post by Brandon Parker on May 3, 2020 11:14:21 GMT -5
In that case, I would change the function to return a string and allow the user to deal with it from there. This will allow for a single function to grab any data for a field. Notice the extra UpTo$() to remove the "}}" if the last item, "error" in this case, is pulled. I'm no JSON expert, nor do I really want to be, so this function could fail in some other way. As for the nesting, that's nothing....you should check out my Rosetta Code Quine and Narcissist examples from years ago. Based on the current data, this function allows you to grab any of the fields...at least from what I see... symbol$ = "VIAC" field$ = "timestamp"
URL$ = "https://query1.finance.yahoo.com/v8/finance/chart/" + symbol$ + "?region=US&lang=en-US" webPage$ = HTTPGet$(URL$)
Print getFieldFromJSON$(field$, webPage$, ",";chr$(34)) End
Function getFieldFromJSON$(field$, json$, endDelim$) key$ = chr$(34) + field$ + chr$(34) + ":" keyPosition = Instr(json$, key$) If Not(keyPosition) Then getFieldFromJSON$ = "ERROR" : Exit Function getFieldFromJSON$ = UpTo$(UpTo$(Mid$(json$, (keyPosition + Len(key$))), endDelim$), "}") End Function {:0) Brandon Parker
|
|
|
Post by svajoklis on May 4, 2020 3:06:23 GMT -5
Well yeah, there's that with the quine and narcissist, but then again - this is the "beginner" subforum, so at least there is enough different material to learn from (:
Now I'm starting to think it isn't really possible to really parse out a JSON fully, since it would require to set up variables dynamically. A scanner would work, it would go through the whole JSON string looking for a specific key and the return it's value, though the value must then be either a string or a number. It would work something like scanJSON(json$, path$) or scanJSON$(json$, path$) where path$ is let's say "data.symbols[0].value".
|
|
|
Post by Brandon Parker on May 4, 2020 8:15:20 GMT -5
Anything is possible! The aim is to always be improving one's understanding of a programming language and push the boundaries of what is "possible" in that language.
I typically store pretty much everything as strings in larger than simple test programs. Since HTTPGet$() returns a string, it is fairly easy to parse for any field if you know the data layout. If you know which fields need to be treated as numbers then all you have to do is wrap the function call to getFieldFromJSON$() inside of Val() and store it to a numerical variable. The added benefit of treating everything as a string from the function is that you can return a set of numbers and then parse them in a different Sub/Function.
Everyone has their own way of doing things, and we can all learn at least a little bit from everyone who positively contributes.
{:0)
Brandon Parker
|
|
|
Post by svajoklis on May 4, 2020 9:21:50 GMT -5
Yeah, but my point is that every time you need to access a value in the JSON you have to scan for it from a string instead of single-passing it and then accessing it from a data structure. More often than not you want more than a single value from a document like this. JSON supports dictionaries and arrays, so the scanner should be able to delve into { "buses": [{ "name": "Michigan", "license": "A1" }, { "name": "New York", "license": "A2" }] } Let's say I want to extract all bus names from this. Since arrays are either global or scoped to a function, and you can't even return an array (only integers or strings), how would you specify what array you want to return the bus names to? It would have to write to a global array that is hardcoded into the function or I'm missing something I know this is a bit off-topic in a way, but at the same time we are getting into JSON parsing here.
|
|