|
Post by mpranger on Jan 29, 2021 11:28:42 GMT -5
I see how Python and R are touted as being good for scraping data from websites. Is there any reason it might be more difficult or impossible with Liberty Basic?
|
|
|
Post by tsh73 on Jan 29, 2021 11:47:56 GMT -5
https support ready-made libraries for web-scraping
|
|
|
Post by mpranger on Jan 30, 2021 4:15:02 GMT -5
https support ready-made libraries for web-scraping
Thank you. But I don't understand this. Where are these libraries to be found?
|
|
Tasp
Full Member
Posts: 215
|
Post by Tasp on Jan 30, 2021 5:08:56 GMT -5
The libraries that tsh73 is referring to are for python not LB. Python will have libraries available already created. You'll have to search for them on Google etc.
LB supports HTTPGET$, more info is in the helpfile.
|
|
|
Post by tsh73 on Jan 30, 2021 6:10:43 GMT -5
It is absence of these things in LB that gives web-scraping difficult with LB. (I actually might be wrong about absence of https support in LB, I do not use these things From reading this forum I got an idea that it is problematic - but I cannot point a finger on it) )
If you google "web scraping python" you will see lot's of examples. This forum show some examples of HTTPGET, do a search.
|
|
|
Post by Brandon Parker on Feb 1, 2021 15:51:26 GMT -5
What do you plan on doing with your "scraping" activities?
{:0)
Brandon Parker
|
|
|
Post by mpranger on Feb 2, 2021 7:24:05 GMT -5
What do you plan on doing with your "scraping" activities? {:0) Brandon Parker I want to scrape statistics from nba.com and other sites. I can scrape it by clicking and dragging
over the area but it would be nice if LB could do it.
Excel can automate the process.And there are websites which offer the service.
|
|
|
Post by tenochtitlanuk on Feb 2, 2021 18:21:28 GMT -5
That's a complex page on NBA, constructed without the data being in the page code, but fetched from a database.
Webpages are increasingly coded to customise them to you and convey data to you visually, but not to make it easy for you to get out what you want!
I used to run a LB program daily to download a synoptic weather chart. Now the equivalent page is built by tiling separate sections and MUCH harder to grab as a single image.
For fun I've written LB code that opens a texteditor. You MANUALLY click/drag through the NBA page's data section, and paste into the LB window, and it strips out the bits you want/don't want, and saves as a csv file. Seems to work fine...
LB could open that in a spreadsheet app of your choice.
Will put up code and how-to if it's any use to you. It gave me an hour or two of fun anyway!
|
|
|
Post by tenochtitlanuk on Feb 3, 2021 11:45:20 GMT -5
|
|