Requests-HTML - Checking out a new HTML parsing library for Python

why use requests-html when I can use requests?

hi, i cant find a way to select an element by its attribute. Can you help please?

Please make a tutorial on how to make asynchronous requests using grequest

Thank you so much!, I got the answer based on your guide. nice helmet ⛑ you got back there

Strange, I have installed requests_html but when I import it in a Python script in Python 2.x or 3.7, I get: ModuleNotFoundError: No module named 'requests_html'

Hello!
Recently I am trying to parse some webpages with Requests-html asynchronously. Theoretically this can be done by working with AsyncHTMLSession. However, I am unable to get result with it most of the time (I also use arender, the attempts to parse the webpages fails due to different reasons - most probably timeouts). Maybe it's just the poor internet connection, but I'd be really grateful if you uploaded a video or help me with this.

for {basically anything you want to name it} in r.html:
you get urls

Sentdex! Can you show scraping from a page with a "show more" button, that loads more of the page in JavaScript ?

I was thinking why not using r.html.find_all('td', {'class':re.compile(r'class name regex')}). turn out that's beautifulsoup funtion. But yeah, having the render() to run js is nice!

You could have used a css selector when parsing the yahoo finance page

Please make a video building a webcrawler, would be very insightful!

you forgot to say what it is good for

I love you man more than Mo Salah

This is cool! And it gave me the idea of a series of videos about how to create a python package

Can I get some help how to install "requests-html" package to be run globally, for example, through Sublime Text?
I am using Conda on Windows 10.

I have been trying to do that, but as I understand so far, it runs only in virtual environment that cannot be used by Sublime? Correct me if I am wrong.

what the extension, who print the result down?

Is it useful for scraping website with ajax ??

to find td‘s or other elements that pertain to multiple classes you just would have had to put dots in between. Read up on css selectors, jquery also uses them, pretty standard nowadays and less headache than xpaths ;)

wow, when did you start using sublime text? i havent seen your videos in a while but back then they were all done in notepad or idle! :)

I think the // for retry in range(100): // part is what is allowing the script to continue after raising the error. From their doc: "The simplest use case is retrying a flaky function whenever an Exception occurs until a value is returned." So this would allow the exception to be printed, yet the script to continue I believe. Great content man, thanks for all of the awesome videos :).

How well does it work on snippets or badly formed HTML?

Thanks for posting this. I've used BS4 and another module to do the JavaScript (render the page) on many projects, it's nice to have it in a concise package.

Btw, I think the pagination on HackerNews failed because it looks for one of three (by default) "next" labels. "next", "more", "older" (DEFAULT_NEXT_SYMBOLS). The CNBC link has "more" in it.

Thank u sentdex you are leading me to the real world from africa

The function couldn't clean up user data because these files were locked by chromium process.

From Turkey👋👋👋

Have you done any tutorials on Dask?

I think for paging one can use threading with a except statement and hopefully it will work.

Make another video building a crawler using it. Nice video!

Best way to remove error
-> comment out raise statement. 😁😂

Should i change my windows to 64-bit version ?

awesome video as always, quick question tho why don't you use linux ?

Great tool for mixing with bs4 to build a robust crawler

Can you make a video about GANS, and some real time appplications of it

Great tool to build crawlers!

Great work! Thank you.... I would be very greatful if you check my privious comments. I need your help please

man, i'm mixing that HTML parsing sauce with my beautiful soup right now

I like this type of video. You should do like a monthly video of new module so people can be aware. This will be very useful people that learn python.

1en