Price scraping is something that you need to do if you want to extract pricing data from websites. It might look easy and just a minor technical detail that needs to be handled but in reality, if you don’t know the best way to get those price values from the HTMLs, it can be a headache over time.
Sending HTTP requests in Python is not necessarily easy. We have built-in modules like urllib, urllib2 to deal with HTTP requests. Also, we have third-party tools like Requests. Many developers use Requests because it is high level and designed to make it extremely easy to send HTTP requests.
Python is our go-to language of choice and Python 2 is losing traction. In order to survive, older programs need to be Python 3 compatible.
It’s the end of an era. Python 2 is on its way out with only a few security and bug fixes forthcoming from now until its official retirement in 2020. Given this withdrawal of support and the fact that Python 3 has snazzier features, we are thrilled to announce that Scrapy Cloud now officially supports Python 3.