Browsed by
Category: releases

Deploy your Scrapy Spiders from GitHub

Up until now, your deployment process using Scrapy Cloud has probably been something like this: code and test your spiders locally, commit and push your changes to a GitHub repository, and finally deploy them to Scrapy Cloud using shub deploy. However, having the development and the deployment processes in isolated steps might bring you some issues, such as unversioned and outdated code running...

Improved Frontera: Web Crawling at Scale with Python 3 Support

Python is our go-to language of choice and Python 2 is losing traction. In order to survive, older programs need to be Python 3 compatible.

Introducing Scrapy Cloud with Python 3 Support

It’s the end of an era. Python 2 is on its way out with only a few security and bug fixes forthcoming from now until its official retirement in 2020. Given this withdrawal of support and the fact that Python 3 has snazzier features, we are thrilled to announce that Scrapy Cloud now officially supports Python 3.

Introducing Portia2Code: Portia Projects into Scrapy Spiders

We’re thrilled to announce the release of our latest tool, Portia2Code!

Introducing the Datasets Catalog

Folks using Portia and Scrapy are engaged in a variety of fascinating web crawling projects, so we wanted to provide you with a way to share your data extraction prowess with the world.

Introducing the Crawlera Dashboard

We’ve been rolling out a lot of updates, upgrades, and new features lately, and we’re continuing this trend by announcing the very first Crawlera Dashboard!

Data Extraction with Scrapy and Python 3

Scrapy 1.1 Release with Official Python 3 Support

Fasten your seat belts, ladies and gentlemen: Scrapy 1.1 with Python 3 support is officially out! After a couple months of hard work and four release candidates, this is the first official Scrapy release to support Python 3.

Introducing Scrapy Cloud 2.0

Scrapy Cloud has been with Scrapinghub since the beginning, but we decided some spring cleaning was in order. To that end, we’re proud to announce Scrapy Cloud 2.0!

Machine Learning with Web Scraping: New MonkeyLearn Addon

Say Hello to the MonkeyLearn Addon

We deal in data. Vast amounts of it. But while we’ve been traditionally involved in providing you with the data that you need, we are now taking it a step further by helping you analyze it as well.

Splash 2.0 Is Here with Qt 5 and Python 3

We’re pleased to announce that Splash 2.0 is officially live after many months of hard work.

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Learn More

Recent Posts