Browsed by
Category: releases

Introducing Data Reviews

One of the things that takes more time when building a spider is reviewing the scraped data and making sure it conforms to the requirements and expectations of your client or team. This process is so time consuming that, in many cases, it ends up taking more time than writing the spider code itself, depending on how well the requirements are written. To make this process more efficient we have...

Announcing Portia, the Open Source Visual Web Scraper!

Note: Portia is no longer available for new users. It has been disabled for all the new organisations from August 20, 2018 onward.

We’re proud to announce the developer release of Portia, our new open source visual scraping tool based on Scrapy. Check out this video:

Introducing Dash

We're excited to introduce Dash, a major update to our scraping platform.

Introducing Crawlera, a Smart Page Downloader

We are proud to introduce Crawlera, a smart web downloader designed specifically for web crawling.

How to Fill Login Forms Automatically

We often have to write spiders that need to login to sites, in order to scrape data from them. Our customers provide us with the site, username and password, and we do the rest.

Spiders activity graphs

Today we are introducing a new feature called Spider activity graphs. These allow you to visualize quickly how your spiders are working, and it's a very useful tool for busy projects to find out which spiders are not working as expected.

Scrapy 0.14 released

After 10 months of work, and many changes, we are pleased to announce the release of Scrapy 0.14.

Introducing w3lib and scrapely

In an effort to make Scrapy code smaller and more reusable, we’ve been working on splitting the Scrapy codebase into two different modules:

Scrapy 0.12 released

Hello everyone, we’re pleased to announce the release of Scrapy 0.12!

Sign up now

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Follow Us

Learn More

Recent Posts