Browsed by
Category: scrapinghub

How to use Crawlera with Scrapy

Crawlera is a proxy service, specifically designed for web scraping. In this article, you are going to learn how to use Crawlera inside your Scrapy spider.

Web Scraping Questions & Answers Part II

In our article last week, we answered some of the best questions we got during Extract Summit. In today’s post we share with you the second part of this series. We are covering questions on Web Scraping Infrastructure and How Machine Learning can be used in Web Scraping.

The First-Ever Web Data Extraction Summit!

The range of use cases for web data extraction is rapidly increasing and with it the necessary investment. Plus the number of websites continues to grow rapidly and is expected to exceed 2 billion by 2020.

Solution Architecture Part 4: Accessing The Technical Feasibility of Your Web Scraping Project

In the fourth post of this solution architecture series, we will share with you our step-by-step process for evaluating the technical feasibility of a web scraping project.

Spidermon: Scrapinghub’s Secret Sauce To Our Data Quality & Reliability Guarantee

If you know anything about Scrapinghub, you know that we are obsessed with data quality and data reliability.

Outside of building some of the most powerful web scraping tools in the world, we also specialise in helping companies extract the data they need for their mission-critical business requirements. Most notably companies who:

  • Rely on web data to make critical business decisions, or;
  • ...

Proxy Management: Should I Build My Proxy Infrastructure In-House Or Use AN Off-The-Shelf Proxy Solution?

Proxy management is the thorn in the side of most web scrapers. Without a robust and fully featured proxy infrastructure, you will often experience constant reliability issues and hours spent putting out proxy fires.

A situation no web scraping professional wants to deal with. Us web scrapers are interested in extracting and using web data, not managing proxies.

In this article, we’re going to...

Looking Back at 2018

What a year 2018 has been for Scrapinghub!!

It’s hard to know where to start…

This year has seen tremendous growth at Scrapinghub, setting us up to have a great 2019.

Here are some of the highlights of 2018…

Shubber GetTogether 2018

It’s hard to believe our annual Shubber GetTogether is already over.

A Sneak Peek Inside What Hedge Funds Think of Alternative Financial Data

Unbeknownst to many, there is a data revolution happening in finance.

Want to Predict Fitbit’s Quarterly Revenue? Eagle Alpha Did It Using Web Scraped Product Data

Throughout the history of the financial markets information has been power. The trader with access to the most accurate information can quickly gain an edge over the market.

Sign up now

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Follow Us

Learn More

Recent Posts