A Career in Remote Working

This year I have reached a major milestone in my life, which is getting my bachelor's degree in mathematics. When I made the decision to go back to college, it was solely because my experience working at Scrapinghub, I figured out that having a math background would be a great foundation for getting into ML-related stuff.

Frontera: The Brain Behind the Crawls

At Scrapinghub we're always building and running large crawls–last year we had 11 billion requests made on Scrapy Cloud alone. Crawling millions of pages from the internet requires more sophistication than getting a few contacts of a list, as we need to make sure that we get reliable data, up to date lists of item pages and are able to optimise our crawl as much as possible.

Scrape Data Visually with Portia and Scrapy Cloud

It’s been several months since we first integrated Portia into our Scrapy Cloud platform, and last week we officially began to phase out Autoscraping in favor of Portia.

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Learn More

Recent Posts