Browsed by
Author: Ian Kerins

Solution Architecture Part 5: Designing A Well-Optimised Web Scraping Solution

In the fifth and final post of this solution architecture series, we will share with you how we architect a web scraping solution, all the core components of a well-optimized solution, and the resources required to execute it.

Solution Architecture Part 4: Accessing The Technical Feasibility of Your Web Scraping Project

In the fourth post of this solution architecture series, we will share with you our step-by-step process for evaluating the technical feasibility of a web scraping project.

Visual Web Scraping Tools: What to Do When They Are No Longer Fit For Purpose?

Visual web scraping tools are great. They allow people with little to no technical know-how to extract data from websites with only a couple hours of upskilling, making them great for simple lead generation, market intelligence and competitor monitoring projects. Removing countless hours of manual entry work for sales and marketing teams, researchers, and business intelligence team in the...

From The Creators Of Scrapy: Artificial Intelligence Data Extraction API

To accurately extract data from a web page, developers usually need to develop custom code for each website. This is manageable and recommended for tens or hundreds of websites and where data quality is of the utmost importance, but if you need to extract data from thousands of sites, or rapidly extract data from sites that are not yet covered by pre-existing code, this is often an...

Scrapinghub’s New AI Powered Developer Data Extraction API for E-Commerce & Article Extraction

Today, we’re delighted to announce the launch of the beta program for Scrapinghub’s new AI powered developer data extraction API for automated product and article extraction.

After much development and refinement with alpha users, our team have refined this machine learning technology to the point that data extraction engine is capable of automatically identifying common items on product and...

Solution Architecture Part 2: How to Define The Scope of Your Web Scraping Project

In this the second post in our solution architecture series, we will share with you our step-by-step process for data extraction requirement gathering.

How to Architect a Web Scraping Solution: The Step-by-Step Guide

For many people (especially non-techies), trying to architect a web scraping solution for their needs and estimate the resources required to develop it, can be a tricky process.

Oftentimes, this is their first web scraping project and as a result have little reference experience to draw upon when investigating the feasibility of a data extraction project.

In this series of articles we’re going...

St Patrick’s Day Special: Finding Dublin’s Best Pint of Guinness With Web Scraping

St Patrick’s Day Special: Finding Dublin’s Best Pint of Guinness With Web Scraping

At Scrapinghub we are known for our ability to help companies make mission critical business decisions through the use of web scraped data.

But for anyone who enjoys a freshly poured pint of stout, there is one mission critical question that creates a debate like no other…

“Who serves the best pint of Guinness?”

Spidermon: Scrapinghub’s Secret Sauce To Our Data Quality & Reliability Guarantee

If you know anything about Scrapinghub, you know that we are obsessed with data quality and data reliability.

Outside of building some of the most powerful web scraping tools in the world, we also specialise in helping companies extract the data they need for their mission-critical business requirements. Most notably companies who:

  • Rely on web data to make critical business decisions, or;
  • ...

Proxy Management: Should I Build My Proxy Infrastructure In-House Or Use AN Off-The-Shelf Proxy Solution?

Proxy management is the thorn in the side of most web scrapers. Without a robust and fully featured proxy infrastructure, you will often experience constant reliability issues and hours spent putting out proxy fires.

A situation no web scraping professional wants to deal with. Us web scrapers are interested in extracting and using web data, not managing proxies.

In this article, we’re going to...

Sign up now

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Follow Us

Learn More

Recent Posts