Custom crawling & News API: designing a web scraping solution

Web scraping projects usually involve data extraction from many websites. The standard approach to tackle this problem is to write some code to navigate and extract the data from each website. However, this approach may not scale so nicely in the long-term, requiring maintenance effort for each website; it also doesn’t scale in the short-term, when we need to start the extraction process in a...

Vehicle API (Beta): Extract Automotive Data at Scale

Today we are delighted to launch a Beta of our newest data extraction API: AutoExtract Vehicle API. With this API you can collect structured data from web pages that contain automotive data such as classified or dealership sites. Using our API, you can get your data without writing site-specific code. If you need automotive/vehicle data, sign up now for a beta version of our Vehicle API.

A Practical Guide to Web Data Extraction QA Part II: Common validation pitfalls

In case you missed the first part of this series, where we went through data validation techniques, you can read it now: A Practical Guide To Web Data  Extraction QA Part I: Validation Techniques