Visual Web Scraping Tools: What to Do When They Are No Longer Fit For Purpose?

Visual web scraping tools are great. They allow people with little to no technical know-how to extract data from websites with only a couple hours of upskilling, making them great for simple lead generation, market intelligence and competitor monitoring projects. Removing countless hours of manual entry work for sales and marketing teams, researchers, and business intelligence team in the...

Solution Architecture Part 3: Conducting a Web Scraping Legal Review

In this the third post in our solution architecture series, we will share with you our step-by-step process for conducting a legal review of every web scraping project we work on.

ScrapyRT: Turn Websites Into Real-Time APIs

If you’ve been using Scrapy for any period of time, you know the capabilities a well-designed Scrapy spider can give you.

Web Data Analysis: Exposing NFL Player Salaries With Python

Football. From throwing a pigskin with your dad, to crunching numbers to determine the probability of your favorite team winning the Super Bowl, it is a sport that's easy to grasp yet teeming with complexity. From game to game, the amount of complex data associated with every team - and every player - increases, creating a more descriptive, timely image of the League at hand.

By using web...