Do What is Right Not What is Easy!

I was recently invited to speak at the IAPP Europe Data Protection Congress in Brussels about web scraping and GDPR. The panel also included Claire François of Hunton Andrews Kurth and Peter Brown from the Information Commissioner’s Office (ICO). For more information you can check out my blog about this topic GDPR Compliance for Web Scrapers: The Step-by-Step Guide.

 

Key takeaways from the event:

 

1: Scraping Personal Data - Legitimate Interest

There are only two legal basises for scraping personal data (1) consent or (2) legitimate interest. While consent is rare in web scraping cases, it’s the cleaner of the two options, so much of the panel discussion at the IAPP Congress was spent on legitimate interest. In reality, legitimate interest will typically be the only legal basis at your disposal when scraping personal data, so is there a compliant way to use legitimate interest as a legal basis when web scraping?? Maybe . . . sometimes . . . if you’re really careful.

 

2: Legitimate Interest Explained

Where no other legal basis is available, many companies are turning to legitimate interest. Legitimate interest can be used where the use case for the personal data is a use that the data subject would reasonably expect and have a minimal privacy impact. When determining if this is the case, this three-factor test can be utilised:

  1. Identify the legitimate interest (for example, Recital 47 of the GDPR states that “...the processing of personal data for direct marketing purposes may be regarded as carried out for a legitimate interest.”);

  2. Show that processing is necessary to achieve that legitimate interest; and

  3. Balance the legitimate interest against the individual’s rights.

Following on from the Recital 47 example above, in order to complete the final two steps, you would need to (1) show that your scraping of the personal data is required in order to achieve your legitimate interest of direct marketing (meaning couldn’t get the data through some other legal basis, like consent), and (2) ensure that your legitimate interest to the data is not outweighed by the individual’s right to privacy. When weighing the interests, think about the privacy impact your use of the data might have on the individuals and whether the people whose data you scrape would be surprised or likely to object to your use of their data.

Always ensure that you document how you assessed legitimate interest, and if you need additional guidance the ICO has published a legitimate interest assessment form on their site. If you are able to successfully pass the three-factor test and assessment, you may be able to use legitimate interest as your legal basis for scraping personal data.

 

3: Protecting Data Subject’s Rights?

Well that’s where things get trickier. If for example you’re using Recital 47 and make a determination that your processing of personal information for direct marketing purposes qualifies as a legitimate interest, how do you inform the data subjects that you have their information or provide them with their right to access data, correct errors, object to processing, and request erasure?

Some ideas considered during our panel discussion:

  1. Conducting a Data Processing Impact Assessment (DPIA)

  2. Review the use case for the data to determine if it aligns with the data subject’s original purpose for sharing the data

  3. Territorial scope -- consider where the scraping is taking place and the location of the company that is conducting the scraping. Remember, GDPR only applies if:

    (a) you are established in the EU and you are scraping data in the context of the activities of your EU establishment; or
    (b) you are not established in the EU and you scrape personal data of individuals in the EU.
  4. If the privacy policy of the website scraped lists categories of third parties that may access the personal data and you fall within those categories

  5. Obtaining consent after scraping the data.

There are potential pitfalls with all of these options that would require legal guidance, but it was great to get this conversation going in an environment full of data protection experts.

 

4: ICO Recommendations

It was great to hear the ICO’s recommendation, given that they are the ones enforcing GDPR. The ICO was clear that they don’t have any specific recommendations on web scraping, but you can look to their recommendations on “Invisible Processing” to get some guidance. Invisible Processing is the “processing of personal data that has not been obtained direct from the data subject in circumstances where the controller considers that compliance with Article 14 would prove impossible or involve disproportionate effort.” The ICO considers Invisible Processing “high risk” and thus requires a DPIA to be conducted prior to such processing.

A DPIA is an assessment that helps you analyse, identify, and minimise the data protection risks of a project, to ensure compliance with GDPR. The ICO provides a step-by-step list for conducting a DPIA, which includes:

  1. Identify the need for a DPIA
  2. Describe the processing
  3. Consider outside consultation
  4. Assess necessity and proportionality
  5. Identify and assess risks
  6. Identify measures to mitigate the risks
  7. Sign off and record outcomes
  8. Integrate outcomes into a project plan
  9. Keep your DPIA under review

There are also various data protection software packages on the market, which walk you through a step-by-step DPIA process. At Scrapinghub, if we were to utilise the DPIA approach, it would be our preference to conduct it within the data protection software we use, so that we’re conducting the most robust and thorough analysis possible.

 

Conclusions

Attending and speaking at the IAPP Congress helped to get web scraping on the minds of some of the leading data protection experts in the world, and we’re hopeful that this will turn into direct guidance from organisations like the ICO about web scraping. In the meantime, Scrapinghub will continue to advocate for fair scraping of public data, and will continue to guide our customers to help them lawfully scrape personal data.

Disclaimer: I am a lawyer, but I am not your lawyer and the recommendations in this post do not constitute legal advice. The commentary and recommendations outlined are based on Scrapinghub’s experience helping our clients (startups to Fortune 100’s) maintain GDPR compliance whilst scraping 7 billion web pages per month. If you want legal advice regarding your specific situation then you should consult a lawyer. 

October 10, 2019 In "Web Scraping" , "Crawlera" , "GDPR" , "Extract Summit" , "Web Data Extraction Summit"
July 25, 2019 In "Web Scraping" , "web crawling" , "GDPR" , "Compliance" , "Legal"
May 23, 2019 In "Web Scraping" , "GDPR" , "Compliance" , "Legal" , "Solutions" , "Architecture" , "solution architecture"
GDPR, Compliance, Legal
Sign up now

Be the first to know. Gain insights. Make better decisions.

Use web data to do all this and more. We’ve been crawling the web since 2010 and can provide you with web data as a service.

Tell me more

Welcome

Here we blog about all things related to web scraping and web data.

If you want to learn more about how you can use web data in your company, check out our Data as a Services page for inspiration.

Follow Us

Learn More

Recent Posts