This is the first part in a new series on Web Scraping Case Law that aims to give you the most up-to-date information on how courts worldwide deal with web scraping cases, the judgments they deliver, and how the law on web scraping affects your company.
The legal landscape around web scraping is not always clear and often relies on case law to clarify the interpretation of laws and regulations regarding web scraping. In this new Web Scraping Case Law series, we aim to provide an overview of the most important legal cases for web scraping.
Key takeaway for web scraping: one cannot be criminally liable for scraping publicly available data.
In this first installment, well look at a recent high-profile data scraping case: hiQ Labs, Inc. v. LinkedIn Corp.
Apify COO and ex-lawyer Ondra Urban discusses the hiQ v. LinkedIn case
What was hiQ Labs?
hiQ Labs Inc. was a people analytics company that offered workforce insights to businesses through its two products: Keeper, which analyzed and predicted the retention risk for the employees and indicated which employees were at greatest risk of being recruited away, and Skill Mapper that aggregated and summarized the breadth and depth of the skills possessed by an employers workforce by analyzing all of the skills its employees listed in LinkedIn public profile data. hiQ Labs primarily relied on LinkedIn for its data by scraping public LinkedIn profiles using automated web scraping tools.
LinkedIns user agreement, which each user needs to accept while creating a new profile, prohibits web scraping. LinkedIn also employs various technical mechanisms to block data scrapers.
Preliminary injunction
LinkedIn started developing a new tool called Talent Insights, very similar to hiQs Skill Mapper. Having been aware of hiQs business for some time, LinkedIn recognized it as a direct competitor. Shortly after the start of the development of Talent Insights, LinkedIn Corp. sent hiQ Labs a cease-and-desist letter asserting that hiQs unauthorized scraping of LinkedIns profiles violates the law, restricted hiQs profile and access to the LinkedIn website, and threatened to sue.
In response to the cease and desist letter, hiQ Labs demanded that LinkedIn recognize its right to access public LinkedIn pages and stop blocking hiQs access. LinkedIn refused. Consequently, hiQ filed a lawsuit against LinkedIn, seeking a preliminary injunction temporary and quick relief before the court decides on the whole dispute. As a result of the preliminary injunction granted to hiQ by the court, LinkedIn Corp. was ordered to reinstate hiQs access to public profiles and avoid restricting its access in any way.
Despite the preliminary injunction that was in place to ensure hiQ may continue its business activities, the uncertainty lingering over its business led to losing funding, clients, and employees, and eventually closing down its operations.
Litigation process
From a procedural standpoint, the litigation process might seem a bit untraditional. The longest part eventually leading to the US Supreme Court and bringing the important precedents was the proceedings about the preliminary injunction. HiQ sought a preliminary injunction to prevent LinkedIn from denying hiQ Labs access to publicly available data on public LinkedIn users profiles. The courts granted and upheld the injunction until hiQs business operations ceased. Following the cessation of hiQs business activities, there was no more need for the protective preliminary injunction as there was no longer any ongoing business activity requiring its protection. The court thus revoked the injunction.
After that followed a summary judgment proceedings. Summary judgment usually constitutes an initial phase of litigation where the most clear legal questions are answered. In the hiQ v LinkedIn case, some legal questions were already conclusively answered during the preliminary injunction phase, some were indeed answered summarily, and the rest remained open.
Ordinarily, the next (and typically longest) phase would be the main proceedings. However, this phase never occurred in the hiQ Labs v. LinkedIn Corp. case, as the parties settled the rest of the dispute out of court.
Criminal liability: Computer Fraud and Abuse Act
Before this important precedent was created, it was a common claim that web scraping violates the Computer Fraud and Abuse Act (CFAA), which was originally explicitly designed to combat federal computer fraud. CFAA is an anti-hacking act that criminalizes accessing protected computers without having the authorization to do so. The interpretation of without authorization or exceeding authorization has been the subject of debate ever since.
During the hiQ Labs, Inc. v. LinkedIn Corp proceedings, the Supreme Court in another case Van Buren v United States upheld the narrow interpretation of the law and held that the CFAAs exceeds authorized access provision covers those who obtain information from computer networks or databases to which their computer access does not extend. It does not cover those who, like Van Buren, have improper motives for obtaining information that is otherwise available to them. Following the Van Buren decision, the Supreme Court, upon LinkedIns petition, ordered that the Court of Appeal (Ninth Circuit) reconsider its decision in light of this new judgment. This has allowed the Ninth Circuit to provide the conclusive precedent that scraping publicly available data is incapable of violating the Computer Fraud and Abuse Act.
The Ninth Circuit built upon the Supreme Courts argumentation from Van Buren, where it used the gates-up-or-down inquiry for access to a protected computer - meaning that if authorization is required and has been given, the gates are up; if authorization is required and has not been given, the gates are down. The Ninth Circuit pointed out that a defining feature of public websites is their lack of limitations on access; therefore, using the gate analogy - there were no gates to lift or lower in the first place. In other words, where there is no authorization required in the first place, there is nothing to withdraw from later. The CFAA concept of without authorization simply does not apply to public websites.
Contractual liability: LinkedIns user agreement
In the summary judgment, the district court decided that, notwithstanding the above, hiQ was not allowed to scrape LinkedIn, not by law, but by contract. hiQ Labs concluded a contract with LinkedIn Corp. by accepting LinkedIns user agreement. The court held that hiQ breached provisions of the terms of use (which hiQ accepted and created a valid contract between the parties), prohibiting scraping and the creation of fake accounts.
Unfair competition
In the summary judgment decision, the court also held that LinkedIns actions, which severely damaged hiQs client relationships and eventually crushed their business, were not unlawful. Normally, such actions might be considered an unfair competition, which is prohibited by law. In LinkedIns case, those actions were lawful as they fell within the scope of California law governing litigation privilege. Such privilege justifies otherwise unlawful actions provided that they are taken to protect ones own rights in anticipation of litigation.
HiQ and LinkedIn settle out of court
While the courts judgments are published, an out-of-court settlement is made between the parties and the contents of it are typically confidential. It is, therefore, not publicly known how hiQ Labs and LinkedIn Corp. settled the dispute. Most likely, they built upon the court decisions and opinions presented in the judgments and agreed on the consequences. This is not an unusual approach, as court proceedings are very lengthy and costly.
Publicly available data not criminally punishable
The most important outcome for web scraping is the Ninth Circuit judgment ruling out the possibility of scraping publicly available data being criminally punishable.
Less groundbreaking, but a good reminder, is the contractual effects of user agreements. When web scraping, you need to consider whether you are subject to a binding user agreement. That often (though not always - read more about the enforceability of terms and conditions) happens when one signs in to a website and is asked to read and accept the websites terms and conditions. The terms can include limitations or even prohibit web scraping altogether. It is essential to be extra vigilant when scraping behind a login.
Final thoughts on hiQ Labs v. LinkedIn Corp.
The hiQ Labs v. LinkedIn Corp. case offers some critical insights into the evolving legal landscape around web scraping. It sets a precedent that scraping publicly available data does not fall under computer fraud, as laid out by the CFAA. This is significant for anyone involved in web scraping. If youre thinking about scraping a public website, the gates are essentially up, legally speaking.
However, its crucial not to overlook the contractual obligations that may come into play. LinkedIn was able to assert its rights through its user agreement, which hiQ had to accept to use the service, thereby rendering their actions unauthorized data scraping. So, while you might be in the clear from a criminal standpoint, make sure youre not stepping on any contractual landmines.
Additionally, its worth noting that actions taken in anticipation of litigation, like what LinkedIn did to hiQ, can be legally justified. This reminds us that while the law provides a framework, each case is unique and can be influenced by various legal mechanisms.
Finally, the case ended in an out-of-court settlement, which, while not publicly disclosed, likely considered the various legal complexities that were brought to light during the court proceedings.
The hiQ v. LinkedIn case is a landmark in web scraping case law. It brings clarity to some extent but also leaves room for interpretation and further legal developments. If youre a business relying on web scraping for data, keeping an eye on cases like this can provide valuable takeaways for navigating the complex and often murky waters of web scraping law.
Well return with another installment in our Web Scraping Case Law series soon.
Top comments (0)