There are few, if any, legal domains where hypocrisy is as baked into the ecosystem as it is with web scraping.
Some of the biggest companies on earth—including Meta and Microsoft—take aggressive, litigious approaches to prohibiting web scraping on their own properties, while taking liberal approaches to scraping data on other companies’ properties.
When we talk about web scraping, what we’re really talking about is data access. All the world’s knowledge is available for the taking on the Internet, and web scraping is how companies acquire it at scale. But the question of who can access and use that data, and for what purposes, is a tricky legal question, which gets trickier the deeper you dig.
Some forms of data are protected by copyright, trademark, or another cognizable forms of intellectual property. But most of the data on the Internet isn’t easily protectible as intellectual property by those who might have an incentive to protect it.