10,000 domains penalized by Google
At least 10,000 domains have already been de-indexed and permanently removed from Google search results for copyright reasons, in addition to billions of URL removals. This is in response to some pirate sites abandoning regular domains and instead publishing their IP addresses.
Copyright owners, over the past decade, have asked Google to remove six billion links to “pirated” search results, and the vast majority of these requests have been granted. Although copyright infringement cannot be eliminated entirely, Google is slowly but steadily approaching itself as a willing partner in the fight against piracy.
When people use search engines to find pirated sites or pirated content, the results they receive today represent a modified subset of what is actually out there.
In response to DMCA notifications sent by rights holders, billions of URLs have been removed from Google search results. Every week, millions of new URLs are processed by Google, and when individual domains are considered to be infringing, Google interprets this as a downranking signal; one of the first to suffer was TorrentFreak .
Rightsholders, in many jurisdictions, can obtain Internet Service Provider (ISP ) blocking injunctions against infringing sites but, at least in most cases, these have no direct effect on search results.
A little over a year ago, however, everything changed.
These injunctions can now be submitted to Google for recognition. The end result is voluntary deindexing, which means that targeted sites will completely disappear from search results for the specified state. Google’s decision to completely de-index pirated sites from search results is spreading throughout Europe. In the Netherlands alone, Google has removed more than 100 domains related to its search in addition to the Pirate Bay.
For years, pirate sites have been implementing new domains to combat ISP blocking. At least initially, the tactic helped keep sites accessible, but domain changes often provoke a quick response from copyright holders to have the new domains blocked. A more recent trend in some states has seen pirate sites abandon domains altogether and rely instead on IP addresses.
It may seem like a trip back to the Stone Age (and it is), but there are short-term benefits.
The Lithuanian Radio and Television Commission (LRTK) is responsible for blocking the actions of pirate sites in Lithuania, to do this requires court injunctions, and over the years dozens of ISPs have been blocked in Lithuania ( list.txt ) .
When pirates attempt to circumvent blocking with new domains, they are handled as part of an administrative process and then added to the ISP’s existing blocking list. Because LRTK has a court injunction, these are sent to Google and the referring domains are deindexed from search results.
“…It should be noted that the domain names of the above 13 websites were previously removed from Google’s search system by order of the LRTK, but the administrators of these websites, trying to avoid the restrictions applied to them, made it possible for users to connect to the websites Using only IP addresses without the domain names“, LTRK explains.
“… Google representatives have informed us that URLs containing IP addresses reported by the Lithuanian Broadcasting Commission, which provide access to illegally published copyrighted content, have been removed from Google’s search system.”
LRTK says it views the removal of domain names and corresponding IP addresses from Google search as “an extremely effective means of preventing access to illegally published copyrighted content.”
However, it appears that the administrators of more than a dozen previously blocked and de-indexed sites have managed to reappear in Google search.
Using IP addresses instead of domain names also has another potential benefit, that of counteracting DNS blocking
When sites use human-readable domain names, those domains must be converted to an IP address so that the sites can be accessed. This is achieved by using the Domain Name System (DNS), which usually works very well.
However, when pirate sites are blocked by court orders or administrative processes, ISPs manipulate DNS records so that domain names are no longer resolved and requests are no longer directed to the relevant servers. By eliminating domain names altogether, DNS becomes surplus to its main function. This means that manipulated records are never accessed, and measures to block sites with DNS are immediately overridden.
Because of the many drawbacks, it seems unlikely that direct access from the IP address to pirate sites will become the next big thing, but that is what is happening. One example involves the Indonesian piracy giant Indoxxi, which reportedly closed in December 2019 but survives on an endless supply of clones and copies.
The block was carried out by Kominfo, the Indonesian government’s Ministry of Communications responsible for general Internet censorship. In 2022 , 3,500 pirate sites were reported to have been blocked by local ISPs , but only in more recent months has the prospect of IP address blocking emerged following requests from rights holders.
Approximately 200 sites were blocked in 2023, apparently for copyright reasons (they may also have reasons other than copyright) but there are no details on the specifics of IP address-only blocking.
The dark side of encouraging Indonesia to further develop ISP blocking is that the very last thing the government needs is encouragement; in fact it already abuses internet blocking measures to silence critics and the public ( pdf ). It is very likely that increased technical ability to block the ISP could turn into abuse.