From 404Media: Reddit Goes Monogamous With Google
Graphic from Ian Furst, CC BY-SA 3.0
In a move that is either protecting a site from AI scraping or increasing the monopolies monopoly – depending on your point of view – Reddit is now only indexing for Google.
Google is now the only search engine that can surface results from Reddit, making one of the web’s most valuable repositories of user generated content exclusive to the internet’s already dominant search engine.
If you use Bing, DuckDuckGo, Mojeek, Qwant or any other alternative search engine that doesn’t rely on Google’s indexing and search Reddit by using “site:reddit.com,” you will not see any results from the last week. DuckDuckGo is currently turning up seven links when searching Reddit, but provides no data on where the links go or why, instead only saying that “We would like to show you a description here but the site won’t allow us.” Older results will still show up, but these search engines are no longer able to “crawl” Reddit, meaning that Google is the only search engine that will turn up results from Reddit going forward. Searching for Reddit still works on Kagi, an independent, paid search engine that buys part of its search index from Google.
The news shows how Google’s near monopoly on search is now actively hindering other companies’ ability to compete at a time when Google is facing increasing criticism over the quality of its search results. This exclusion of other search engines also comes after Reddit locked down access to its site to stop companies from scraping it for AI training data, which at the moment only Google can do as a result of a multi-million dollar deal that gives Google the right to scrape Reddit for data to train its AI products.
“They’re [Reddit] killing everything for search but Google,” Colin Hayhurst, CEO of the search engine Mojeek told me on a call.
This is one heck of a deal…
But, oddly enough, it doesn’t strike me as being indicative of health.
Google search has degraded over the past few years. It used to be the only search engine that you’d ever want. Why would you need another? But then… the SEO turned to crap and people kept coming up with new tricks to make Google work the way it used to, or closer to how it used to.
One of the big tricks that I’ve seen passed around is “add reddit to the end of the search”.
So if your problem was something like “minecraft game runs out of memory”, you may or may not get a decent hit but if you searched for “minecraft game runs out of memory reddit”, holy cow, you’d immediately get five threads that talk about this and two of them have good advice for your specific problem.
Making it so that Reddit only lets google spiders inside the front door makes Google more valuable… but only because it’s good at searching Reddit. Not because it’s good at searching the web.
A million years ago, this would have been a situation ripe for Google to get Digged.
It might be okay to limp along, but Reddit is slowly choking itself at the same time.
In the short run, though… this is a good move. It just doesn’t address the fundamental reasons why it was such a good move.Report
Related:
Report