Edward Zitron has been reading all of google’s internal emails that have been released as evidence in the DOJ’s antitrust case against google.
This is the story of how Google Search died, and the people responsible for killing it.
The story begins on February 5th 2019, when Ben Gomes, Google’s head of search, had a problem. Jerry Dischler, then the VP and General Manager of Ads at Google, and Shiv Venkataraman, then the VP of Engineering, Search and Ads on Google properties, had called a “code yellow” for search revenue due to, and I quote, “steady weakness in the daily numbers” and a likeliness that it would end the quarter significantly behind.
HackerNews thread: https://news.ycombinator.com/item?id=40133976
MetaFilter thread: https://www.metafilter.com/203456/The-core-query-softness-continues-without-mitigation
[Warning: “ideas guy” tier babble]
It’s somewhat clear that search engines are too prone to go to shit, either due to malice or something worse (like stupidity).
Based on that, I wonder if a user-run, free-as-speech and open source decentralised search system wouldn’t work. Roughly in the spirit of torrents - where anyone can use the system but if you’re using it you’re expected to contribute with it too.
You just described the categories pages many search engines had before Google. Or proto Web 2.0 bookmark sharing sites like del.icio.us. Sites like Metafilter also existed as a kind of Internet index before everyone was adding reddit.com to their Googling. It’s a laudable idea, but these systems all seem to fall prey to market manipulation in much the same way that SEO helped kill Google.
It’s interesting that you mention MetaFilter, because they’re literally in the process of transitioning fully to a non-profit organization.
https://metatalk.metafilter.com/26430/MeFi-Nonprofit-Update-March-26-2024
They’re the only aggregator that still isn’t flooded with ads and has pretty decent moderation policies.
There’s absolutely a reason I linked to the discussion over there: because it’s quality, and it’s the first place I saw the article pop up.
Wow, that’s really neat.
Thanks for letting me know about MetaFilter and its transition to NPO. This really seems like a great move for the site.
I’ve heard of the site before, but haven’t had the chance to try it before. Guess a bit late is better than never, right? :D
I was thinking on something slightly different. It would be automatic; a bit more like “federated Google” and less like old style indexing sites. It’s something like this:
It would be vulnerable to SEO, but less so than Google - because SEO tailored to the algorithm being used by one server won’t necessarily work well for another server.
Please, however, note that this is “ideas guy” tier. I wouldn’t be surprised if it’s unviable, for some reason that I don’t know.
I think you could do it in Lemmy itself combined with RSS feeds. The mods would curate a list of RSS feeds, and use the keywords to pick the ones for a bot to automatically post (which means if a programming blog did a post about windsurfing, it wouldn’t show up as long as the meta keywords didn’t match). Mods could take suggestions each week for feeds to add or remove.
There was (is?) the yacy project which used a distributed index, and the individual nodes would contribute to the index.
A hybrid of original Yahoo! and Google is probably the best option. Sites submit themselves, they get reviewed, and an algorithm catalogs the contents. So curation and automatic indexing together.
https://en.m.wikipedia.org/wiki/YaCy
That exists, it’s called Searx, and in my experience it isn’t great
No that still uses traditional search engines in the background
Searx is a meta-engine, as bdonvr mentioned.
since i changed the instances i use, it works great for me
I’ve been using foss.family for a little while with good results.
You have to spend a bit of time selecting and using different instances before you find one that suits you. I tried about three before this one and I don’t think I’ll need to change again.
One part of this (which isn’t really covered in the article) is that Google historically had a give-and-take relationship with people gaming search engine results. SEO has been a thing for a long time, and it’s impossible to make it go away. However, Google used to punish sites that took it too far. It wasn’t necessarily ideal, but it worked well enough to keep egregious spam out of the top level results, and companies could still direct users to their site when they had something they were actually looking for. SEO consulting companies sprang up who knew Google’s rules well, and that arguably meant a bunch of grifters being overpaid, but at least the results stayed relevant.
Google seems to have given up on enforcing many of those rules.
There’s also some minor discussion in the MeFi thread about “federated search” as well.
Self-hosted search also seems like a strong possibility.
The problem that I see with self-hosting is that it isn’t a practical reality for most people, due to different tech expertises and machine capabilities. Instead I think that a better system would allow you to simply install some software, and contribute as much as you can while you use it.
I’m not informed on MetaFilter. From your other comment it seems that it’s also an indexing site (besides being a community - from their “About” page). Is this correct?
Yes, it’s got a really old-school layout, because it’s been around since 1999. To me, the fact that they’ve managed to avoid advertising for 25 years while having the main indexing site as well as things like Ask MetaFilter, IRL meetups, and even a jobs board, it means they’ve been pretty darn good at managing their finances and figuring out how to support the site long-term without ads. They’re also in the process of becoming an actual non-profit organization. They pay their moderators a living wage, because it’s a job. That’s… pretty amazing.
The comment section takes a bit to get used to, because it’s just chronological order of comments, no sorted threads. Very, very old school web ethos. However, if you can get used to it, some really amazing discussion can happen in there.
One of the benefits of the ways MeFi posts work is often you have users doing massive amounts of research and providing literally mountains of links and analysis, you can get pretty lost in the weeds on some posts.
It’s been the source of high quality discussions for a long time and there’s some really interesting professionals on there who have been staples of the community for a long time. Think hackernews and how many people it has from the industry, but instead of it all being tech people (MeFi has it own share of techies) it is thoughtful and sometimes expert opinion from a large variety of disciplines, as well as first person accounts from people of all walks of life.
It’s also where I first found this link (The Man Who Killed Google Search) and decided to post it here.
Seems like something the public library system should be doing. That and hosting websites for the community not for profit but as a public service.
While I’m on wish list tangent, post offices should be municipal banks and be a free email domain provider.