Maybe I’m mistaken, but something interesting seems to be happening with web search. More and more people appear to be looking for alternatives to Google and other mainstream search engines. Not necessarily because they dislike Google, but because they’re looking for something different. And I’m not talking about privacy-focused search engines like DuckDuckGo. Those are interesting too, of course, but what I’m seeing goes a little further.
People are experimenting with search engines that focus on small, independent websites. Others are bringing back old-fashioned web directories, collecting interesting links by hand, or building tools that send you to a completely random website. Some are even using AI to find websites they would otherwise never have encountered.
What interests me most is that these people aren’t necessarily trying to find better answers to their questions. They’re trying to discover things they didn’t know existed. And I think there’s an important difference.
How do you search for something you don’t know exists?
I recently wrote about discovering Ethiojazz (https://www.rousette.org.uk/archives/ethiojazz/), a wonderful combination of musical influences I had never really explored before. I came across it through Bubbles (https://bubbles.town/), which led me to someone else’s blog. I wasn’t searching for Ethiojazz. In fact, I wouldn’t even have known what to search for. Someone else had discovered something interesting, written about it, and shared it. Through a series of links, it eventually reached me.
Of course, recommendation algorithms can introduce us to new things. But they usually start with what they already know about us. Listen to a particular metal band and you’ll probably get recommendations for similar bands. Enjoy old-school jazz and you’ll be offered more of it. Nothing wrong with that. But it does mean we’re often being shown variations of things we already like. What about the things we don’t know we like yet?
The return of the small search engine
I’ve been collecting examples of alternative approaches to web search, and there are quite a few interesting ones. One is Marginalia Search (https://marginalia-search.com/), which pays particular attention to the smaller, less commercial parts of the web. Its results can feel quite different from those of a mainstream search engine. Another is Wiby (https://wiby.me/), which focuses on simple, old-fashioned websites. It even has a surprise function that takes you somewhere unexpected.
Then there’s Kagi Small Web (https://kagi.com/smallweb/), which brings together thousands of independent blogs and personal websites. Rather than entering a search query, you can simply browse recent posts or click through to another website. Kagi has expanded this project with categories and tools that make exploring easier. And there are more. Search My Site (https://searchmysite.net/) for example focuses on independent websites, while Feedle (https://feedle.world/) lets you search blogs and podcasts.
I also came across this collection of search engines for wandering (https://www.are.na/elan-kiderman-ullendorff/search-engines-for-wandering) on Are.na and an interesting article about folk search engines (https://escapethealgorithm.substack.com/p/folk-search-engines). That last description is rather nice, isn’t it? Folk search engines.
The common theme is that these projects are trying to make parts of the web visible that don’t necessarily perform well in conventional search results. Not because those websites are less interesting, but because they may have little search engine optimization, few incoming links, no marketing budget and no particular interest in attracting large amounts of traffic. They’re simply websites made by people who have something to share.
Remember when we just wandered around the web?
There’s another category of tools that takes this a step further. Instead of helping you search, they help you wander. Cloudhiker (https://cloudhiker.net/) sends visitors to unusual and interesting websites. Indieblog page(https://indieblog.page/) lets you discover random independent blogs. And Neocities (https://neocities.org/browse) is full of personal websites, experimental designs and projects that probably wouldn’t survive a meeting with a professional web design consultant.
These projects remind me of the early days of the web, when clicking a link could take you almost anywhere. You’d start by reading about computers, follow a few links, and suddenly find yourself on a website about someone collecting vintage radios in their garage. You hadn’t planned to spend twenty minutes learning about vintage radios. But you did. And you enjoyed it.
Today, much of our online experience is designed to eliminate that kind of uncertainty. Search engines try to give us the most relevant result. AI tools increasingly provide the answer directly, sometimes removing the need to visit a website at all. Very convenient, obviously. But efficiency isn’t always what I’m looking for when I’m browsing the web. Sometimes I want to get lost.
Perhaps we need more humans in search
Another development I find interesting is the return of the human-curated web directory. Remember those? Before search engines became so dominant, people collected websites and organized them into categories.
There are modern versions of that idea, including ooh directory (https://ooh.directory/), Blogroll.org (https://blogroll.org/) and the many directories listed by the Indieweb community (https://indieweb.org/directory). And let’s not forget blogrolls and webrings. Both are wonderfully simple concepts. Someone maintains a list of websites they enjoy, or a group of websites links to one another, allowing visitors to move from one to the next. No complicated algorithm required.
What makes these approaches interesting is that they introduce another person’s curiosity into the process. I may not know anything about experimental electronic music, forgotten programming languages or unusual types of photography. But somebody else does. And if that person maintains a blogroll or shares interesting links, I can benefit from their curiosity. That’s how we end up discovering things we would never have thought to search for ourselves.
RSS could become a discovery engine
Anyone who has read this blog before knows that I’m a big fan of RSS. For me, RSS has always been a practical way to follow websites without depending on social media platforms or recommendation algorithms. I decide which sources I want to follow, and their new posts appear in my reader.
But perhaps I’ve been thinking about RSS too narrowly. Normally, we use RSS to follow sources we already know. What if we used it to discover sources we don’t know? There are projects such as PowRSS (https://powrss.com/) that are worth exploring in this context. Kagi Small Web also offers RSS feeds, and various directories and independent websites publish feeds of their own.
Imagine combining feeds from dozens of independent blogs, directories and interesting corners of the IndieWeb. Instead of showing every new post, a small tool could select a handful of articles each day. And here’s the important bit: it shouldn’t simply select articles that match my existing interests. I’d rather have it show me something unexpected. Maybe an article about an obscure computer from the 1980s. Or someone building a homemade musical instrument. Or a personal website written in a language I don’t speak, covering a subject I’ve never heard of.
I might ignore most of them. That’s fine. The occasional unexpected discovery would make the whole thing worthwhile.
Could AI help us get lost?
This is where things become particularly interesting to me. I’m a big fan of AI, and I use it for all sorts of projects, including coding and experimenting with websites and journalism tools. Most discussions about AI and search focus on making information easier to find. Ask a question, get an answer, preferably without having to read ten different websites.
But what if we turned that around? Instead of asking AI to answer a question, we could ask it to find questions we haven’t thought of asking. Or websites we didn’t know existed. For example, I could ask an AI tool to find fifteen small, independently maintained websites about unusual technology projects. Preferably from different countries, written in different languages, and without the usual big technology publications.
Or I could ask it to find surprising connections between old computer technology, music and DIY electronics. Of course, AI isn’t automatically good at this. It can recommend the same familiar websites over and over again, and sometimes it invents links that don’t exist. So actual web searching and verification would still be necessary.
But combine AI with RSS feeds, independent web directories and collections of manually selected websites, and suddenly you have the ingredients for a rather interesting discovery engine. I already experiment with local AI models, RSS and scripts for collecting information from websites around the world. Building something like this wouldn’t be completely outside the sort of projects I enjoy.
Perhaps I should give it a try. A script could collect links from different sources, use a small local model to identify unusual or surprising subjects, and present five completely different websites every morning. Not a personalized recommendation engine designed to give me more of what I already like. Quite the opposite. A recommendation engine designed to occasionally get things wonderfully wrong.
Maybe search doesn’t always need to be better
Perhaps we’ve become so focused on finding the right answer that we’re overlooking another reason to explore the web. Curiosity.
The interesting thing about all these small search engines, directories, random website tools, blogrolls and RSS experiments is that they make room for things we weren’t looking for. They remind us that the web isn’t just an enormous collection of answers waiting for our questions. It’s also millions of people making things, writing things, experimenting, sharing ideas and connecting subjects in ways we might never have imagined.
Sometimes the most interesting website isn’t the one that best matches our search query. It’s the one we would never have searched for in the first place. Maybe that’s something worth rediscovering.