A Google search for “onion” doesn’t return The Onion homepage anywhere in the results. A little more digging reveals that the Onion homepage isn’t in the Google index at all. Any idea why? Maybe Aaron knows.
Backlinks
Occasionally, there’s an idea so simple and powerful that you have to drop whatever you’re doing and implement it immediately.
Yesterday, I read the Jon Udell article that’s making the rounds (via Mefi and Flutterby). I didn’t immediately grok it, but seeing it in action (1, 2, 3) did the trick.
Visually, I was inspired by Mark Pilgrim’s concise display, but didn’t want to periodically parse through my Apache logs. I wanted real-time results without limiting myself to one particular web server log format. So I wrote a Perl script that’s now included on every entry page via SSI, using flat files to store the data.
As a result, there may be some issues with scalability on heavily trafficked sites, but I’d think most weblogs wouldn’t have a problem. Anyway, if you want to try it, all it requires is Perl, server-side includes, and a world-writable directory to store the files in. Download Waxy Backlinks now. Installation info inside.
The Forbidden Web
Unethical idea of the day: ‘The Forbidden Web,’ a search engine that only indexes files disallowed by robots.txt files. For example, CNN’s robots.txt file asks search engines to avoid their transcripts, jobs, website statistics, and development directories. The Forbidden Web would index only those forbidden (and often intriguing) directories. Evil, isn’t it?
A glance at the robots.txt files on some popular sites: New York Times, Google, Hotwired, eBay, Slashdot, Verisuck, Kuro5hin, Filepile, ZDNet, Epinions, IMDB, BBC, IBM, USA Today, Jakob Neilsen.
You can search Google for more robots.txt files.
Playstation 2 Emulation
A Playstation 2 emulator for Windows/Linux is being developed by the authors of PCSX, a Playstation 1 emulator. It only runs some simple demos at the moment, but it’s amazing that it exists at all. No word on a functional Xbox emulator yet…
The Roots of Hotmail
In 1992, Sabeer Bhatia posts a classified ad from his Stanford e-mail account offering his ’88 Toyota Tercel for sale. In June 1996, Jack Smith mentions Hotmail for the first time in Usenet. One month later, on July 4, they publically announced the product they created. Three years and eight million users later, they sold Hotmail to Microsoft for $400 million. It’s a good story, if you haven’t read it already.