A Google search for “onion” doesn’t return The Onion homepage anywhere in the results. A little more digging reveals that the Onion homepage isn’t in the Google index at all. Any idea why? Maybe Aaron knows.
Comments
Comments are closed.
A Google search for “onion” doesn’t return The Onion homepage anywhere in the results. A little more digging reveals that the Onion homepage isn’t in the Google index at all. Any idea why? Maybe Aaron knows.
Comments are closed.
they’ve got an anti-robot meta tag in their source. 🙂
<META NAME=”robots” content=”noindex,follow”>
Well, that answers that. I looked for a robots.txt file, but didn’t think to look at the META tags. Thanks!
But it begs the question, why would The Onion want to exclude all search engine referers from their site?
that -is- a good question. and thanks for fixing my meta tag. 🙂 you’d think, if anything, they’d want to generate more traffic. maybe it has something to do with the caching that google does? though it’s not specific to the googlebot.
Google shows 13K links in and Blogdex shows them as the #11 all-time entry; maybe they feel like they don’t want to risk giving the milk away for free if so many folks are already advertising their particular cow for them.
I understand they start writing every article with the headline and work backward, so maybe they want to ensure they register your impression before you even get the punchline, y’know? If there were a way to read their stuff without suffering the increasingly invasive ads, I know I would do it. 😉