Reddit blocks all crawlers in robots.txt, gives exclusive access to Google with AI deal

— this means they’re blocking the Internet Archive and every other search engine, a huge over-reaction ostensibly to stop AI training via Simon Willison #