Telehack is the most interesting game I’ve played in the last year… a game that most users won’t realize is a game at all.

It’s a tour de force hack — an interactive pastiche of 1980s computer history, tying together public archives of Usenet newsgroups, BBS textfiles, software archives, and historical computer networks into a multiplayer adventure game.

Among its features:

• Connect to over 24,000 simulated hosts, with logged-in ghost users with historically-accurate names culled from UUCP network maps.
• Hacking metagames, using simplified wardialers and rootkit tools.
• User classes that act as an achievements system.
• Group chat with relay, and one-on-one chat with send or talk.
• Reconstructed Usenet archives, including the Wiseman collection.
• A BASIC interpreter with historical programs from the SIMTEL archives.
• Standalone playable games, including Rogue and a Z-code interpreter for text adventure games like Adventure and Zork.
• Hidden hosts and programs, discoverable only by hacking Telehack itself.

The entire project was engineered by “Forbin,” an anonymous Silicon Valley engineer named after the protagonist of Colossus: The Forbin Project. Like the chief engineer of the film, Forbin’s created a networked supercomputer that defies all expectations. (Hopefully it won’t gain sentience and enslave the human race.)

I had to know more. With the help of Paul Ford, I interviewed Forbin about the project — using Telehack’s send utility, naturally. Read on for the full interview about his motivations, how it’s built, and why he’s chosen to remain anonymous.

Andy Baio: So, first off, I want to tell you how much I’m in love with Telehack. You’ve made something truly unique… I’d love to hear about your inspiration for building it.

Forbin: Thank you. I’m glad that people are enjoying it. The inspiration was my son. I had shown him the old movies Hackers, Wargames, and Colossus: The Forbin Project and he really liked them. After seeing Hackers and Wargames, he really wanted to start hacking stuff on his own.

I’d taught him some programming, but I didn’t want him doing any actual hacking, so I decided to make a simulation so he could telnet to hosts, hack them, and get the feel of it, but safely.

What did he think?

He really liked it. At first he thought it was all real, and he was actually hacking into government computers and such. It was great. Eventually though, like Santa Claus, he figured out he wasn’t really wardialing all those systems.

He’s been my best beta-tester. 🙂

When do you think he’ll be ready for real-world hacking? Is there a path to graduation from Telehack?

Telehack can help you learn commandline basics. I’ve told him not to ever enter any real systems without permission.

I’m curious if you’ve played Digital: A Love Story. It seems like Christine Love was trying to do something similar — conjuring an earlier time in computing history, but without worrying too much about historical accuracy.

I played a bit of Digital: A Love Story. I thought it was wonderful. I really liked the atmosphere it evoked. That was the same effect I was going for in Telehack, but in a different way. Silent, no sound… Just green text and more code, but the same emotion.

Regarding historical accuracy, one of the surprising things about Telehack is how those old systems were hard to use. Whenever a movie or a book looks back at the past, it can look through a set of lenses that make the past seem more engaging and accessible, and sometimes add a narrative. Someone on Hacker News referred to Telehack as “MovieOS,” and that’s exactly right.

There are actual TOPS-10 systems on the net you can get accounts on. They’re not easy to get into. I wanted to reduce the usability barriers to the old commandline interfaces, while giving the same feel of the systems, and blend some of the good things I remembered from various systems together.

Telehack seems to borrow quite a bit from modern videogame theory… Integrated tutorials, slowly ramping up difficulty, multiple avenues to exploration without a linear path. Paul Ford suspected you’re a game designer.

I have done some computer games, although that is not my profession. I really admire the advances that have occurred in game design, although I’m not much of a gamer myself. Mostly, I wanted to help newcomers get across barriers of accessibility which is what the old tutorial manuals were all about.

Read an old DECsystem-20 manual. It tells you, in excruciating detail, how to type control-C to interrupt a program. It turns out people still need to know that but aren’t being told anymore.

The other part is that the system is open-ended. It has all this old code that is animated by resurrecting an interpreter for a dead language. People can run programs again that haven’t been run — and experienced — for 20 years. And see old files through a lens that makes them look like they used to. That’s fun.

Speaking of old files, the way you used the textfiles.com and Usenet archives feels brand new. You’ve created a playful environment for exploring archival material in a new way. It made me wonder what other data sources could be reinvented by making them game-like.

What Jason Scott did with textfiles.com is heroic. He’s saving away all this stuff that is completely unique, and irretrievable otherwise. But to make people want to see it, byte by byte, I thought it would be neat to offer it up, a piece at a time, in a format similar to how the files were originally experienced.

You dial some modem number — not busy! It actually connects and then you see what files they have and download some of them. Most of them are crazy stuff but period-relevant. So it’s a way to animate old text files.

Same thing with the Usenet archive, although my Usenet reader needs some work. It’s pretty crufty. There is so much in there, I haven’t really found a good way to get people back into it yet.

Folks would wait in anticipation for Usenet — the daily poll — where your modem would call some hub and get you the news. I have to find some way to bring that back to the archive.

Google Groups doesn’t give that feeling. Neither does Telehack’s usenet command currently. Still noodling on that.

Does it advance in real-time? Are you adding “new” articles to the archive in a rolling period?

No, although that’s a good idea for a way for it to be more dynamic and engaging.

You’ve been adding features incrementally since you launched it, but how long did it initially take you to build from conception?

If you type uptime, it says sysgen was 454d ago at 07-Mar-10 20:26:00. That’s when I started working on it. Not full-time, there were months when I didn’t have any time to work on it. It’s mostly been a small side project.

What was the hardest thing to get right?

There is one feature that works in the telnet interface, but not in the http client yet — the baud rate simulation. If you dial or wardial into a host and you’re connected to Telehack with telnet, it will actually give you a 2400 or 9600 or whatever connection, but that doesn’t work on the html interface yet sadly.

It’s not the same when you dial into a host and the text renders instantly. For the full authentic feel, you should have to wait for the lines to appear slowly, as we once did. 🙂

I keep my 300 baud VicModem on my desk as a reminder of how good we have it now.

300 baud was really slow! I started there too, with a Hayes on an Apple II. Type baud 300 on Telehack, it’s hard to see how we could use those systems, but we did. They were amazing, even at that speed.

I’d love to hear more about the technology behind it.

Telehack is built in Perl, in a single process, in an epoll-driven event loop. There are two interpreters — Z-code, which is an interpreter called Rezrov in CPAN, and a BASIC interpreter, which I wrote. Currently it doesn’t fork any external code. Various functions, including the 6502/VAX CPU and such, are simulated.

Any plans to open-source any of it? I’m sure some of the community would love to hack on Telehack, to extend it in different directions.

Yes, I plan to open it up at some point.

So, I have to ask: why the anonymity? The mystery definitely adds to the fun, but most people would love to take credit for such an impressive project.

Well, I have a day job, and I didn’t want this to be a distraction. I also made this for my kid, but didn’t really want to expose him to a larger internet just yet.

Are you worried that coming out from behind the curtain will bring attention to him too? I’m the parent of a six-year-old, and can definitely appreciate that.

At an earlier point in my career, I got a lot of press for a project I did. The intense interest from that made me very cautious about what I put online. I took down my personal photos and such. I would be sad if I felt like I had to take down Telehack.

Makes sense. What have you thought about the reaction? Like several others, when I first saw Telehack, I completely underestimated its depth. It seems like an deep rabbit hole that endlessly rewards curiosity.

I’m extremely happy that people are enjoying it. I was a bit sad that some commenters initially dismissed it as a simple JS shell. It was pretty cool when the first person found ptycon and the secret entry points in the system monitor.

With zero fans, I’d be pretty disappointed. With any n > 0, I’m happy. I don’t expect it’s a huge audience though. Doesn’t have to be. It’s an artistic/historical project to me.

You’ve done some incredible data archaeology here — reconstructing 25,000 hosts from the early Internet, along with the people that used them at the time. One guy on Hacker News was able to find himself and his two best friends at the time logged into the host he first used when he get online. How did you do it?

Well, a lot of this information is available online, but you need to look at the right way to interpret it.

Were you able to find yourself in the archives?

Yup, I’m in there.

Any specific rules for how you scattered the text and game files across hosts?

There is some topic-clustering for the text files, the rest is mostly random. But I’m still working on the game parts of this thing.

If Telehack ever takes off, would you ever consider doing it for a living? I have no idea what you do for a living, but I can’t help but think you’ve missed your true calling as a game designer.

Well, thank you for that. 🙂 I’ll have to get back to you on that. At this point, I’m mostly interested in fleshing it out, so I’m happy with it.

Telehack’s a pastiche of many different systems, networks, and tools from the mid- to late-1980s. It’s rich with nostalgia. Is there anything you miss from that era?

Good question. The commandline was a universal language. You had to learn it, there was a curve, but it wasn’t that hard. Heck, we were all being taught it in the new classes in middle school. But the GUI’s mission was to kill it.

My worry is that the CLI was symbolic, algebraic, whereas the GUI is… pictorial, or one-step, or something.

hosts | grep foo

It’s important that you understand that. If you’re graduating from any university today, it’s an algebra more important than… uh, algebra, maybe.

I actually don’t miss anything from that era. But I want the best of what was known then to propagate today.

This weekend, I was invited to participate in Rhizome’s Seven on Seven in NYC — an event that pairs seven artists with seven technologists, challenging them to create something in one day and present it to an audience the next day.

The other teams were a humbling roster of creative geeks, including Ricardo “mrdoob” Cabello, Ben Cerveny, Jeri Ellsworth, Zach Lieberman, Kellan Elliott-McRea, Chris “moot” Poole, Bre Pettis, and Erica Sadun.

I was paired with Michael Bell-Smith, whose digital art I’d admired and linked to in the past. It was a perfect match, and we’re very happy to announce the result of our collaboration: Supercut.org (warning: NSFW audio).

Supercut.org is an automatic supercut composed entirely out of other supercuts, combined with a way to randomly shuffle through all of the supercut sources.

The Idea

When we first started work on Friday morning, Michael and I started brainstorming what we wanted to accomplish: something visual, high-concept (i.e. explainable in a tweet), and hopefully with a sense of humor.

We quickly realized that our interest in supercuts was fertile ground. Michael’s work often touches on structural re-edits and remixes, such as Oonce-Oonce, Battleship Potemkin: Dance Edit, Chapters 1-12 of R. Kelly’s Trapped in the Closet Synced and Played Simultaneously, and his mashup album mixing pop vocals over their ringtone versions.

Both of us were fascinated by this form of Internet folk art. Every supercut is a labor of love. Making one is incredibly time-consuming, taking days or weeks to compile and edit a single video. Most are created by pop culture fans, but they’ve also been used for film criticism and political commentary. It’s a natural byproduct of remix culture: people using sampling to convey a single message, made possible by the ready availability of online video and cheap editing software.

So, supercuts. But what? Making a single supercut seemed cheap. I first suggested making a visual index of supercuts, or a visualization of every clip.

But Michael had a better idea — going meta. We were going to build a SUPERSUPERCUT, a supercut composed entirely out of other supercuts. And, if we had time, we’d make a dedicated supercut index.

Making the SuperSupercut

There were three big parts: index every supercut in a database, download all the supercuts locally, break each video into its original shots, and stitch the clips back together randomly.

Using my blog post as a guide, Michael added every supercut to a new table in MySQL, along with a title, description, and a category. While Michael did that, I wrote a simple script that used youtube-dl to pull the video files from YouTube and store them locally.

To split the clips, we needed to find a way to do scene detection — identifying cuts in a video by looking at movement between two consecutive frames. But we needed a way to do it from my Linux server, which ruled out everything with a GUI.

After some research, I found a very promising lead in an obscure, poorly-maintained French Linux utility called shotdetect. It did exactly what we needed — analyze a video and return an XML file of scene start times and durations.

The most challenging part of the entire day, by far, was simply getting shotdetect to compile. The package only had binaries for Debian and the source hadn’t been updated since September 2009. Since then, the video libraries it needed have changed dramatically, and shotdetect wouldn’t compile without locating them.

Three frustrating hours later, we called in the help of one of the beardiest geeks I know, Daniel Ceregatti, an old friend and coworker. After 20 minutes of hacking the C++ source, we were up and running.

With the timecodes and durations from shotdetect, we used ffmpeg to split each supercuts into hundreds of smaller MPEG-2 videos, all normalized to the same 640×480 dimensions with AAC audio. The results weren’t perfect — many scenes were broken during dialogue because of camera changes — but it was good enough.

As ffmpeg worked, I stored info about each newly-generated clip in MySQL. From there, it was simple to generate a random ten-minute playlist of clips between a half-second to three seconds in length.

With that list, we used the Unix `cat` utility to concatenate all the videos together into a finished supersupercut. We tweaked the results after some early tests, which you can see on YouTube.

While the videos processed overnight, we registered the domain, built the rest of the website, and designed our slides for Saturday’s event — taking time out for wonderful Korean food with the fellow team of Ben Cerveny and Liz Magic-Laser. I finally got to sleep at 5:30am, but I’m thrilled with the results.

The Future

There were several things we talked about, but simply didn’t have time to do.

I’m planning on using the launch of Supercut.org to finally retire my old supercut list by adding a way to browse and sort the entire index of supercuts by date, source, and genre. Most importantly, I’m going to add a way for anyone to submit their own supercuts to the index.

And of course, when any supercut is added, it will automatically become part of the randomized supersupercut on the homepage: an evolving tribute to this unique art form.

The newest beta of ThinkUp adds an API to the app for the first time, allowing developers to easily build apps on top of data coming from ThinkUp.

The JSON API was created by Sam Rose, a 20-year-old student from Wales and an active contributor in the ThinkUp community. His 7,000 line contribution — composed of 40% new tests and 40% documentation — earned him first place in the ThinkUp bounty contest and a brand new iPad 2. Congrats, Sam!

I thought it’d be fun to try building a hack with his new API, so I made a simple visualization of your entire Twitter archives in ThinkUp — ThinkBack, a ThinkUp-powered time capsule. Take a look at my history, or on the @whitehouse account to get the gist.

ThinkBack analyzes your entire Twitter history, extracts entities from the text, and colors them based on category. Grouped by month, it also gives you a quick glimpse at your posting activity over time.

For the entity extraction, I used an excellent free web service called AlchemyAPI to extract people, places, events, product names and other keywords from everything I’ve ever posted. They provide a category for each, which I assigned a color.

I also tested two other free web services that offer entity extraction, text-processing.com and Zemanta. Finding and categorizing keywords from short status updates is no small feat, but AlchemyAPI does a remarkable job. (If you’d like to play around with all three, support for both Zemanta and text-processing.com is commented out in the source code, but easily swappable with AlchemyAPI.)

ThinkBack also uses four typefaces from Google Web Fonts, my first time using them and dead simple to implement. For free fonts, the quality’s surprisingly great, with several faces commissioned by Google itself. For a quick, free hack, it’s a great alternative to Typekit.

I also used a very simple PHP templating language called RainTPL, which I chose as a lightweight alternative to Smarty. In practice, I found it too simple. Its handling of complex data structures and loops required me to jump through hoops that shouldn’t be necessary. (I’ll stick with Smarty next time.)

Anyway, you can download the code here, it only requires PHP and access to a recent version of ThinkUp. Feel free to fork it and submit a pull request for anything you add!

Every year, I think it’ll be my last, and every year, I keep going. Why? Because what makes SXSW Interactive special isn’t the panels, parties, BBQ, or endless free alcohol (though those all help). It’s the unique group of creative individuals that shows up in Austin every year — a wonderful mess of people, all stuck in the same city at the same time.

Those unique set of circumstances create a serendipity machine, and the people I meet each year keep me coming back, even as it busts at the seams. To that end, please track me down or say hi! That’s why I’m there. And here’s where you’ll find me:

Worst Website Ever II: Too Stupid to Fail

Monday, March 14 at 11am

Hilton, Salon D

After a three year absence, I’m bringing Worst Website Ever back to SXSW with an all-star lineup of designers, developers, and entrepreneurs. Join us as these very talented people pitch their worst website/app/startup ideas to a live audience in short, five-minute rounds.

This year, the lineup’s pretty amazing:

Gina Trapani (Lifehacker, Expert Labs)

Jeffery Bennett, (BetamaXmas, 2008’s runner-up with Image Search for the Blind)

Josh Millard, (Metafilter, viral musician)

Jonah Peretti, (Buzzfeed, Huffington Post)

Mike Lacher, (Wonder-Tonic, Wolfenstein 1-D, Geocities-izer)

Ze Frank (The Show, Star.me)

And like last time, it’s judged by a real VC, Rob Hayes from First Round Capital. Winner gets funded!

ThinkUp: Austin Meetup

Saturday, March 12 at 5pm

The Ginger Man, outdoor patio

Curious about ThinkUp, or want to meet the people behind it? I’ll be joining Gina Trapani, Amy Unruh, Jed Sundwall, and other developers/users of ThinkUp at our second SXSW meetup. Come on out! We’ll be in the back patio of The Ginger Man, if weather permits.

Stalk Me

This year, I’ll be tracking interesting sessions on Lanyrd, and evening events on Plancast. And, of course, I’ll be mentioning any unusually great activity in real-time over on Twitter.

See you there!

For the last three weeks, I’ve indexed The Daily. Now that my free trial’s up, I’ve had an intimate look at what they have to offer and, sad to say, I don’t plan on subscribing. As a result, I’m ending The Daily: Indexed, my unofficial table of contents for every article they published publicly.

I’m surprised and grateful that The Daily executive and legal team never tried to shut it down. On the contrary, when asked directly about it, publisher Greg Clayman said, “If people like our content enough to put it together in a blog and share it with folks, that’s great! It drives people back to us.” They seem like a nice bunch of folks, and I hope they succeed with their big publishing experiment.

But now that I’m ending it, I can finally address the most common question — how did I do it?

The Daily: Indexed is just a list of article headlines, bylines, and links to each article on The Daily’s official website. Anyone can grab the links from the Daily iPad app by clicking each article’s “Share by Email” button, but that would’ve taken me far too long. So, how to automate the process?

When you first start The Daily application, it connects to their central server to check for a new edition, and then downloads a 1.5MB JSON file with the complete metadata for that issue. It includes everything — the complete text of the issue, layout metadata, and the public URLs.

But how can you get access to that file? My first attempt was to proxy all of the iPad’s traffic through my laptop and use Wireshark to inspect it. As it turns out, The Daily encrypts all traffic between your iPad and their servers. I was able to see connections being made to various servers, but couldn’t see what was being sent.

Enter Charles, a brilliantly-designed web debugging proxy for Mac, Windows, and Linux. By default, Charles will listen to all of your HTTP network traffic and show you simple, but powerful, views of all your web requests. But it can also act as an SSL proxy, sitting in the middle of previously-secure transactions between your browser and an SSL server.

After grabbing the JSON, I was able to write a simple Python script to extract the metadata I needed and spit out the HTML for use on the Tumblr page. Here’s how to do it.

Configuring Charles

1. Download and install Charles on your desktop machine. On your iPad, navigate to http://charlesproxy.com/charles.crt to trust Charles’ SSL certificate.

2. For Mac users, start Network Utility to get your desktop’s local IP address. Start your iPad, make sure it’s on the same wireless network as your desktop, and go into Settings>Network>Wi-Fi. Select the wireless network, and click the right arrow next to it to configure advanced settings. Under “HTTP Proxy,” select “Manual.” Enter the IP address of your desktop for “Server” and enter in “8888” for the port.

3. Now, start Charles on your desktop and, on the iPad, try loading any website. You should see assets from that website appear in Charles. If so, you’re ready to sniff The Daily’s iPad app.

Indexing the Daily

1. Start the Daily app on the iPad. Wait for it to download today’s issue. In Charles, drill down to https://app.thedaily.com/ipad/v1/issue/current, and select “JSON Text.”

2. Copy and paste the raw JSON into a text file.

3. This Python script takes the JSON file as input, and spits out a snippet of HTML suitable for blogging. I simply pasted the output from that script into Tumblr, made a thumbnail of the cover, and published.

The End

So, that’s it! Hope that was helpful. If any fan of The Daily out there wants to take over publishing duties, I’ll happily pass the Tumblr blog on to you.

Anybody else think it’s weird that The Daily, News Corp’s new iPad-only magazine, posts almost every article to their official website… but with no index of the articles to be found? They spent $30M on it, but apparently forgot a homepage. (That’s a joke, people.)

So I went ahead and made one for them! Introducing, The Daily: Indexed…

Why did I do this? The Daily’s publishing free, web-based versions to every article, but without an index, it’s (deliberately) hard to find or link to the individual articles from the web. And since the iPad app only carries today’s edition, it makes finding any historical articles you’ve paid for nearly impossible.

I love that this kind of experimentation is happening in journalism. I love journalism dearly and want to see new models emerge, and charging for content is a great way to align a media organization’s interests with those of its readership. That said, if you do charge for access, you can’t publish free versions to the web and hope that people don’t find them.

I’m also very curious about their reaction. This isn’t illegal or a copyright violation — all I’m doing is linking to the versions they’re publishing on their site. The ability to link to any webpage without permission is part of what makes the web great, and it should never be discouraged. It’s also worth noting that Google’s slowly indexing all the articles too, and search engines aren’t blocked in their robots.txt file.

But I’m still recovering from a legal nightmare last year (more on that soon), so if asked to stop publishing and delete the Tumblr, I will. (Lawyers: My email address is at the top of this page.)

In the meantime, enjoy!

(Special thanks to Rex Sorgatz for the inspiration.)

Update: At The Daily’s press conference, editor-in-chief Jesse Angelo addressed the question of public sharing on the web.

“For the pages in the application that we can do it, we create mirror HTML pages. Those pages are out there on the web — they can be shared, they can be searched, you can find them out there… We know there are billions of other people sharing content on the web, and we want to be part of that.”

Thanks to Ryan Tate for the video.

February 4: Some people seem to have been a bit confused by my original post, so I edited it a bit, explaining a bit more clearly why I made this. I never thought that The Daily actually forgot to make a homepage/index; that was tongue-in-cheek. I also added a comment answering some of the frequently asked questions about the project.

Over the last few years, I’ve been collecting examples of metagames — not the strategy of metagaming, but playable games about videogames. Most of these, like Desert Bus or Quest for the Crown, are one-joke games for a quick laugh. Others, like Cow Clicker and Upgrade Complete, are playable critiques of game mechanics. Some are even (gasp!) fun.

Since I couldn’t find an exhaustive list (this TV Tropes guide to “Deconstruction Games” is the closest), I thought I’d try to pull one together along with some gameplay videos.

This is just a starting point, please post your additions in the comments or email me and I’ll add them in. Note: I’ve tried to stay away from specific game parodies (like Pong Kombat or Pyst), and stick to games that comment on game design, mechanics, or culture.

Continue reading “Metagames: Games About Games” →

Today, Netflix posted some interesting research, tracking the performance of their streaming service on the top ISPs in the U.S.

Sadly, the charts were completely useless to me — a pile of mostly-indistinguishable lines. Along with one out of every 14 American males (about 7%), I’m red-green colorblind.

This is hard for non-colorblind people to understand, so I pulled together a couple examples. Here’s a split comparison of the original chart, showing what people with normal vision see compared to me and my crappy eyes.

(Click to view large.)

Two simple solutions:

1. Label your lines. When you have more than three data points in a line chart, legends fall apart quickly whether you’re colorblind or not. A label next to each line makes any chart much more readable. Here’s a quick remake I whipped up. (Thanks to Greg for helping me get the colors right.)

(Click to view large.)

2. Pick colorblind-safe colors. If you have to use a legend, be kind and pick something people like us can see. Photoshop’s supported drop-dead simple colorblind simulation since CS4, or you can check your images or webpages for free using the Vischeck colorblind simulator.

When doing the right thing is this easy, it’s really disturbing when it’s dismissed as a waste of time.

A couple years ago, I contacted the husband-and-wife team behind Snopes, the essential resource on urban legends, to let them know about a similar issue. The red/green icons they use to indicate true/false urban legends looked absolutely identical to me. I let them know about the problem and prepared alternate GIFs for them, with a darker red and lighter green. (Incidentally, that’s why colorblind people don’t have trouble with stoplights.)

They not only refused the new images, but actually added a new entry to their FAQ, defending their position:

We chose our red-yellow-green coding system because its “traffic light” pattern can be understood by most of our readers with little or no explanation. While we understand that about 8% of our readership experiences some form of color blindness and therefore cannot distinguish the different colors of bullets, other alternatives we have tried have proved confusing to many of our non-color blind readers. Therefore, we have chosen to stick with a system that works very well for 92% of our readers.

Instead, they recommended hovering over every icon to see the tooltip text. I absolutely adore the work they do on Snopes, but that interaction’s left a sour taste in my mouth ever since. It just doesn’t seem defensible — is slightly darkening a shade of red and brightening a green too much to ask?

I wouldn’t expect anyone to be able to perfectly anticipate every person’s needs; accessibility is extremely hard to get right 100% of the time. But if your ultimate goal is conveying information, open ears and a little empathy can go a long way.

Update: Alex Bischoff took the three images I made for Snopes, and wrote a user script that replaces their images with mine. Install it here for your browser of choice.

The Oscar nominations were announced yesterday, which means it’s time again to see who’s winning in the eternal fight between the movie studios, the Motion Picture Academy, and the loosely-organized group of spunky kids known as The Scene.

Yesterday morning, along with an anonymous group of spectators, I updated the ever-growing spreadsheet now spanning the last nine years of Oscar-nominated film. I added this year’s 29 nominees to the list, a collection of 274 films in total. (You can read about more sources and methodology at the end of the entry.)

Don’t miss the Statistics sheet, which covers all the aggregate year-by-year stats. Download or view it below, or read on for my findings. As always, if you have any additions or corrections, let me know.

View full-size on Google Spreadsheets.

Download: Excel (with formulas) or CSV

Findings

Note: These numbers will change as we get closer to the ceremony, and I’ll do my best to keep them updated until Oscar day.

Continuing the trend from the last couple years, fewer screeners are leaking online by nomination day than ever. Last year at this time, only 41% of screeners leaked online; this year, that number drops again slightly to 38%.

But if you include retail DVD releases along with screeners, 66% of this year’s nominated films have already leaked online in high quality. This makes sense; if a retail DVD release is already available, there’s no point in leaking the screener. But I think it’s safe to say that industry efforts to watermark screeners and prosecute leaks by members have almost certainly contributed to the decline.

The gap between theatrical and DVD release dates seems to have stabilized, hovering around 105 days for the last few years. This year, the gap between US release to first leak seems to have dipped slightly, from a median 23 days last year to 17 days.

The chart below shows how camcorder and telesync leaks for Oscar-nominated films continue to decline in popularity, while nearly every nominated film is eventually leaked on DVD. (The only exception seems to be 2008’s Il Divo, which never appeared to get a US retail release.)

One prediction: The end of the DVD screener is near. This year, Fox Searchlight distributed three screeners with iTunes — 127 Hours, Black Swan, and Conviction — to all 93,000 voting members of the Screen Actors’ Guild, marking the first time a major studio’s used Apple’s service for screener distribution.

Voters get the additional convenience of being able to watch films on their computers, Apple TVs, iPads and iPhones, while studios save the time and expense of distributing physical media. If this experiment’s successful, it seems likely other studios will follow.

Miscellanea

Some random notes:

• This year, three films were leaked online within a day of their theatrical release — Iron Man 2, Alice in Wonderland, and Harry Potter.
• The Rabbit Hole screener was leaked online eight days before its theatrical release, while Winter’s Bone was the slowest to leak online (so far) at 125 days after its theatrical release.
• Oscar-nominated films tend to get released late in the year, but how late? More nominated films have been released on December 25 than any other day, but the median date is October 20.
• For the first year, the first high-quality leak of a film — Harry Potter and the Deathly Hallows — was a PPV rip, most likely from a hotel’s new movie releases on pay-per-view.
• Retail Blu-Ray rips are now frequently being leaked online now before retail DVDs, so I’ve modified the “Retail DVD” column to include them.

Methodology

As usual, I included the feature films in every category except documentary and foreign films (even makeup and costume design). I used Yahoo! Movies for US release dates, always using the first available date, even if it was a limited release. Cam, telesync, R5, and screener leak dates were taken from VCD Quality, with occasional backup from ORLYDB. I always used the first leak date, with the exception of unviewable or incomplete nuked releases.

Finally, the official screener dates came from Academy member Ken Rudolph, who lists the date he receives every screener on his personal homepage. Thanks again, Ken!

For previous years, see 2004, 2005, 2007, 2008 (part 1 and part 2), 2009, and 2010.

I’ve been dealing with a family illness, but couldn’t let the Wikileaks Cablegate incident pass without comment. In between hospital visits, I’ve been jotting down links related to the historic leak.

It’s a stunning experiment of forced transparency, prying open government against its will without much care or concern about the ramifications. Wikileaks is the Pirate Bay of journalism — an unstoppable force disrupting whole industries because they can.

To help make sense of my own opinions about it, I rounded up some of the more interesting responses and visualizations. Enjoy.

Continue reading “Wikileaks Cablegate Reactions Roundup” →