Telehack is the most interesting game I’ve played in the last year… a game that most users won’t realize is a game at all.

It’s a tour de force hack — an interactive pastiche of 1980s computer history, tying together public archives of Usenet newsgroups, BBS textfiles, software archives, and historical computer networks into a multiplayer adventure game.

Among its features:

• Connect to over 24,000 simulated hosts, with logged-in ghost users with historically-accurate names culled from UUCP network maps.
• Hacking metagames, using simplified wardialers and rootkit tools.
• User classes that act as an achievements system.
• Group chat with relay, and one-on-one chat with send or talk.
• Reconstructed Usenet archives, including the Wiseman collection.
• A BASIC interpreter with historical programs from the SIMTEL archives.
• Standalone playable games, including Rogue and a Z-code interpreter for text adventure games like Adventure and Zork.
• Hidden hosts and programs, discoverable only by hacking Telehack itself.

The entire project was engineered by “Forbin,” an anonymous Silicon Valley engineer named after the protagonist of Colossus: The Forbin Project. Like the chief engineer of the film, Forbin’s created a networked supercomputer that defies all expectations. (Hopefully it won’t gain sentience and enslave the human race.)

I had to know more. With the help of Paul Ford, I interviewed Forbin about the project — using Telehack’s send utility, naturally. Read on for the full interview about his motivations, how it’s built, and why he’s chosen to remain anonymous.

Andy Baio: So, first off, I want to tell you how much I’m in love with Telehack. You’ve made something truly unique… I’d love to hear about your inspiration for building it.

Forbin: Thank you. I’m glad that people are enjoying it. The inspiration was my son. I had shown him the old movies Hackers, Wargames, and Colossus: The Forbin Project and he really liked them. After seeing Hackers and Wargames, he really wanted to start hacking stuff on his own.

I’d taught him some programming, but I didn’t want him doing any actual hacking, so I decided to make a simulation so he could telnet to hosts, hack them, and get the feel of it, but safely.

What did he think?

He really liked it. At first he thought it was all real, and he was actually hacking into government computers and such. It was great. Eventually though, like Santa Claus, he figured out he wasn’t really wardialing all those systems.

He’s been my best beta-tester. 🙂

When do you think he’ll be ready for real-world hacking? Is there a path to graduation from Telehack?

Telehack can help you learn commandline basics. I’ve told him not to ever enter any real systems without permission.

I’m curious if you’ve played Digital: A Love Story. It seems like Christine Love was trying to do something similar — conjuring an earlier time in computing history, but without worrying too much about historical accuracy.

I played a bit of Digital: A Love Story. I thought it was wonderful. I really liked the atmosphere it evoked. That was the same effect I was going for in Telehack, but in a different way. Silent, no sound… Just green text and more code, but the same emotion.

Regarding historical accuracy, one of the surprising things about Telehack is how those old systems were hard to use. Whenever a movie or a book looks back at the past, it can look through a set of lenses that make the past seem more engaging and accessible, and sometimes add a narrative. Someone on Hacker News referred to Telehack as “MovieOS,” and that’s exactly right.

There are actual TOPS-10 systems on the net you can get accounts on. They’re not easy to get into. I wanted to reduce the usability barriers to the old commandline interfaces, while giving the same feel of the systems, and blend some of the good things I remembered from various systems together.

Telehack seems to borrow quite a bit from modern videogame theory… Integrated tutorials, slowly ramping up difficulty, multiple avenues to exploration without a linear path. Paul Ford suspected you’re a game designer.

I have done some computer games, although that is not my profession. I really admire the advances that have occurred in game design, although I’m not much of a gamer myself. Mostly, I wanted to help newcomers get across barriers of accessibility which is what the old tutorial manuals were all about.

Read an old DECsystem-20 manual. It tells you, in excruciating detail, how to type control-C to interrupt a program. It turns out people still need to know that but aren’t being told anymore.

The other part is that the system is open-ended. It has all this old code that is animated by resurrecting an interpreter for a dead language. People can run programs again that haven’t been run — and experienced — for 20 years. And see old files through a lens that makes them look like they used to. That’s fun.

Speaking of old files, the way you used the textfiles.com and Usenet archives feels brand new. You’ve created a playful environment for exploring archival material in a new way. It made me wonder what other data sources could be reinvented by making them game-like.

What Jason Scott did with textfiles.com is heroic. He’s saving away all this stuff that is completely unique, and irretrievable otherwise. But to make people want to see it, byte by byte, I thought it would be neat to offer it up, a piece at a time, in a format similar to how the files were originally experienced.

You dial some modem number — not busy! It actually connects and then you see what files they have and download some of them. Most of them are crazy stuff but period-relevant. So it’s a way to animate old text files.

Same thing with the Usenet archive, although my Usenet reader needs some work. It’s pretty crufty. There is so much in there, I haven’t really found a good way to get people back into it yet.

Folks would wait in anticipation for Usenet — the daily poll — where your modem would call some hub and get you the news. I have to find some way to bring that back to the archive.

Google Groups doesn’t give that feeling. Neither does Telehack’s usenet command currently. Still noodling on that.

Does it advance in real-time? Are you adding “new” articles to the archive in a rolling period?

No, although that’s a good idea for a way for it to be more dynamic and engaging.

You’ve been adding features incrementally since you launched it, but how long did it initially take you to build from conception?

If you type uptime, it says sysgen was 454d ago at 07-Mar-10 20:26:00. That’s when I started working on it. Not full-time, there were months when I didn’t have any time to work on it. It’s mostly been a small side project.

What was the hardest thing to get right?

There is one feature that works in the telnet interface, but not in the http client yet — the baud rate simulation. If you dial or wardial into a host and you’re connected to Telehack with telnet, it will actually give you a 2400 or 9600 or whatever connection, but that doesn’t work on the html interface yet sadly.

It’s not the same when you dial into a host and the text renders instantly. For the full authentic feel, you should have to wait for the lines to appear slowly, as we once did. 🙂

I keep my 300 baud VicModem on my desk as a reminder of how good we have it now.

300 baud was really slow! I started there too, with a Hayes on an Apple II. Type baud 300 on Telehack, it’s hard to see how we could use those systems, but we did. They were amazing, even at that speed.

I’d love to hear more about the technology behind it.

Telehack is built in Perl, in a single process, in an epoll-driven event loop. There are two interpreters — Z-code, which is an interpreter called Rezrov in CPAN, and a BASIC interpreter, which I wrote. Currently it doesn’t fork any external code. Various functions, including the 6502/VAX CPU and such, are simulated.

Any plans to open-source any of it? I’m sure some of the community would love to hack on Telehack, to extend it in different directions.

Yes, I plan to open it up at some point.

So, I have to ask: why the anonymity? The mystery definitely adds to the fun, but most people would love to take credit for such an impressive project.

Well, I have a day job, and I didn’t want this to be a distraction. I also made this for my kid, but didn’t really want to expose him to a larger internet just yet.

Are you worried that coming out from behind the curtain will bring attention to him too? I’m the parent of a six-year-old, and can definitely appreciate that.

At an earlier point in my career, I got a lot of press for a project I did. The intense interest from that made me very cautious about what I put online. I took down my personal photos and such. I would be sad if I felt like I had to take down Telehack.

Makes sense. What have you thought about the reaction? Like several others, when I first saw Telehack, I completely underestimated its depth. It seems like an deep rabbit hole that endlessly rewards curiosity.

I’m extremely happy that people are enjoying it. I was a bit sad that some commenters initially dismissed it as a simple JS shell. It was pretty cool when the first person found ptycon and the secret entry points in the system monitor.

With zero fans, I’d be pretty disappointed. With any n > 0, I’m happy. I don’t expect it’s a huge audience though. Doesn’t have to be. It’s an artistic/historical project to me.

You’ve done some incredible data archaeology here — reconstructing 25,000 hosts from the early Internet, along with the people that used them at the time. One guy on Hacker News was able to find himself and his two best friends at the time logged into the host he first used when he get online. How did you do it?

Well, a lot of this information is available online, but you need to look at the right way to interpret it.

Were you able to find yourself in the archives?

Yup, I’m in there.

Any specific rules for how you scattered the text and game files across hosts?

There is some topic-clustering for the text files, the rest is mostly random. But I’m still working on the game parts of this thing.

If Telehack ever takes off, would you ever consider doing it for a living? I have no idea what you do for a living, but I can’t help but think you’ve missed your true calling as a game designer.

Well, thank you for that. 🙂 I’ll have to get back to you on that. At this point, I’m mostly interested in fleshing it out, so I’m happy with it.

Telehack’s a pastiche of many different systems, networks, and tools from the mid- to late-1980s. It’s rich with nostalgia. Is there anything you miss from that era?

Good question. The commandline was a universal language. You had to learn it, there was a curve, but it wasn’t that hard. Heck, we were all being taught it in the new classes in middle school. But the GUI’s mission was to kill it.

My worry is that the CLI was symbolic, algebraic, whereas the GUI is… pictorial, or one-step, or something.

hosts | grep foo

It’s important that you understand that. If you’re graduating from any university today, it’s an algebra more important than… uh, algebra, maybe.

I actually don’t miss anything from that era. But I want the best of what was known then to propagate today.

This weekend, I was invited to participate in Rhizome’s Seven on Seven in NYC — an event that pairs seven artists with seven technologists, challenging them to create something in one day and present it to an audience the next day.

The other teams were a humbling roster of creative geeks, including Ricardo “mrdoob” Cabello, Ben Cerveny, Jeri Ellsworth, Zach Lieberman, Kellan Elliott-McRea, Chris “moot” Poole, Bre Pettis, and Erica Sadun.

I was paired with Michael Bell-Smith, whose digital art I’d admired and linked to in the past. It was a perfect match, and we’re very happy to announce the result of our collaboration: Supercut.org (warning: NSFW audio).

Supercut.org is an automatic supercut composed entirely out of other supercuts, combined with a way to randomly shuffle through all of the supercut sources.

The Idea

When we first started work on Friday morning, Michael and I started brainstorming what we wanted to accomplish: something visual, high-concept (i.e. explainable in a tweet), and hopefully with a sense of humor.

We quickly realized that our interest in supercuts was fertile ground. Michael’s work often touches on structural re-edits and remixes, such as Oonce-Oonce, Battleship Potemkin: Dance Edit, Chapters 1-12 of R. Kelly’s Trapped in the Closet Synced and Played Simultaneously, and his mashup album mixing pop vocals over their ringtone versions.

Both of us were fascinated by this form of Internet folk art. Every supercut is a labor of love. Making one is incredibly time-consuming, taking days or weeks to compile and edit a single video. Most are created by pop culture fans, but they’ve also been used for film criticism and political commentary. It’s a natural byproduct of remix culture: people using sampling to convey a single message, made possible by the ready availability of online video and cheap editing software.

So, supercuts. But what? Making a single supercut seemed cheap. I first suggested making a visual index of supercuts, or a visualization of every clip.

But Michael had a better idea — going meta. We were going to build a SUPERSUPERCUT, a supercut composed entirely out of other supercuts. And, if we had time, we’d make a dedicated supercut index.

Making the SuperSupercut

There were three big parts: index every supercut in a database, download all the supercuts locally, break each video into its original shots, and stitch the clips back together randomly.

Using my blog post as a guide, Michael added every supercut to a new table in MySQL, along with a title, description, and a category. While Michael did that, I wrote a simple script that used youtube-dl to pull the video files from YouTube and store them locally.

To split the clips, we needed to find a way to do scene detection — identifying cuts in a video by looking at movement between two consecutive frames. But we needed a way to do it from my Linux server, which ruled out everything with a GUI.

After some research, I found a very promising lead in an obscure, poorly-maintained French Linux utility called shotdetect. It did exactly what we needed — analyze a video and return an XML file of scene start times and durations.

The most challenging part of the entire day, by far, was simply getting shotdetect to compile. The package only had binaries for Debian and the source hadn’t been updated since September 2009. Since then, the video libraries it needed have changed dramatically, and shotdetect wouldn’t compile without locating them.

Three frustrating hours later, we called in the help of one of the beardiest geeks I know, Daniel Ceregatti, an old friend and coworker. After 20 minutes of hacking the C++ source, we were up and running.

With the timecodes and durations from shotdetect, we used ffmpeg to split each supercuts into hundreds of smaller MPEG-2 videos, all normalized to the same 640×480 dimensions with AAC audio. The results weren’t perfect — many scenes were broken during dialogue because of camera changes — but it was good enough.

As ffmpeg worked, I stored info about each newly-generated clip in MySQL. From there, it was simple to generate a random ten-minute playlist of clips between a half-second to three seconds in length.

With that list, we used the Unix `cat` utility to concatenate all the videos together into a finished supersupercut. We tweaked the results after some early tests, which you can see on YouTube.

While the videos processed overnight, we registered the domain, built the rest of the website, and designed our slides for Saturday’s event — taking time out for wonderful Korean food with the fellow team of Ben Cerveny and Liz Magic-Laser. I finally got to sleep at 5:30am, but I’m thrilled with the results.

The Future

There were several things we talked about, but simply didn’t have time to do.

I’m planning on using the launch of Supercut.org to finally retire my old supercut list by adding a way to browse and sort the entire index of supercuts by date, source, and genre. Most importantly, I’m going to add a way for anyone to submit their own supercuts to the index.

And of course, when any supercut is added, it will automatically become part of the randomized supersupercut on the homepage: an evolving tribute to this unique art form.

The newest beta of ThinkUp adds an API to the app for the first time, allowing developers to easily build apps on top of data coming from ThinkUp.

The JSON API was created by Sam Rose, a 20-year-old student from Wales and an active contributor in the ThinkUp community. His 7,000 line contribution — composed of 40% new tests and 40% documentation — earned him first place in the ThinkUp bounty contest and a brand new iPad 2. Congrats, Sam!

I thought it’d be fun to try building a hack with his new API, so I made a simple visualization of your entire Twitter archives in ThinkUp — ThinkBack, a ThinkUp-powered time capsule. Take a look at my history, or on the @whitehouse account to get the gist.

ThinkBack analyzes your entire Twitter history, extracts entities from the text, and colors them based on category. Grouped by month, it also gives you a quick glimpse at your posting activity over time.

For the entity extraction, I used an excellent free web service called AlchemyAPI to extract people, places, events, product names and other keywords from everything I’ve ever posted. They provide a category for each, which I assigned a color.

I also tested two other free web services that offer entity extraction, text-processing.com and Zemanta. Finding and categorizing keywords from short status updates is no small feat, but AlchemyAPI does a remarkable job. (If you’d like to play around with all three, support for both Zemanta and text-processing.com is commented out in the source code, but easily swappable with AlchemyAPI.)

ThinkBack also uses four typefaces from Google Web Fonts, my first time using them and dead simple to implement. For free fonts, the quality’s surprisingly great, with several faces commissioned by Google itself. For a quick, free hack, it’s a great alternative to Typekit.

I also used a very simple PHP templating language called RainTPL, which I chose as a lightweight alternative to Smarty. In practice, I found it too simple. Its handling of complex data structures and loops required me to jump through hoops that shouldn’t be necessary. (I’ll stick with Smarty next time.)

Anyway, you can download the code here, it only requires PHP and access to a recent version of ThinkUp. Feel free to fork it and submit a pull request for anything you add!

Every year, I think it’ll be my last, and every year, I keep going. Why? Because what makes SXSW Interactive special isn’t the panels, parties, BBQ, or endless free alcohol (though those all help). It’s the unique group of creative individuals that shows up in Austin every year — a wonderful mess of people, all stuck in the same city at the same time.

Those unique set of circumstances create a serendipity machine, and the people I meet each year keep me coming back, even as it busts at the seams. To that end, please track me down or say hi! That’s why I’m there. And here’s where you’ll find me:

Worst Website Ever II: Too Stupid to Fail

Monday, March 14 at 11am

Hilton, Salon D

After a three year absence, I’m bringing Worst Website Ever back to SXSW with an all-star lineup of designers, developers, and entrepreneurs. Join us as these very talented people pitch their worst website/app/startup ideas to a live audience in short, five-minute rounds.

This year, the lineup’s pretty amazing:

Gina Trapani (Lifehacker, Expert Labs)

Jeffery Bennett, (BetamaXmas, 2008’s runner-up with Image Search for the Blind)

Josh Millard, (Metafilter, viral musician)

Jonah Peretti, (Buzzfeed, Huffington Post)

Mike Lacher, (Wonder-Tonic, Wolfenstein 1-D, Geocities-izer)

Ze Frank (The Show, Star.me)

And like last time, it’s judged by a real VC, Rob Hayes from First Round Capital. Winner gets funded!

ThinkUp: Austin Meetup

Saturday, March 12 at 5pm

The Ginger Man, outdoor patio

Curious about ThinkUp, or want to meet the people behind it? I’ll be joining Gina Trapani, Amy Unruh, Jed Sundwall, and other developers/users of ThinkUp at our second SXSW meetup. Come on out! We’ll be in the back patio of The Ginger Man, if weather permits.

Stalk Me

This year, I’ll be tracking interesting sessions on Lanyrd, and evening events on Plancast. And, of course, I’ll be mentioning any unusually great activity in real-time over on Twitter.

See you there!

For the last three weeks, I’ve indexed The Daily. Now that my free trial’s up, I’ve had an intimate look at what they have to offer and, sad to say, I don’t plan on subscribing. As a result, I’m ending The Daily: Indexed, my unofficial table of contents for every article they published publicly.

I’m surprised and grateful that The Daily executive and legal team never tried to shut it down. On the contrary, when asked directly about it, publisher Greg Clayman said, “If people like our content enough to put it together in a blog and share it with folks, that’s great! It drives people back to us.” They seem like a nice bunch of folks, and I hope they succeed with their big publishing experiment.

But now that I’m ending it, I can finally address the most common question — how did I do it?

The Daily: Indexed is just a list of article headlines, bylines, and links to each article on The Daily’s official website. Anyone can grab the links from the Daily iPad app by clicking each article’s “Share by Email” button, but that would’ve taken me far too long. So, how to automate the process?

When you first start The Daily application, it connects to their central server to check for a new edition, and then downloads a 1.5MB JSON file with the complete metadata for that issue. It includes everything — the complete text of the issue, layout metadata, and the public URLs.

But how can you get access to that file? My first attempt was to proxy all of the iPad’s traffic through my laptop and use Wireshark to inspect it. As it turns out, The Daily encrypts all traffic between your iPad and their servers. I was able to see connections being made to various servers, but couldn’t see what was being sent.

Enter Charles, a brilliantly-designed web debugging proxy for Mac, Windows, and Linux. By default, Charles will listen to all of your HTTP network traffic and show you simple, but powerful, views of all your web requests. But it can also act as an SSL proxy, sitting in the middle of previously-secure transactions between your browser and an SSL server.

After grabbing the JSON, I was able to write a simple Python script to extract the metadata I needed and spit out the HTML for use on the Tumblr page. Here’s how to do it.

Configuring Charles

1. Download and install Charles on your desktop machine. On your iPad, navigate to http://charlesproxy.com/charles.crt to trust Charles’ SSL certificate.

2. For Mac users, start Network Utility to get your desktop’s local IP address. Start your iPad, make sure it’s on the same wireless network as your desktop, and go into Settings>Network>Wi-Fi. Select the wireless network, and click the right arrow next to it to configure advanced settings. Under “HTTP Proxy,” select “Manual.” Enter the IP address of your desktop for “Server” and enter in “8888” for the port.

3. Now, start Charles on your desktop and, on the iPad, try loading any website. You should see assets from that website appear in Charles. If so, you’re ready to sniff The Daily’s iPad app.

Indexing the Daily

1. Start the Daily app on the iPad. Wait for it to download today’s issue. In Charles, drill down to https://app.thedaily.com/ipad/v1/issue/current, and select “JSON Text.”

2. Copy and paste the raw JSON into a text file.

3. This Python script takes the JSON file as input, and spits out a snippet of HTML suitable for blogging. I simply pasted the output from that script into Tumblr, made a thumbnail of the cover, and published.

The End

So, that’s it! Hope that was helpful. If any fan of The Daily out there wants to take over publishing duties, I’ll happily pass the Tumblr blog on to you.