How often is Friendfeed hitting Flickr, and how many Friendfeed users are on Flickr?

We now have a glimpse into Monday’s traffic, thanks to a snapshot provided by Kellan and Rabble’s in their talk, Beyond Rest: Building Data Services with XMPP PubSub, presented earlier today at OSCON in Portland:

On July 21, 2008, Friendfeed hit Flickr 2.9 million times to get the latest photos of 45,754 users, of which 6,721 visited Flickr in that 24-hour period, and could have potentially uploaded a photo.

Three million requests for 6,000 updates. Clearly, polling isn’t ideal. Don’t miss the rest of the slides.

(Also, at its peak, Flickr is currently receiving 60 uploaded photos a second, “roughly 10 times the number of people born on Earth per second.”)

With its vibrant oversized photographs and minimalist design, the Boston Globe’s The Big Picture weblog launched on June 1 to instant global acclaim. It’s designed, programmed, and written by Alan Taylor, an old-school web programmer and blogger, in his spare time while working on community features at Boston.com. (You might know Alan from his popular MegaPenny Project, Amazon Light, or his other projects.)

The idea’s simple, but extremely effective. Spend a few minutes with the Iowa floods, the faces of Sudan, or the daily life in Sadr City, and you feel like you’ve opened a window to another world.

I interviewed Alan about the inspiration for the site, his methodology, and what it’s like being a programmer in a journalist’s world.

The Big Picture’s become an essential read for me, and I totally agree with Jason Kottke when he called it “the best new blog of the year.” What inspired it?

Alan Taylor: Lots of things — my parents used to always have Life and National Geographic magazines around the house, I fell in love with the visual storytelling way back then. When I was getting my feet wet in the online journalism world as a developer at msnbc.com, I had the good fortune of working alongside Brian Storm and a few others in MSNBC’s photo department, who were just phenomenal as far as selection, editing and presentation.

I wondered why other sites didn’t reach that level. Many have by now, but I was still frustrated by the presentation — either far too small, or trapped in click-after-click interfaces that were in Flash or just acted as ad farms.

Continue reading “Interview with Alan Taylor, Creator of Boston Globe's The Big Picture” →

In honor of the release of Firefox 3.0, I’m offering up a video that documented its very beginning in 1998 — the first open-source release of Netscape’s browser and the foundation of the Mozilla project.

Independent filmmakers followed the Mozilla team from March 1998 to April 1999, as they worked to open Netscape Communicator’s source code to the world, in a last-ditch effort to save the company. The result is an amazing snapshot of computer history, capturing the people that worked on it, the first internal beta test, the moment Jamie Zawinski uploaded the first builds publicly, the launch party, the all-hands meeting announcing the AOL acquisition, and so much more. It aired on PBS nationally in March 2000, the same month as the beginning of the dot-com collapse.

Out-of-print and never released on DVD, the used VHS copies start at $50 on Amazon. Like all the videos I release on Waxy.org, this material is commercially unavailable. If they ever come back into print, or the copyright holders contact me, I’ll take them down immediately.

Important Update (September 16): At the request of the the director, I’ve removed the video from Waxy.org and Viddler. I’ve interviewed the director about his plans for releasing the film and the unreleased footage.

Update (July 31): The documentary is back online, legally released under a Creative Commons license.

I’ve done my best to annotate the video, but many people in the film aren’t identified. I’ve left Viddler annotations open to everyone, so if you want to identify the people, places, or notable objects/events/trivia in the film, then please add your inline comments the video! (Or IM/email me and I’ll take care of it.)

The video’s now offline, but I’ve saved all the annotations. Thanks to Tman for creating the subtitle file, which can be used in video players like Media Player Classic or VLC, or simply viewed as plain text.

Now go download Firefox 3.0 and help make history!

Interviews and Appearances

• Jamie Zawinski: Left Netscape on April 1, 1999, now the owner of DNA Lounge in San Francisco
• Jim Barksdale, CEO
• Michael Toy
• Jim Roskind
• Tara Hernandez: Now an infrastructure engineer at Pixar
• Scott Collins: Now works on the Slashdot engineering team
• Jeff Weinstein
• Marc Andreessen
• Stuart Parmenter (and his parents)
• Brendan Eich: CTO at Mozilla
• David Readerman, Tech Analyst
• Po Bronson, Wired Magazine
• Kara Swisher, Wall Street Journal
• Gregg Zachary, Wall Street Journal
• Ellen Ullman, Author of Close to the Machine

Here’s the fifth and final episode of The Machine That Changed the World, this one focusing on global information networks including the Internet, and the communication benefits and privacy risks they create. This is the most familiar material of the documentary, so I’m going to skip the notes and annotations this time. I hope you enjoyed the documentary as much as I did.

And, as promised, here’s the BitTorrent file for high-resolution copies of all five videos. It’s a 3.1GB download with five H.264 encoded MP4 files. (If you only want a single video, use your BitTorrent client to select only the videos you need.) Enjoy!

Watch them all: Part 1, Part 2, Part 3, Part 4, Part 5

Interviews:

Robert Lucky (AT&T Bell Labs), Dave Hughes, Kathleen Bonner (Trader, Fidelity), George Hayter (Former Head of Trading, London Stock Exchange), Ben Bagdikian (UC Berkeley), Arthur Miller (Harvard Law School), Forman Brown (songwriter, died in 1996), Tan Chin Nam (Chairman, National Computer Board of Singapore), B.G. Lee (Minister of Trade and Industry, Singapore), Lee Fook Wah, (Assistant Traffic Manager, MRT Singapore), David Assouline (French Activist, now a senator), Mitch Kapor (founder, Lotus), Michael Drennan (Air traffic controller, Dallas-Fort Worth)

The fourth episode of The Machine That Changed the World covers the history of artificial intelligence and the challenges that come from trying to teach computers to think and learn like us.

Continue reading “The Machine That Changed the World: The Thinking Machine” →

The third episode of The Machine That Changed the World covers the development of the personal computer and the modern graphical user interface, which made computing easy to use for everyone. Highlights include interviews with Apple’s Steve Jobs and Steve Wozniak, drawing with a computer in 1963, great footage from Xerox PARC, and some 1992-era predictions of the future from Apple and others.

Continue reading “The Machine That Changed the World: The Paperback Computer” →

The first part of The Machine That Changed the World covered the earliest roots of computing, from Charles Babbage and Ada Lovelace in the 1800s to the first working computers of the 1940s. The second part, “Inventing the Future,” picks up the story of ENIAC’s creators as they embark on building the first commercial computer company in 1950, and ends with the moon landing in 1969 and the beginning of the Silicon Valley.

Watch them all: Part 1, Part 2, Part 3, Part 4, Part 5

Notes:

Shortly after the war ended, ENIAC‘s creators founded the first commercial computer company, the Eckert-Mauchly Computer Corporation in 1946. The early history of the company’s funding and progress is told through interviews and personal home videos. They underestimated the cost and time to build UNIVAC I, their new computer for the US Census Bureau, quickly sending the company into financial trouble. Meanwhile, in London, the J. Lyons and Co. food empire teamed up with the EDSAC developers at Cambridge to build LEO, their own computer to manage inventory and payroll. It was a huge success, inspiring Lyons to start building computers for other companies.

The Eckert-Mauchly company was in trouble, with several high-profile Defense Department contracts withdrawn because of a mistaken belief that John Mauchly had Communist ties. After several attempts to save the company, the company was sold to Remington-Rand in 1950. The company, then focused on electric razors and business machines, gave UNIVAC its television debut by tabulating live returns during the 1952 presidential election. To CBS’s amazement, it accurately predicted an Eisenhower landslide with only 1% of the vote. UNIVAC soon made appearances in movies and cartoons, leading to more business.

IBM was late to enter the computing business, though they’d built the massive SSEC in 1948 for scientific research. When the US Census ordered a UNIVAC, Thomas Watson, Jr. recognized the threat to the tabulating machine business. IBM introduced their first commercial business computers in 1953, the mass-produced IBM 650. While inferior technology, it soon dominated the market with their strong sales force, relative affordability, and integration with existing tabulating machines. In 1956, IBM soared past Remington-Rand to become the largest computer company in the world. By 1960, IBM captured 75% of the US computer market.

But developing software for these systems often cost several times the hardware itself, because programming was so difficult and programmers were hard to find. FORTRAN was one of the first higher-level languages, designed for scientists and mathematicians. It didn’t work well for business use, so COBOL soon followed. This led to wider adoption in different industries, as software was developed that could automate human labor. “Automation” become a serious fear, as humans were afraid they’d lose their jobs to machines. Across the country, companies like Bank of America (with ERMA) were eliminating thousands of tedious tabulating jobs with a single computer, though the country’s prosperity and booming job market tempered some of that fear.

In the ’50s, vacuum tubes were an essential component of the electronics industry, located in every computer, radio, and television. Transistors meant that far more complex computers could be designed, but couldn’t be built because wiring them together was a logistical nightmare. The “tyranny of numbers” was solved in 1959 with the first working integrated circuit, developed and introduced independently by both Texas Instruments and Fairchild. But ICs were virtually ignored until adopted by NASA and the military for use in lunar landers, guided missiles, and jets. Electronics manufacturers soon realized the ability to mass-produce ICs. Within a decade, ICs cost pennies to produce while becoming a thousand times more powerful. The result was the birth of the Silicon Valley and a reborn electronics industry.

Interviews:

Ted Withington (network engineer, industry analyst), Paul Ceruzzi (Smithsonian), J. Presper Eckert (ENIAC co-inventor, died 1995), Morris Hansen (former US Census Bureau, died 1990), John Pinkerton (Chief Engineer, LEO, died 1997), Thomas J. Watson, Jr. (Chairman Emeritus, IBM, died 1993), James W. Birkenstock (retired Vice President, IBM, died 2003), Jean Sammet (programming language historian), Dick Davis (retired Senior V.P., Bank of America), Robert Noyce (co-inventor, integrated circuit, died 1990), Gordon Moore (former Chairman of the Board, Intel), Steve Wozniak (Co-founder, Apple)

Up Next…

Part 3: The Paperback Computer. The development of the personal computer and user interfaces, from Doug Engelbart and Xerox PARC to the Apple and IBM PCs.

The Machine That Changed the World is the longest, most comprehensive documentary about the history of computing ever produced, but since its release in 1992, it’s become virtually extinct. Out of print and never released online, the only remaining copies are VHS tapes floating around school libraries or in the homes of fans who dubbed the original shows when they aired.

It’s a whirlwind tour of computing before the Web, with brilliant archival footage and interviews with key players — several of whom passed away since the filming. Jointly produced by WGBH Boston and the BBC, it originally aired in the UK as The Dream Machine before its U.S. premiere in January 1992. Its broadcast was accompanied by a book co-written by the documentary’s producer Jon Palfreman.

With the help of Simon Willison, Jesse Legg, and (unofficially) the Portland State University library, we’ve tracked down and digitized all five parts. This week, I’m uploading them, annotating them with Viddler, and posting them here as streaming Flash video as they’re finished. Also, the complete set is available for download as high-quality MP4 downloads via BitTorrent.

Here’s the first of the five-part series, The Machine That Changed the World. Enjoy!

Watch them all: Part 1, Part 2, Part 3, Part 4, Part 5

Continue reading “The Machine That Changed the World: Giant Brains” →

Back in March, I posted the first episode of a BBC Two series called The Net from 1994. It’s a great time capsule of how the media portrayed cyberculture in the early 1990s, very much like a TV version of early Wired Magazine.

Thanks again to Martin Brewer, who also contributed the Horizon show, here are two more episodes from the first season of The Net from 1994.

Continue reading “BBC Two's "The Net," Episodes 2 and 5 from 1994” →

This Friday, I’ll be speaking at the Webvisions conference in Portland about Internet memes, how they spread, and how their distribution’s changed over time.

As part of that research, I’ve been digging into my original server logs from the Star Wars Kid debacle, five years after I played a major role in what some say is the biggest viral video of all-time.

Be warned, this is more detail than you’ll ever want about the origins of the Star Wars Kid meme and how it spread. You don’t care about this level of detail, but I’m writing this all down so that I never have to think about it again.

In addition, I’ve decided to release the first six months of server logs from the meme’s spread into the public domain — with dates, times, IP addresses, user agents, and referer information. (Download it below.)

Early Origins

Like I mentioned in my original entry, the video was first released by Ghyslain’s schoolmates to Kazaa on April 19, 2003 with the original filename “ghyslain_razaa.wmv.” Within three days, it was being passed around in the offices of Raven Software in Madison, Wisconsin, where a game developer named Bryan Dube posted it on his personal website on April 22. Two days later, he created the first Star Wars Kid remix, adding lightsabers and sound effects in a new video titled “TheLastHope.avi.”

On April 27, a mostly-NSFW online community called Sensible Erection linked to the video on Bryan’s website. Later that evening, an SE user cross-posted it to a private file-sharing community I belong to with the new filename “star_wars_guy.wmv.” It quickly became the most popular file on the site, which is where I found it the following day, April 28 at 7:52pm.

On April 29, I renamed it Star_Wars_Kid.wmv and posted it to my site at 4:49pm — inadvertently giving the meme its permanent name. (Yes, I coined the term “Star Wars Kid.” It’s strange to think it would’ve been “Star Wars Guy” if I was any lazier.) An hour later, Scott Gowell becomes the first person to link to the video.

From there, for the first week, it spread quickly through news site, blogs and message boards, mostly oriented around technology, gaming, and movies. Throughout the life of the meme, most of the referers are blank, suggesting people were primarily sending the links by email or instant message.

The chart below shows the distinct top-level domains that appeared in the referral logs grouped by day.

It’s worth noting that the majority of sites sent less than 10 referers in that first month, and 21% of domains referred only one user. (Note: The chart below is on a logarithmic scale for both axes.)

Mainstream News Coverage

Here’s some of the highlights from the mainstream media coverage. The New York Times was the first major paper to report on it, almost a week after I tracked Ghyslain down, Jish and I interviewed him for the first time, and we started the fundraiser.

May 19, New York Times

May 19, Wired News

May 20, Public Radio International’s “The World” (radio program)

May 20, Globe and Mail

May 20, National Post

May 23, The Mirror UK

Jun 6, LA Times

Jul 4, The Independent UK

Jul 12, The Age

Jul 23, Wired News

Jul 25, BBC News

Jul 31, NPR w/Tavis Smiley (radio interview)

Aug 21, USA Today, syndicated Associated Press article

Aug 25, NBC’s Today Show (TV program)

Aug 26, MSNBC’s Countdown (TV program)

Aug 28, USA Today

Aug 30, Seattle P-I

Sep 8, SF Chronicle

Sep 15, Variety

Sep 16, Globe and Mail

Nov 18, CBS Evening News

Statistics

Here’s what the Star Wars Kid meme did to my overall traffic. At its peak, I received almost a million pageviews in a single day.

That includes all pageviews on my weblog entries. Isolating only the video downloads from my site, or later redirected to one of the mirrors, gives the following chart.

Download the Data

This file is a subset of the Apache server logs from April 10 to November 26, 2003. It contains every request for my homepage, the original video, the remix video, the mirror redirector script, the donations spreadsheet, and the seven blog entries I made related to Star Wars Kid. I included a couple weeks of activity before I posted the videos so you can determine the baseline traffic I normally received to my homepage.

The file is 158 megabytes — 1.6GB uncompressed — so I’m distributing it with BitTorrent. The data is public domain. If you use it for anything, please drop me a note!

Download: star_wars_kid_logs.zip.torrent