How has the record industry changed in the last 50 years? Using the Whitburn Project spreadsheet I talked about yesterday, I’ve been trying to dig into some of the underlying trends. Today, I’ll be tackling the longevity and diversity of pop songs, and a look at which decades had one-hit wonders.

Longevity of a Pop Song

One of the trickier questions I’ve been trying to visualize is how long pop songs are staying on the charts relative to the past. Are they staying on the charts longer than in the past?

In the chart below, I plotted the total number of weeks charted for all 23,924 songs that appeared on the Billboard Hot 100 from 1957 to earlier this year. (In other words, a little dot on the “60” line means there was a song released that week that stayed on the Hot 100 chart for 60 weeks.)

See the heavy dropoff on the 20th week starting in 1991? In an attempt to increase diversity and promote newer artists and songs, Billboard changed their methodology, removing tracks that had been on the Hot 100 for twenty consecutive weeks and slipped below the 50th position. These songs, called “recurrents,” were then moved to their own chart in 1991, the Hot 100 Recurrent.

Unfortunately, this shift makes it much harder to compare the last 15 years to the decades before it. In the chart below, I’ve isolated the effect by only showing songs that reached the top 50.

A couple interesting observations… Looking at the very bottom of the chart, you can see that in the last couple years, it’s become very common for a single to appear in the Top 50 and fall out of the Hot 100 within four weeks. Prior to the mid-1990s, this almost never happened.

Also, songs are staying in the Top 50 for far longer than they used to. Unfortunately, I don’t have any actual sales numbers to compare to, so it’s hard to say if these 30-70 week singles are massive megahits eclipsing the #1 singles of the past, or if it’s because the record industry is producing fewer hits than before.

Diversity

Did Billboard’s methodology changes in 1991 make the charts more diverse, like they hoped? By looking at the total number of unique songs that have charted yearly, it’s clear their changes did nothing to slow the decline.

According to Billboard, the late 1960s were the peak of musical diversity in popular music, with 743 different songs appearing on the 1966 Billboard Top 100. It’s fallen consistently since, hitting an all-time low in 2002 with only 295 songs. Since then, it’s improved only slightly, with 351 unique songs appearing on last year’s Top 100.

One Hit Wonders

I’ve always thought the 1970s were the decade of the one-hit wonder, but now I have the data to see for sure.

In raw numbers, the 1960s had more one-hit wonders than any other decade, followed closely by the 1950s. But that’s not entirely fair since, as we saw earlier, there were simply more unique songs on the 1960s charts. To find out the true numbers, we need to look at the number of one-hit wonders as a percentage of all songs in the Top 100.

This tells a totally different story. The 1960s, 1970s, and 1980s all had about the same ratio of one-hit wonders to hits by more established artists. The big surprise for me is that 1950s, 1990s, and 2000s really seem to be the eras where one-hit wonders dominated the charts.

Joshua Porter was wondering about the longest-charting one-hit wonders of all time. The longest-charting one-hit wonder to hit the #1 spot is Daniel Powter’s “Bad Day” from 2006, which stayed on the charts for 32 weeks. The one-hit wonder that stayed at the #1 longest is Anton Karas’ “The Third Man Theme” from 1950, which stayed in the #1 position for 11 weeks. Finally, the longest-charting one-hit wonder to appear anywhere in the Top 100 is Duncan Sheik’s “Barely Breathing” from 1997, which peaked at #16 but stayed in the top 100 for 55 weeks.

Have any other questions about the data, or done any analysis yourself? I’d love to hear about it.

May 20: Don’t miss Mike Frumin’s chart of pop longevity,from 1998-2002.

May 21: Using the Whitburn data, Tom Whitwell generated a tag cloud showing the top 100 commonly-used words in song names. Dianne Warren should write a #1 hit called “Love my Baby Blue Heart: A Girl’s Night Song.”

July 14: Pedro did some additional analysis, including artists with multiple hits in the same week and one-week wonders.

For the last ten years, obsessive record collectors in Usenet have been working on the Whitburn Project — a huge undertaking to preserve and share high-quality recordings of every popular song since the 1890s. To assist their efforts, they’ve created a spreadsheet of 37,000 songs and 112 columns of raw data, including each song’s duration, beats-per-minute, songwriters, label, and week-by-week chart position. It’s 25 megs of OCD, and it’s awesome.

As far as I know, this is the first time the project and its data have ever been discussed outside of Usenet. Despite its illegality, they’ve created a wonderful resource and you can do some fun things with the data. For the next three days, I’m going to publish some analysis and insights gleaned from their work. Update: I published an entry about one-hit wonders and pop longevity.

Continue reading “The Whitburn Project: 120 Years of Music Chart History” →

Continuing the series of portrayals of technology from the VHS era, here’s another contribution from Martin Brewer, who brought us the first episode of The Net. This is another BBC Two show, an episode of Horizon from 1993 on “The Electronic Frontier.”

The broad theme of the episode is the information economy, and it’s a whirlwind tour of influential people and tech in the pre-Web 1990s. They focus heavily on Microsoft, Apple, and General Magic, with interviews from key players from each. (Plus, great footage of their headquarters and workspaces.) There’s plenty of footage of vintage ’90s tech in here, including giant cell phones, Windows NT 3.1, the Newton, General Magic’s Magic Cap, Corbis, Encarta on CD-ROM, interactive TV and software agents. Some of the highlights, with screenshots, after the video below.

Note: Like all the other materials I post here, this video is completely out-of-print and unavailable commercially, digitized from an old VHS tape. If it ever comes back into print, or the copyright holders contact me, I’ll take it down immediately. (Update: YouTube is blocking my upload in some countries, so I replaced the video embed below with this copy from the Internet Archive.)

Continue reading “BBC Two's Horizon on "The Electronic Frontier" in 1993” →

For the last few weeks, I’ve been spending every Friday with a small group of brilliant geeks — Rael Dornfest, Asha Dornfest, Chris Anderson, Greg Borenstein, and Adam Greene — for a weekly one-day hackathon. We call it Bottlecap Labs, a place to bounce around ideas, support each other’s work, and occasionally whip up a project.

The first is ZombieURL, a TinyURL-clone with a twist.

Put in a URL, send it to a friend, and enjoy. Keep a camera nearby to capture the fun. (Here’s a Zombie-fied version of Waxy, for example.)

(Warning: We’re not responsible for emotional scarring, concussions, or heart failure as a result of using ZombieURL.)

From an anonymous source close to the company, I’ve found myself in possession of the “Infocom Drive” — a complete backup of Infocom’s shared network drive from 1989. This is one of the most amazing archives I’ve ever seen, a treasure chest documenting the rise and fall of the legendary interactive fiction game company. Among the assets included: design documents, email archives, employee phone numbers, sales figures, internal meeting notes, corporate newsletters, and the source code and game files for every released and unreleased game Infocom made.

For obvious reasons, I can’t share the whole Infocom Drive. But I have to share some of the best parts. It’s just too good.

So let’s start with the most notorious — Milliways: The Restaurant at the End of the Universe, the unreleased sequel to Infocom’s The Hitchhiker’s Guide to the Galaxy. For the first time, here’s the full story: with never-before-seen design documents, internal emails, and two playable prototypes. Sit back, this might take a while.

Note: I’ve pieced together this history from emails and notes from the Infocom Drive. I haven’t contacted any of the people mentioned, so if you’re a primary source or authority, please get in touch so I can make corrections.

Update: Don’t miss the comments section. Infocom alumni Dave Lebling, Steve Meretzky, Amy Briggs, and Tim Anderson all comment on the story, Zork co-author Marc Blank helps correct an error, and writer Michael Bywater provides an alternative view of the events.

November 26, 2009:

After much consideration, I’ve edited this entry to remove the private correspondence that I never sought permission to publish. In my eagerness to share the story, I rushed to post instead of taking the time to reach out to everyone involved, against the recommendations of my source. At the time, I felt that 20 years of distance and the material’s historical value was enough to warrant bringing it to light without contacting each individual. I was wrong, and I’d like to apologize to each of the Infocom employees for my lapse in judgment.

On these pages, I’ve frequently waded into murky legal waters and I’m used to catching a certain amount of heat. But this article cost me far more than I ever anticipated — I lost a very close friend because of it. I deeply regret the error.

— Andy.

Continue reading “Milliways: Infocom's Unreleased Sequel to Hitchhiker's Guide to the Galaxy” →