Waxy.org is the sandbox of How often is Friendfeed hitting Flickr, and how many Friendfeed users are on Flickr?
We now have a glimpse into Monday's traffic, thanks to a snapshot provided by Kellan and Rabble's in their talk, Beyond Rest: Building Data Services with XMPP PubSub, presented earlier today at OSCON in Portland:
On July 21, 2008, Friendfeed hit Flickr 2.9 million times to get the latest photos of 45,754 users, of which 6,721 visited Flickr in that 24-hour period, and could have potentially uploaded a photo.
Three million requests for 6,000 updates. Clearly, polling isn't ideal. Don't miss the rest of the slides.
(Also, at its peak, Flickr is currently receiving 60 uploaded photos a second, "roughly 10 times the number of people born on Earth per second.")
7:46 AM
That is incredibly irresponsible.
12:12 PM
Definitely a problem. I wrote up some thoughts as other possible solutions at: http://joshua.schachter.org/2008/07/beyond-rest.html
12:32 PM
As Kellan and Rabble say, it's not FriendFeed's fault. They're not doing anything wrong... They're limiting polling to one request every 24 minutes for each photostream, and using etags and conditional GETs.