For the last two days, Elon Musk has claimed that Twitter is under attack from “several hundred organizations” who were conducting “EXTREME levels of data scraping,” forcing them to bring “large numbers of servers online on an emergency basis” and enact emergency measures.
Yesterday, Twitter started blocking all logged-out access to Twitter, requiring signing in to view any tweet or profile. Elon Musk called it a “temporary emergency measure,” claiming they “were getting data pillaged so much that it was degrading service for normal users!”
Apparently, it didn’t stop the crush of traffic and, this morning, Musk announced they escalated their actions against supposed “extreme levels of data scraping” by rate-limiting the number of tweets you can view.
Immediately, Twitter users started seeing “Rate Limit Exceeded” messages and every trending topic was about the collapse of Twitter:
Are shadowy AI companies scraping Twitter for training data? Maybe!
But on Mastodon this morning, web developer Sheldon Chang noticed another source of unusual traffic: a bug in Twitter’s web app that is constantly sending requests to Twitter in an infinite loop:
This is hilarious. It appears that Twitter is DDOSing itself.
The Twitter home feed’s been down for most of this morning. Even though nothing loads, the Twitter website never stops trying and trying.
In the first video, notice the error message that I’m being rate limited. Then notice the jiggling scrollbar on the right.
The second video shows why it’s jiggling. Twitter is firing off about 10 requests a second to itself to try and fetch content that never arrives because Elon’s latest genius innovation is to block people from being able to read Twitter without logging in.
This likely created some hellish conditions that the engineers never envisioned and so we get this comedy of errors resulting in the most epic of self-owns, the self-DDOS.
Unbelievable. It’s amateur hour.
He posted a video of the bug in action, sending hundreds of requests a minute.
On Twitter, software engineer Nelson Minar independently reproduced the bug with his own video capture.
It’s currently unclear when this bug went into production, or how much it’s actually impacting their traffic, so it’s hard to determine whether this bug inadvertently inspired Twitter to block unregistered access and add rate limits, or if the bug was triggered by the rollout of those changes.
On Bluesky, Twitter’s former head of trust and safety Yoel Roth wrote, “For anyone keeping track, this isn’t even the first time they’ve completely broken the site by bumbling around in the rate limiter. There’s a reason the limiter was one of the most locked down internal tools. Futzing around with rate limits is probably the easiest way to break Twitter.”
Sheldon suspects the bug was related to yesterday’s decision to block unregistered users from accessing Twitter, but in a followup, wrote that it’s “probably not the cause of their scraping panic and most of these requests are being blocked.”
It seems very likely that killing free access to the Twitter API led to a big increase in scraping, since countless businesses, organizations, and individuals used it for their projects. It’s also plausible that these issues are entirely unrelated.
Still, how funny would it be if this “emergency,” from start to finish, was brought on by a Javascript bug that caused Twitter to DDOS itself, spawning all of these truly terrible decisions? At this point in Twitter’s downward spiral, nothing would surprise me.
If you know more, leave a comment or get in touch. Confidentiality guaranteed.
This isn’t entirely new. Searching Twitter via Duck Duck Go stopped working almost as soon as he took over, for instance. You just got a login screen.
Someday someone will write a history of Twitter as one of the greatest follies the world has ever seen. It only seems fitting that the biggest jester in the world should be running it right now.
I think this is a secondary issue. The primary issue is Twitter’s contract with Google Cloud ended on 30 June, which Musk didn’t want to renew.
I thought that might be related, but Bloomberg reported on June 21 that they’d settled their Google Cloud bill and continued services.
I noticed issues starting early evening yesterday (Friday), when I couldn’t get more than 24 hours worth of any account’s tweets to load. By this morning, that had extended to my *own* account, where I could only see tweets and responses from the following day. Then came the error message, but it was only on the mobile app. Then on web, then I couldn’t see *any* of my own tweets, just my banner and bio. People are responding to my tweets, but I’m not always sure which one, because I can only see their response, not the originating tweet (that’s been a bit buggy for a while). I have no doubt this is tied to their installing walls around Twitter, one of the most incredibly stupid ideas they’ve ever had. Twitter isn’t Facebook. Many professionals of all stripes have rich communities, and being able to search via Google for certain users or tweets can be superior to using Twitter’s built-in search. This has been one of the biggest, dumbest clusters I’ve ever seen, even after working in web and mobile products for a *long* time. I’m guessing their QA dept. is down to two people and an overworked hamster.
Musk was bragging about usage seconds being at its highest ever. I wonder if this bug was giving him that bad data?
I wondered the same thing, though he said it was last week’s activity.
FWIW, a React useEffect callback will fire every time your data changes (if you specify said data as a dependency). Put a server request inside that callback, and it will change the data — which will cause React to fire off the callback again, which will cause another server request, and another data redefinition, and another callback firing, etc., etc.
It’s a death spiral of re-rendering that I wish I could say I’ve never accidentally created.