Just Like That – Twitter Makes Move to Cut Down on Automated Follower Scraping
I was going through my list of Twitter followers tonight to see who I needed to follow back and weed out some of the less interesting people, when I noticed a change:
See how the number there is now a big integer, instead of a page number like 1, 2, 3, 4, etc.?
Why This Matters to Spammers & Hackers
Have you ever visited the Twitter page of one of your major competitors (company, personal brand, whatever) and noticed that they have a big chunk of their following list that closely resembles who is following you? Odds are, if they’re smart, they went and followed a lot of the people who you were following (or were following you depending on what they’re trying to accomplish). They game the system by simply unfollowing anyone who doesn’t follow them back, “purging” people who are uninterested if you will to keep the ratio of followers to following balanced.
Despite the fact that it is generally frowned upon as bad Twitter ettiquette, a lot of services are selling exactly this so it will be interesting to see how they innovate around this change. The reality is that Twitter probably should have included obfuscation of all ordered pages long ago to prevent spam and auto-mated scripting on their site outside of the API.
How Much Does it Really Slow Spammers Down?
You can still page through a person’s following or followers manually, and it looks like the page numbers are not dynamically generated so running a script to click the “Next” button and collect the page IDs for a user who you want to scrape relationships from wouldn’t be that hard. But it is interesting to notice these small changes Twitter is making to enhance security by making it a bigger pain in the ass to scrape than it was before.
One Comment
Robert von Heeren
Hi Danielle,
interesting change indeed. Everything you wrote is true, but there is another aspect to it: to me these figures appear to be random. If so it would be some caching prevention. Since I subscribed to twitter in July the followers count is far above the true ones I can see. Twitter had some status message since beginning of August that they are aware of this problem. Sometimes my followers count decreases although I see no changes in the list (noone appeared to unfollow), so my educated guess is, that’s also a caching problem on twitters side (they have to cache such lists, don’t they). A random URL-parameter would prevent browsers and servers from caching. At least the figure changes from page to page.