Twitter Favorites Bankruptcy

Update June 2013: This script no longer works and cannot be made to work easily. It uses basic password authentication, while Twitter now requires all write APIs to use token-exchange, which is far more complex to implement. This post is kept around for historical reasons only. No, I don’t know of another way to delete all of your old favorites. In fact, I’ve had a change of heart since writing this script – I no longer want to treat my old twitter favorites as a sort of “inbox” that needs emptying. They’re more like Facebook “Likes” – you wouldn’t go back and try and unlike everything you’ve ever liked, would you? My advice: Just leave them alone. They’re not hurting anything.

———-

I read Twitter primarily on the iPhone, and find tons of great links I want to read in a proper browser later on (I personally find reading most web sites on an iPhone to be more hassle than it’s worth). Perfect solution: Side-swipe an item in Tweetie and tap the star icon to mark it as a favorite. Later, visit the Favorites section at twitter.com to follow up.

Unfortunately, over the past couple of years I’ve favorited way more things than I’ll ever have time to read. As of now, I’ve got 1600 favorites waiting to be read. Ain’t never gonna happen. I declare Twitter Favorite bankruptcy! Needed a way to batch-unfavorite the whole collection, and twitter.com doesn’t provide a tool for that. Time to dive into the Twitter API.

I heart Python, and Tweepy is the best Twitter API tool for Python (documentation here). So start by getting Tweepy installed and on your Python path. If it’s installed correctly you should be able to type “import tweepy” in a python shell and get no errors.

Once installed, you’ll need to authenticate to Twitter. Note that we’re using basic authentication here. The “right” way is to use OAuth, but that requires registering your application with twitter.com – something users with this simple need won’t want to get into. Let’s explore.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

import tweepy
auth = tweepy.BasicAuthHandler('your_username', 'your_password')
api = tweepy.API(auth)
 
# Get recent items from your timeline:
posts = api.user_timeline('shacker')
[p.text for p in posts]
 
# Cool, it works. Now get the list of  favorites for the 
# authenticating user (you).
favs = api.favorites()
[p.text for p in favs]
 
# How many did you get?
>>> len(favs)
20
 
# Oops, that's only 20. But we need all 1600! Unfortunately, 
# you can only get a "page" of favorites at a time - which 
# is usually a batch of 20. We're going to have to cycle 
# through them. The docs say to use:
 
API.favorites([id][, page])
 
# So since page is the 2nd arg, we're going to need 
# your user ID in order to access pages. So find your user ID:
 
>>> api.get_user('your_username').id
14098497
 
# So 14098497 is your user ID. Now you can go back and 
# use the favorites API to get things in pages. 
# IOTW for page 2 of the results, use:
 
>>> favs = api.favorites('14098497', '2')
>>> [t.text for t in favs]
 
# So how do we un-favorite a tweet? The docs say:
# API.destroy_favorite(id)
# So if we instead list the tweet IDs of the result set:
 
>>> [t.id for t in results]
[4601622480, 4587326808, 4527173532, 4514186302, 4497778507, 4472272715, 4431075519, 4425332211, 4048507627, 4043010657, 4023672253, 4023524092, 4022466872, 4022334337, 4016169277, 4015641948]
 
# We can grab the first ID in the list and test with that. 
# Go to twitter.com, click on Favorites, find the first 
# tweet listed, and click the timestamp. The URL should 
# include the ID of the first tweet in the list. 
# Back in the shell, try and unfavorite it:
 
api.destroy_favorite(4601622480)
 
# Now refresh your Favorites page and notice that the 
# first tweet is gone from the list. Groovy. All we 
# have to do now is script the process.

Some of the tweepy methods provide a ‘cursor’ argument to help you iterate through pages, but the favorites() method does not. But we don’t really need one – we can simply iterate through pages until the results list comes back empty.

But there’s a hitch: Twitter’s basic authentication is limited to 150 API requests per hour, and each un-favorite counts as one request. If you have a lot of favorites like I did, you’ll end up hitting your API limit pretty quickly. The solution is to “sleep” the script between each page of results. The math says we should be able to sleep for 8-10 minutes per page and come in under the 150/hour limit. But in practice, I found myself hitting the limit anyway (browser and desktop client running at the same time skew the results, and we want to be safe). In other words, this is not a script that will complete in a few seconds – with a 15 minute pause between each cycle, you’ll want to run it in the background while you do other things. The 15-minute pause is designed to allow for enough spare cycles every hour that you can continue using Twitter normally. No rush, right? Chillax!

So save the following as a text file (in the same directory as tweepy.py if you don’t grok python paths), change the username and password at the top, and run it with python filename.py.

Warning: This script commits total bankruptcy on your favorites – it will remove ALL of them. Run it only if you’re sure you want to do that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

import tweepy
from time import sleep
 
username = 'your_username'
password = 'your_password'
 
auth = tweepy.BasicAuthHandler(username, password)
api = tweepy.API(auth)
userid = api.get_user(username).id
 
# Counter is used for reporting purposes only
i = 1
 
'''
Tweepy will return pages of favorites as lists.
In Python, an empty list is considered false, 
so we we don't need to know the number of result pages -
just keep going until the results list is empty. 
'''
 
# Initiate an infinite loop, to be broken when we run out of favorites.
while True: 
 
    results = api.favorites(userid)
    if results:
        print "Page %d exists. Unfavoriting %d tweets." % (i, len(results))
        for r in results:
            api.destroy_favorite(r.id) # Here's the money shot.
 
        # Rest for a while.
        print "Script is sleeping for 15 minutes to accommodate Twitter rate limits."
        print
        i += 1        
        sleep(900)       # 900 seconds = 15 minutes
    else:
        print "No more favorites. Buh-bye."
        break

By morning, you should have no favorites left. Enjoy the clean slate while it lasts!

Oh, by the way, I’m shacker on Twitter.

Update: If you have a lot of favorites, you may encounter some weirdness after running this script for a while. I found that after the first thousand un-favorite requests, Twitter started rate-limiting me down from 20 tweets per page to 14, then to 8, then 2, then 1. When it got to one, I tweaked the script to request one per minute instead of waiting 15 minutes between. At that rate, I was making only 60 requests per hour. But eventually, twitter started returning zero. And lo and behold, when I accessed twitter.com/favorites in a browser, I got a “Something is technically wrong” message from Twitter.

I logged out and then in again with another account and /favorites was working fine – they had basically made it impossible to access my favorites in any way from the account that had been making all the un-favorite requests. This was puzzling and troubling. I had taken great pains to play with the API by the stated rules, to not exceed the rate limits, and to be gentle on their system… but still they treated this script like it was a bad actor. If anyone from Twitter reads this and can explain what’s going on, I’d love to hear what I can do to make it better.

A post at TechCrunch explains that Twitter has been slashing its rate limits lately, which is probably related to the behavior I’m experiencing.

If you have fewer than a thousand favorites, this script should still work fine, with no ill effects.