Tuesday, July 26, 2022

Twitter Removed the Blocked Account Export - Let's Put it Back!

I'm reposting my post from Stack Overflow here (+ the script) in case it gets deleted.

Other users have also noticed Twitter's "Export/Import Blocked Accounts" functionality disappear. They ask how to do it now:

 

https://webapps.stackexchange.com/a/165739

 

The way I did it wasn't pretty, but they severely limited our options by removing the export/import block list functionality. I requested my Data Archive (https://help.twitter.com/en/managing-your-account/how-to-download-your-twitter-archive). You'll notice that it only contains link out to the web version of the block list. Since that list is dynamically loaded, it is not easy to web scrape without Selenium or a proxy, and it will get ugly.

So, instead, I dug around in the Data Archive and eventually found block.js, which is a JSON object of all the blocked accounts in their ideas. From there you can write a quick Python script to use the Twitter API to resolve all of those IDs to usernames. 

Here's some sloppy Python that technically works: 

# Burninator 2022
# Export block list functionality that Twitter randomly removed
# Used to block all the promoted content, will be posting a list of about 5000 blocked accounts shortly



from requests_oauthlib import OAuth1Session
import os
import json
import time


def getblocklist(fancyfile):
    consumer_key = "HAHA"
    consumer_secret = "NICETRYHACKERS"

    # Get request token
    request_token_url = "https://api.twitter.com/oauth/request_token?oauth_callback=oob&x_auth_access_type=write"
    oauth = OAuth1Session(consumer_key, client_secret=consumer_secret)

    try:
        fetch_response = oauth.fetch_request_token(request_token_url)
    except ValueError:
        print(
            "There may have been an issue with the consumer_key or consumer_secret you entered."
        )

    resource_owner_key = fetch_response.get("oauth_token")
    resource_owner_secret = fetch_response.get("oauth_token_secret")
    print("Got OAuth token: %s" % resource_owner_key)

    # Get authorization
    base_authorization_url = "https://api.twitter.com/oauth/authorize"
    authorization_url = oauth.authorization_url(base_authorization_url)
    print("Please go here and authorize: %s" % authorization_url)
    verifier = input("Paste the PIN here: ")

    # Get the access token
    access_token_url = "https://api.twitter.com/oauth/access_token"
    oauth = OAuth1Session(
        consumer_key,
        client_secret=consumer_secret,
        resource_owner_key=resource_owner_key,
        resource_owner_secret=resource_owner_secret,
        verifier=verifier,
    )
    oauth_tokens = oauth.fetch_access_token(access_token_url)

    access_token = oauth_tokens["oauth_token"]
    access_token_secret = oauth_tokens["oauth_token_secret"]

    # Make the request
    oauth = OAuth1Session(
        consumer_key,
        client_secret=consumer_secret,
        resource_owner_key=access_token,
        resource_owner_secret=access_token_secret,
    )

    logMe = open(fancyfile, 'r') #Block.js from user's data archive download
    leoutput = open("block_names.txt", 'a') #save account names here
    thecounter = 1

    for line in logMe:
        thatsTheTweet = "Current blocked account ID: " + line
        line = line.strip()
        thecounter = thecounter + 1

        # Making the request
        response = oauth.get(
            "https://api.twitter.com/2/users/"+line+"?user.fields=name"
        )

        #print (str(response.content))
        if (response.status_code != 201) and (response.status_code != 200):
            print("Request returned an error: {} {}".format(response.status_code, response.text))

        if ("Not Found Error" in response.text):
            print("Response code User Not Found: {}".format(response.status_code))
            continue

        if ("Forbidden" in response.text):
            print("Response code Forbidden: {}".format(response.status_code))
            continue

        if response.status_code == 429:
            print("Response code 429 usually is rate limit: {}".format(response.status_code))
            logMe.close()
            leoutput.close()
            exit(1337)


        print("Response code: {}".format(response.status_code))

        # Saving the response as JSON
        json_response = response.json()
        print(json.dumps(json_response, indent=4, sort_keys=True))

        snarf= json.dumps(json_response, indent=4, sort_keys=True)
        thejson = json.loads(snarf)
        writeme = thejson['data']['username'] #     "data" "username"
        leoutput.write(writeme+"\n")

        if (thecounter < 200):
            time.sleep(20)  # make it 20, 38 wroks
        else:
            time.sleep(63)

    logMe.close()
    leoutput.close()
    print("done")
    exit(1338)


if __name__ == '__main__':
    getblocklist('firstshot_twitter_block_reduced.txt')

 

 https://webapps.stackexchange.com/a/165739