Originally on www.santiagomartins.com
Twitter is a double edged sword: on one hand it gives you access to bleeding edge discussion on the most varied topics, on the other it's a hate-fuelled attention slot machine where the signal to noise ratio often makes the whole experience a waste of time at best, and positively harmful at worst.
One of the big causes of this is Twitter's own algorithm. Its workings aren't public, but whatever they are, they determine what you see and consume. One way to get around both these problems is by organising your Twitter into topic lists. Lists show you tweets in chronological order so aren't as vulnerable to the negative aspects of algorithmic curation. The problem is, we often have far too many follows for it to be practical to do this manually, but this is where we can leverage Twitter's API.
The first thing we want to do is to extract all of our followed users so we can begin organising them into the lists we want to create. The best place to do this is a Google Sheet, because we can easily assign lists to each user, and then use the Google Sheets API in combination with Twitter's to get the list of users we want to organise and put them in their designated list.
To organise our followed users into lists we need to do the following:
- Create a Twitter app
- Create a Google app
- Organise users into lists
- Run the scripts to assign users to those lists.
- Create a Twitter app
This can be done at https://developer.twitter.com/en/portal/dashboard. Make sure to keep the access credentials and set to read & write.
- Create a Google app
Go to the Google developer console and create a new app. Activate the Google Sheets API and download the json file with your credentials.
- Organise users into lists
This is where the fun starts. We want to create a spreadsheet with the following structure so that we can organise our followed users into lists:
| ID | Name | List | Description |
|---|---|---|---|
| 5367537 | john bob | Economics professor at an Ivy League | |
| 942847 | jazz | Being a footballer in the 80s took its toll | |
| 1235342790 | granny | Building 12 philosophy startups. ex-@memeco |
So the first thing we need to do is export this data from our twitter account into a csv file.
@client =
Twitter::REST::Client.new do |config|
config.consumer_key = 'your consumer key'
config.consumer_secret = 'your consumer secret'
config.access_token = 'your access token'
config.bearer_token = 'your bearer token'
config.access_token_secret = 'your access token secret'
end
def export_users(user)
File.open('List.csv', 'w+') do |file|
@client
.friend_ids(user)
.each do |f|
file <<
"#{f}, #{@client.user(f).name}, #{@client.user(f).description.gsub("\n", ' ')}\n"
rescue Twitter::Error::TooManyRequests => e
# NOTE: Your process could go to sleep for up to 15 minutes but if you
# retry any sooner, it will almost certainly fail with the same exception.
puts 'retrying..'
sleep e.rate_limit.reset_in + 1
retry
end
end
end
export_users('my_username')
Then we want to upload that CSV to a Google Sheet and fill in the list name we want each user to belong to.
- 4. Once we've done that we can retreive that table and work with it.
# frozen_string_literal: true
require "google/apis/sheets_v4"
require "googleauth"
require "googleauth/stores/file_token_store"
require "fileutils"
OOB_URI = "urn:ietf:wg:oauth:2.0:oob"
APPLICATION_NAME = "Google Sheets API Ruby Quickstart"
CREDENTIALS_PATH = "lib/credentials.json"
# The file token.yaml stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
TOKEN_PATH = "token.yaml"
SCOPE = Google::Apis::SheetsV4::AUTH_SPREADSHEETS_READONLY
##
# Ensure valid credentials, either by restoring from the saved credentials
# files or intitiating an OAuth2 authorization. If authorization is required,
# the user's default browser will be launched to approve the request.
#
# @return [Google::Auth::UserRefreshCredentials] OAuth2 credentials
def authorize
client_id = Google::Auth::ClientId.from_file CREDENTIALS_PATH
token_store = Google::Auth::Stores::FileTokenStore.new file: TOKEN_PATH
authorizer = Google::Auth::UserAuthorizer.new client_id, SCOPE, token_store
user_id = "default"
credentials = authorizer.get_credentials user_id
if credentials.nil?
url = authorizer.get_authorization_url base_url: OOB_URI
puts "Open the following URL in the browser and enter the " \
"resulting code after authorization:\n" + url
code = gets
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id, code: code, base_url: OOB_URI
)
end
credentials
end
# Initialize the API
service = Google::Apis::SheetsV4::SheetsService.new
service.client_options.application_name = APPLICATION_NAME
service.authorization = authorize
# use your spreadsheet
spreadsheet_id = "id_goes_here"
range = "Sheet2!A:C"
response = service.get_spreadsheet_values spreadsheet_id, range
response_array = []
response.values.drop(1).each do |row|
# Print columns A and C, which correspond to indices 0 and 2.
unless row[2].nil?
response_array << (row[0]).to_i
response_array << (row[2]).to_s
end
end
response_array.drop(0)
$hash = Hash[*response_array]
Finally, we can use the Twitter API to put each user in their designated list.
require "twitter"
require_relative "sheets"
@client =
Twitter::REST::Client.new do |config|
config.consumer_key = 'your consumer key'
config.consumer_secret = 'your consumer secret'
config.access_token = 'your access token'
config.bearer_token = 'your bearer token'
config.access_token_secret = 'your access token secret'
end
all_lists = @client.lists
$hash.each do |user_id, list_name|
user_id_name = @client.user(user_id).name
if all_lists.any? { |list| list.name == list_name } # if list exists
lists = all_lists.select { |list| list.name == list_name } # get lists with the hash list name
lists.each do |list| # for each list
if @client.list_members("duarteosrm", list.slug).any? { |member| member.name == user_id_name }
puts "#{user_id_name} already in #{list.name} list"
else # unless user exists in list
@client.add_list_member("duarteosrm", list.slug, user_id) # add user
puts "added #{user_id_name} to existing list, #{list.name}"
end
end
else # else if list does not exist
@client.create_list(list_name) # create list
puts "list created: #{list_name}"
lists = @client.lists.select { |list| list.name == list_name } # get lists with the hash list name
lists.each do |list| # for each list
@client.add_list_member("duarteosrm", list.slug, user_id) # add user
puts "added #{user_id_name} to new list, #{list.name}"
end
end
rescue Twitter::Error::TooManyRequests => e
# NOTE: Your process could go to sleep for up to 15 minutes but if you
# retry any sooner, it will almost certainly fail with the same exception.
puts "retrying.."
sleep e.rate_limit.reset_in + 1
retry
rescue Twitter::Error::NotFound
puts "user not found"
next
rescue Twitter::Error::Forbidden
puts "cannot add to lists"
next
end
Bonus: I wanted to unfollow all of the users I had put in lists that didn't follow me back, so I created the following script to do so:
require "twitter"
require_relative "sheets"
@client =
Twitter::REST::Client.new do |config|
config.consumer_key = 'your consumer key'
config.consumer_secret = 'your consumer secret'
config.access_token = 'your access token'
config.bearer_token = 'your bearer token'
config.access_token_secret = 'your access token secret'
end
begin
id = @client.user.id
followers = @client.follower_ids(id)
rescue Twitter::Error::TooManyRequests => e
# NOTE: Your process could go to sleep for up to 15 minutes but if you
# retry any sooner, it will almost certainly fail with the same exception.
puts "retrying.."
sleep e.rate_limit.reset_in + 1
retry
end
$hash.each do |user_id, list_name|
@client.unfollow(user_id) unless followers.any? { |follower| follower == user_id }
puts "unfollowed #{@client.user(user_id).name} unless they follow back"
rescue Twitter::Error::TooManyRequests => e
# NOTE: Your process could go to sleep for up to 15 minutes but if you
# retry any sooner, it will almost certainly fail with the same exception.
puts "retrying.."
sleep e.rate_limit.reset_in + 1
retry
rescue Twitter::Error::NotFound
puts "user not found"
next
rescue Twitter::Error::Forbidden
puts "cannot add to lists"
next
end
Top comments (0)