Skip to content

#131: Working With Bookmarks in Tweepy

Since I concluded my experiments with Tweepy in April, a new and important Twitter endpoint finally arrived: Bookmarks. Let's have a look how we can access our Twitter bookmarks with Tweepy.

Update Tweepy

Before we can do anything, we need to make sure we have the newest version of Tweepy. You can install Tweepy or update to the newest version with this command:

1
pip install -U tweepy

Make sure that you have at least version 4.8:

1
pip show tweepy

Name: tweepy Version: 4.10.0 Summary: Twitter library for Python ...

Get the right kind of user token

According to the Twitter API v2 authentication mapping, we need an “OAuth 2.0 Authorization Code with PKCE” to access the bookmark endpoints:

The mapping table shows bookmark endpoints are only supported by PKCE

We can use the OAuth 2.0 Authorization Code Flow with PKCE (User Context) to get the right token with Tweepy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import tweepy
import os
from dotenv import load_dotenv

load_dotenv()

# read keys from .env
client_id = os.getenv('client-id')
client_secret = os.getenv('client-secret')

# prepare OAuth2Handler
oauth2_user_handler = tweepy.OAuth2UserHandler(
    client_id=client_id,
    redirect_uri="https://127.0.0.1:6006/callback",
    # minimal scope to work with bookmarks
    scope=["bookmark.read", "bookmark.write",
        "tweet.read","users.read"],
    client_secret=client_secret
)

print(oauth2_user_handler.get_authorization_url())

verifier = input("Enter whole callback URL: ")

access_token = oauth2_user_handler.fetch_token(
    verifier
)

# store these tokens in .env file:
print(f"\naccess-token-pkce={access_token['access_token']}")

This is similar to the token we used before, but with a different protocol.

You must ask for the scopes "bookmark.read", "bookmark.write","tweet.read" and "users.read". Otherwise, your calls to the bookmark endpoint will end with this error:

tweepy.errors.Forbidden: 403 Forbidden

We can put the token in our .env file and access it where we want to work with bookmarks:

1
2
3
4
5
6
7
8
9
import tweepy
import os
from dotenv import load_dotenv
load_dotenv()

# read keys from .env
access_token_pkce = os.getenv('access-token-pkce')

client = tweepy.Client(access_token_pkce)

Bookmark a tweet

We can bookmark a tweet with the bookmark() method:

1
2
3
4
5
python_release = "1536895050176667649"

# add tweet to bookmarks
response = client.bookmark(tweet_id=python_release)
print(f"Tweet {python_release} bookmarked: {response.data['bookmarked']}")

In the response we get the confirmation that this tweet got added to our bookmarks:

Tweet 1536895050176667649 bookmarked: True

Read your bookmarks

We can get up to 800 bookmarks of the authenticated user with the method get_bookmarks():

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
response = client.get_bookmarks(
    expansions="author_id,attachments.media_keys",
    tweet_fields="created_at,public_metrics,attachments",
    user_fields="username,name,profile_image_url",
    media_fields="public_metrics,url,height,width,alt_text")
# print(response)

tweets = response.data

users = {}
for user in response.includes['users']:
    users[user.id] = f"{user.name} (@{user.username}) [{user.profile_image_url}]"

# process media attachment
media = {}
if 'media' in response.includes:
    for item in response.includes['media']:
        media[item.media_key] = f"{item.url} - {item.height}x{item.width} - Alt: {item.alt_text}"

tweets = response.data

# The expanded tweet offers a lot more data
for tweet in tweets:
    print('-' * 50)
    print(f"{tweet.id} ({tweet.created_at}) - {users[tweet.author_id]}:\n {tweet.text} \n")
    metric = tweet.public_metrics
    print(f"retweets: {metric['retweet_count']} | likes: {metric['like_count']}")
    if tweet.attachments is not None:
        for media_key in tweet.attachments['media_keys']:
            print(f"Media attachment: #{media[media_key]}")

As with search in the V2 client, we need to declare what data we want. In the example above we ask for the tweet itself, the user who tweeted it, the media files that may be attached to the tweet and the metrics. That gives us an output like this:

-------------------------------------------------- 1534959411185209348 (2022-06-09 18:03:36+00:00) - Python Software Foundation (@ThePSF) [https://pbs.twimg.com/profile_images/1517584461017370624/DF3DpXUW_normal.jpg]: Check out our latest blog post, where you’ll see our 2021 Annual Report! https://t.co/rjkzvLwLXu

2022 is well underway, but there’s still time to reflect on the challenges and successes of 2021, and to give kudos to the many people behind our work and progress last year. https://t.co/O1EwPMkv4h

retweets: 46 | likes: 183 Media attachment: #https://pbs.twimg.com/media/FU1Dz2BWUAc2DoN.png - 400x524 - Alt: The cover image for the PSF's 2021 annual report, which features a drawing of a python slithering diagonally across to almost fill the page, in neon versions of the Python and PSF colors - blue and yellow. The words "The 20th ANNIVERSARY of the PYTHON SOFTWARE FOUNDATION" are written on the snake's body. The background is very dark blue, and the feel of the cover is noisy and neon (almost a 20s take on 80s aesthetic?). "2021 Annual Report" is in the top left corner. The Python Softward Foundation logo in monochrome yellow to match the ssnake is in the top right corner. White lightning bolts surround the snake as well as geometric squiggles and lines in complementary ombre'd colors. -------------------------------------------------- 1536895050176667649 (2022-06-15 02:15:08+00:00) - Python Hub (@PythonHub) [https://pbs.twimg.com/profile_images/1152895149594284032/wb1uTnTk_normal.jpg]: Python 3.10.5 is available

The latest bugfix drop for Python 3.10 is here: Python 3.10.5. This release packs more than 230 bugfixes and docs changes.

https://t.co/0HlfPhSVNz

retweets: 11 | likes: 40 --------------------------------------------------

Remove a bookmarked tweet

We can delete a bookmarked tweet with the remove_bookmark() method:

1
2
3
# remove tweet from bookmarks
response = client.remove_bookmark(tweet_id=python_release)
print(f"Tweet {python_release} bookmarked: {response.data['bookmarked']}")

If we run this code, the tweet is no longer bookmarked:

Tweet 1536895050176667649 bookmarked: False

Conclusion

The bookmark endpoint was the final missing part to access Twitter through the API. This gap is now closed, and the new methods are straightforward. However, there is the challenge of getting the right token, that took me a while to figure out. I hope this post gives you a smoother experience.