Skip to main content

How to create Twitter keyword monitoring with Python and Slack notifications

ยท 12 min read
Norah Sakal

Cover image

This reply sparked an idea to build a keyword monitoring tool for Twitter with Slack notifications whenever someone tweets anything with the keywords I'm monitoring โฌ‡๏ธ

Monitoring tool idea

The reply is to this awesome Indie Hackers post $0 - $20K in 2 months, here's tactics that worked

In this walkthrough, we'll build a social media listener for indie hackers looking to shamelessly promote what they're making by monitoring these keywords:

What are you building?

But you can monitor any keywords.

Dependenciesโ€‹

Here's what we'll use:

1. Python ๐Ÿโ€‹

2. Twitter API ๐Ÿฆโ€‹

3. Slack webhooks ๐Ÿ“ฃโ€‹


Stepsโ€‹

Here're the steps:

1. Get Twitter developer API credentialsโ€‹

2. Create a Twitter search queryโ€‹

3. Get Slack webhook URLโ€‹

4. Write monitoring scriptโ€‹


1. Get Twitter developer API credentialsโ€‹

You first need your Twitter developer token to query for keywords and call the search/recent API endpoint in Twitter.


Disclaimerโ€‹

OAuth 1.0a

This guide covers the steps to use the OAuth 1.0a method. You could also go with OAuth 2.0 Authorization Code with PKCE, which will give you a slightly higher rate limit (450 requests instead of 180 requests per 15-minute window).

I've written another guide to get an OAuth 2.0 Authorization Code with PKCE in this post


Go to the dashboard of your Twitter developer account, click + Add App, choose Create new, pick an environment and then choose a name.

Make sure to save the bearer token revealed at the last stage, and you'll need the token when we're making calls to the API endpoints later on.

Twitter developer portal

The Twitter API is reachable at https://api.twitter.com/2/<ENDPOINT>, and the endpoint that gives you the tweets from the last seven days that match a search query:

API_URL_FOLLOWING = "https://api.twitter.com/2/tweets/search/recent"

Now that we have the bearer token let's create the search query for the keywords we'd like to monitor.


2. Create a Twitter search queryโ€‹

For this guide, we'll create a query that searches for tweets where we can shamelessly plug our indie hacker stuff.

Let's decide on the phrases. I'll go with these but feel free to add the ones you find suitable:

- what are you building - what are you working on - anyone building cool things

Why these? Well, those are some I've seen and answered, so it's a great starting point.

For the query itself, let's exclude retweets and replies:

"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply

This query says we want either (OR) of the 3 sentences.

So far, our search query looks like this:

search_resp = requests.get(
search_url,
headers=search_headers,
params={
"query":'"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply'
}
)

Let's also fetch some data about the tweet by adding some tweet.fields to the query:

search_resp = requests.get(
search_url,
headers=search_headers,
params={
"query":'"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply',
'tweet.fields':'author_id,public_metrics,created_at,entities'
})

We'll ask for the author id, public metrics, when the tweet was created, and entities.

To be able to ask for these, we'll also need to expand on the author id:

search_resp = requests.get(
search_url,
headers=search_headers,
params={
"query":'"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply',
'tweet.fields':'author_id,public_metrics,created_at,entities',
'expansions':'author_id',
})

Finally, let's fetch some data about the user tweeting the post:

search_resp = requests.get(
search_url,
headers=search_headers,
params={
"query":'"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply',
'tweet.fields':'author_id,public_metrics,created_at,entities',
'expansions':'author_id',
'user.fields':'username,description,public_metrics,profile_image_url'
})

This addition will give us the username, the profile description, public metrics for the user, and the profile image URL.

This data will give us additional info about the user behind the tweet.


3. Get Slack webhook URLโ€‹

Start by creating a workspace on Slack. Skip this step if you already have a workspace you'd like to use:

Create Slack workspace

The first step is to create a new Slack app. We'll walk through the steps in this guide, but here is a more comprehensive walkthrough by Slack: Getting started with Incoming Webhooks

Visit https://api.slack.com/apps and click Create new app to start creating a new Slack app:

Create Slack app

Choose From scratch:

Create Slack app from scratch

Then pick a name and the workspace you'd like to use the app in, finally, click Create app:

Pick name and workspace

Now you have a Slack app. Click on Incoming webhooks to enable webhooks:

Incoming webhooks

Activate incoming webhooks:

Enable webhooks

Authorize the app and pick the channel to post to:

Slack channel to post to

Once you have activated incoming webhooks, scroll down to Webhook URL and click Add New Webhook to Workspace:

Add webhook URL

Now that you have an URL, copy it and save it for the comping steps:

Copy webhook URL

We have a Slack webhook URL ready. In the next step, let's write the script that will look for the mentioned keyword.


4. Write monitoring scriptโ€‹

Start by importing the libraries needed. We'll be doing endpoint calls, so start with requests and JSON:

import json
import requests

The next step is to add the Twitter query we previously created and add your bearer token to the header as well:

bearer_token = "YOUR_BEARER_TOKEN"
search_headers = {
'Authorization': f'Bearer {bearer_token}'
}

search_resp = requests.get(
search_url,
headers=search_headers,
params={
"query":'"what are you building" -is:retweet -is:reply OR "what are you working on" -is:retweet -is:reply OR "anyone building cool things" -is:retweet -is:reply',
'tweet.fields':'author_id,public_metrics,created_at,entities',
'expansions':'author_id',
'user.fields':'username,description,public_metrics,profile_image_url'
})

If you run this, you'll get a dict with a list of tweets:

Tweet example from query

The next step is to create a dict with all the tweet data needed for each Slack message. Here's an example of data I'm using, make a loop and add each tweet to a new dict:

tweet_summary = []
for i,tweet in enumerate(search_resp.json()['data']):
temp_dict = {}
temp_dict['tweet'] = tweet['text']
temp_dict['created_at'] = tweet['created_at']
temp_dict['retweet_count'] = tweet['public_metrics']['retweet_count']
temp_dict['reply_count'] = tweet['public_metrics']['reply_count']
temp_dict['like_count'] = tweet['public_metrics']['like_count']
temp_dict['quote_count'] = tweet['public_metrics']['quote_count']
temp_dict['tweet_url'] = f"https://twitter.com/{search_resp.json()['includes']['users'][i]['username']}/status/{tweet['id']}"

# User info
temp_dict['user_description'] = search_resp.json()['includes']['users'][i]['description']
temp_dict['name'] = search_resp.json()['includes']['users'][i]['name']
temp_dict['username'] = search_resp.json()['includes']['users'][i]['username']
temp_dict['followers_count'] = search_resp.json()['includes']['users'][i]['public_metrics']['followers_count']
temp_dict['following_count'] = search_resp.json()['includes']['users'][i]['public_metrics']['following_count']
temp_dict['tweet_count'] = search_resp.json()['includes']['users'][i]['public_metrics']['tweet_count']
temp_dict['profile_img'] = search_resp.json()['includes']['users'][i]['profile_image_url']

# Check if hashtags
if 'entities' in tweet:
if 'hashtags' in tweet['entities']:
tweet_hashtags = []
for hashtag in tweet['entities']['hashtags']:
tweet_hashtags.append(hashtag['tag'])
temp_dict['tweet_hashtags'] = tweet_hashtags

tweet_summary.append(temp_dict)

tweet_summary

The last part checks for hashtags in a tweet and adds them to the dict as well.

Here's how the previous tweet looks in our new dict:

Tweet example from new dict

Let's create the dict needed to post a message to our Slack channel. Start with an empty dict:

slack_message_data = {}

The first key we'll add is icon_emoji, which is the icon you'll see in the Slack message:

Slack icon

Get the icon name by picking an emoji in Slack and then hovering to get the name:

Slack emoji name

Now add your chosen emoji to the Slack dict:

slack_message_data = {
"icon_emoji": ":mailbox_with_mail:"
}

:mailbox_with_mail: will give you this emoji: ๐Ÿ“ฌ

The following key we'll add to the Slack dict is text, simply the message sent in Slack.

Here's what I'll use:

"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}"

Add the message text to the Slack dict:

slack_message_data = {
"icon_emoji": ":mailbox_with_mail:",
"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}"
}

Now pick a username and add it to the Slack dict. That will be the username for each Slack message:

Slack username

Then add the username to the Slack dict:

slack_message_data = {
"icon_emoji": ":mailbox_with_mail:",
"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}",
"username": "New promotion tweet ๐Ÿ“ฃ",
}

The following Slack key is the channel; you'll need to pick the channel you want the message sent to.

Make sure to select the channel you picked in the earlier step when you enabled the incoming webhook:

Slack channel to post to

Then add your channel to the Slack dict. Make sure to start with a #:

slack_message_data = {
"icon_emoji": ":mailbox_with_mail:",
"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}",
"username": "New promotion tweet ๐Ÿ“ฃ",
"channel":'#YOUR_CHANNEL',
}

The last key is optional, I'm using attachments for the Twitter profile picture like this:

"attachments": [
{
"fallback": "Required plain-text summary of the attachment.",
"text": "Optional text that appears within the attachment",
"image_url": f"{tweet['profile_img']}",
"thumb_url": f"{tweet['profile_img']}",
}
]

It'll make it easier to recognize the Twitter profiles you're following.

Finally, add the attachments to the Slack dict:

slack_message_data = {
"icon_emoji": ":mailbox_with_mail:",
"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}",
"username": "New promotion tweet ๐Ÿ“ฃ",
"channel":'#YOUR_CHANNEL',

# Optional, makes it easier to recognize profiles
"attachments": [
{
"fallback": "Required plain-text summary of the attachment.",
"text": "Optional text that appears within the attachment",
"image_url": f"{tweet['profile_img']}",
"thumb_url": f"{tweet['profile_img']}",
}
]
}

We're ready to create a loop where each tweet is posted as a Slack message:

slack_key = "YOUR_SLACK_WEBHOOK_URL"
for tweet in tweet_summary:
temp_tweet = tweet['tweet'].replace('\n', ' ') # Optional to remove \n
slack_message_data = {
"icon_emoji": ':mega:',
"text": f"Promote yourself here \n *Tweet:* {temp_tweet} \n *Author:* {tweet['name']} \n *Description:* {tweet['user_description']} \n *URL:* {tweet['tweet_url']}",
"username": "New promotion tweet ๐Ÿ“ฃ",
"channel":'#integration-testing',
"attachments": [
{
"fallback": "Required plain-text summary of the attachment.",
"text": "Optional text that appears within the attachment",
"image_url": f"{tweet['profile_img']}",
"thumb_url": f"{tweet['profile_img']}",
}
]
}

response = requests.post(
slack_key,
data=json.dumps(slack_message_data),
headers={"Content-Type": "application/json"}
)

This loop will post each tweet as a new Slack message.

Let's add one last thing; some tweets will include hashtags, add an if-statement for those cases just before sending a new Slack message:

if 'tweet_hashtags' in tweet:
slack_message_data['text'] += f"\n *Hashtags:* {', '.join(tweet['tweet_hashtags'])}"

You're all set. Each tweet will be posted as a Slack message. Here's how the example tweet will look like when sent as a Slack message in the loop:

Slack message

Now you have a script that fetches tweets with the keywords you want to monitor and then sends each tweet as a Slack message to your Slack channel.


Summaryโ€‹

Here's a summary of what we did

1. Got our Twitter developer API bearer tokenโ€‹

2. Created a tweet search query for the keywords to monitorโ€‹

3. Created a new Slack workspace and a new Slack appโ€‹

4. Enabled incoming webhooks and authorized our app to send messages with the webhookโ€‹

5. Fetched Tweets with our search queryโ€‹

6. Sent the tweets with the keywords to our Slack channel with the incoming webhook URLโ€‹


5. Improvementsโ€‹

Search query There are many ways to improve the search query to get more relevant tweets. Check this query guide: Twitter search query guide. Try to exclude some keywords or add new ones.

Slack message formatting The message in this guide is pretty simple just one way to build the Slack message. Here's a guide with more advanced formatting: Making it fancy with advanced formatting

Cron jobs You'll probably want to run the tweet search query regularly, like once every hour. Add the key since_id to the search query and then save it after each API call to only fetch tweets that are published after the last time you ran your query.

since_id Returns results with a Tweet ID greater than (that is, more recent than) the specified ID.

Prewritten response You might want to have a prewritten response to each tweet.

Perhaps a like like this: https://twitter.com/intent/tweet?text=I%20want%20to%20have%20this%20prewritten%20tweet This link prewrites a tweet when you click is.

Here's how the link is created:

import urllib.parse
query = 'I want to have this prewritten tweet'
parsed_query = urllib.parse.quote(query)
print(f"https://twitter.com/intent/tweet?text={parsed_query}")

Next stepsโ€‹

1. Repo with source code Here is the repo with the source code if you'd like to implement this on your own โฌ‡๏ธ https://github.com/norahsakal/create-keyword-monitoring-in-twitter-with-slack-notifications

2. Do you need help with implementing your own keyword monitoring SaaS? Or do you have other questions? I'm happy to help, don't hesitate to reach out โžก๏ธ norah@quoter.se

3. Want this but don't feel like coding? Join the waitlist to get Twitter keyword monitoring โฌ‡๏ธ https://keyword-monitoring.carrd.co

Or shoot me a DM on Twitter, I'd love to help out @norahsakal