Mass Text Messaging With the GroupMe API and Twilio⌗

Automated SMS is a powerful tool for organizations if utilized properly. There are a number of services that offer robust platforms for automated SMS but are inaccesable to organizations on a tight budget. Over the past three semesters I’ve built out an automated SMS system for a student organization at my collage. It consitsts of a few lines of python and the Twilio/GroupMe APIs. While the system itself is simple, running a program that you know will make a potentially unwelcomed disturbance in the name of your oginization to hundreds of people is nerve racking.

20220730163411.png

You can’t unsend or edit SMS messages so there is little room for error, also phones numbers and phones are expensive so there’s not a lot of good ways to test at scale. When I was writing the program I could not find any examples to go off of so my first attempt was a real shot in the dark. If you somehow found yourself responsible for writing a program to handle bulk sms for your organization on a $25 budget I hope this post helps you prevent repeating some of the mistakes I made.

Use case⌗

The organization I wrote this for competes for new members at the start of every semester with other similar student organizations. The process for new member recruitment is managed by a governing organization that has new members sign up for recruitment through a google form. organizations are not allowed to start recruiting new members until a set date and the contact information of potential new members is not released until that date. So when the contact information is finally released it’s a rush to invite as many people on the list to recruitment events. This is usually handled by the recruitment chair using his personal phone number and good old copy and paste. 20220730163453.png Doing this with over 500 numbers takes hours, clogs up the recruitment chairs phone, and provides no personalization or transparency. Having the ability to send out bulk customized messages at the press of a button and respond to followup questions would be a massive advantage over the other organizations using the copy paste method.

How it works⌗

The program reads a csv file with names and phone numbers then uses the Twilio python library to send a personalized message to every number in the file. The “personalized message” is just a copy paste message with their name replaced. If the recipient responds to that message Twilio will send post request with the content to a web hook I have set up using flask on a remote server. When flask gets this post request, it matches the recipients senders name to their number and uses the groupMe API to forward that message to a group chat.

20220730163518.png

To set this up you need to buy a VoIP number on Twilio for like one dollar 20220730163529.png

Then link that number to a messaging service by adding it to the “sender pool”. 20220730163608.png Twilio has a python quickstart example gives you a good idea of how easy it is to send a message.

import os
from twilio.rest import Client

# these are in twilio.com/console 
account_sid = "[TWILIO_ACCOUNT_SID]"
auth_token = "[TWILIO_AUTH_TOKEN]" 

client = Client(account_sid, auth_token)

# Creating this varable "sends" the message with the varable body 
# as the message
message = client.messages \
                .create(
                     body="PING",
                     from_='+123457891',
                     to='+1987654321'
                 )

Result ping.png After testing this out on my own phone number and some friends I got way too cocky and went ahead with trying to send invites out to about 500 different people like this:

import os
from twilio.rest import Client

def main():
    send_all()

def send_all():
    with open("new_member_list.csv", "r") as f:
        lines = [line.split(",") for line in f.readlines()]
        for line in lines:
            name = line[0]
            number = line[1].strip()
            send_msg(name, number)

def send_msg(name, number):
        # stored credental as environment varables
    account_sid = os.environ['TWILIO_ACCOUNT_SID']
    auth_token = os.environ['TWILIO_AUTH_TOKEN']
    ms_sid = os.environ['MS_SID']

    msg = f"Hey {name} come join us at 8:00PM today at our adress...etc"
    client = Client(account_sid, auth_token)
    message = client.messages \
        .create(
             body=msg,
             messaging_service_sid=ms_sid,
             to=f"{number}"
         )
    print(message.sid)

if __name__ == '__main__':
    main()

Data validation⌗

I felt really proud watching it send over a hundred messages in under a minute until it errored out with something like this:

Unable to create record: The 'To' number +1666666666 is not a valid phone number.

I went down a rabithole trying to use regex to validate phone numbers but gave up. This can be easily fixed with a try / except statement however my bigger problem was that I had no idea exactly how many messages I sent and who I sent them to. Always log outgoing messages, this prevents you from losing your place in your send list if your power goes out or you get an unexpected error which can lead to accidental double texts in the name of your organization. No one likes a double texter.

It took some time but I finally found out you could export your message logs from Twilios dashboard. I used some bash to remove the numbers if they showed up more than once in my original csv. While doing this I realized that I already double texted a couple people that submitted their application more than once. So yeah Remove duplicates before hitting send.

I sent the rest of the messages on my list after adding the precautions of logging, datacleaning and error handling. What could possibly go wrong? pong.png

It wasn’t until I was prepairng the bot for next semester that I realized if a user responds to your message without a callback url defined in your settings it sends a goofy message like the one above It turns out alot of people responded to these messages with a “thanks” or “I’ll be there” and received this message. You can stop it from sending a responce by selecting the “drop the message” in your messaging Integration options.

20220730163913.png

This method is basically shouting into the void and you miss alot of opertunties to get feadback or provide more information to the recipient. Twilio allows you to configure a web hook to accept incoming message data. A webhook is basically a url endpoint on a server that accepts incoming post requests when a specified event happens on the envoking program. In this case the Twilio api is our envoking program and the event that triggers the post request is when a user responds to our message.

I spun up a linux instance on vultr and installed a minimalist flask application to receave data.

import os, uuid, json
import subprocess, argparse, csv, base64
import datetime as dt
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
	if request.method == 'POST':
		fname = "API/responces/" + str(uuid.uuid1())+".json"
	    msg = request.values.get('Body', '').lower()
	    phone_num = request.values.get('From','')
	    city = request.values.get('FromCity','')
	    send_message(msg, phone_num)
        return '', 200 
    else:
        abort(400)

The above snippet is all you need to receave data from Twilio, what you do with that data is up to you. I chose to send that data to a group message using the groupMe API. I made a video a while ago going over the groupMe API and I even made a fork of an old python library python3 groupme tools to work with the api.

import requests
from requests.structures import CaseInsensitiveDict
def send_message(msg,num):
    name = get_name_from_num(num)
    url = "https://api.groupme.com/v3/bots/post"
    headers = CaseInsensitiveDict()
    headers["Content-Type"] = "application/json"

    data_dict = {
    "bot_id"  : "[bot_id]",
    "text"    : f"Number: {num} Name: {name} Message: {msg}"
    }

    data = json.dumps(data_dict)
    resp = requests.post(url, headers=headers, data=data)

    print(resp.status_code)

Twilio offers caller ID but we don’t need to use that if we already have a list of names and numbers.

def get_name_from_num(num):
    numeric_filter = filter(str.isdigit, num)
    parsed_num = "".join(numeric_filter)
    parsed_num = parsed_num[1:]

    with open("API/full_name_number.csv", newline='\n') as f:
        name_reader = csv.reader(f, delimiter=',')
        for row in name_reader:
            try:
                if row[1] == parsed_num:
                    return row[0]
            except:
                return "unable to find name"
if __name__ == "__main__":
    app.run(host='0.0.0.0')

examplemsg.png