I just got a new Switch (because what else is there to do these days?!) And I really wanna get the new Paper Mario game. Except, I'm not keen on paying full pop for the title... So. I built a price monitoring Slack Bot to tell me if it's on sale! Here's how you can build one too~
First, install gazpacho, hickory, and slackclient:
pip install gazpacho hickory slackclient
Identify the page you want to scrape:
Grab the URL, open a Text Editor (I like Atom), create a python file called mario.py and import gazpacho:
from gazpacho import get, Soup
Download the html with the get function and stuff it in a Soup wrapper:
url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)
Figure out how the price is being rendered on the page by right clicking on it and opening "Inspect Element:
Note the specific tag and element attributes:
In this case the relevant block of html on the page is:
<div class="price_FHDfG large_3aP7Z " aria-hidden="true">
$79
<span class="rightEndPrice_6y_hS">99</span>
</div>
Use the find method with partial matching (strict=False) enabled to extract the price and format it as a float:
price = soup.find("div", {"class": "price"}, strict=False)
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))
Might as well grab the title too:
title = soup.find('h1', {'class': 'productName'}).text
Print out both to confirm that everything is working:
print(title, price)
# Paper Mario: The Origami King (Switch) 79.99
Your entire script should look like this:
# mario.py
from gazpacho import get, Soup
url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)
title = soup.find('h1', {'class': 'productName'}).text
price = soup.find("div", {"class": "price"})
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))
print(title, price)
To schedule this scraper and turn it into a Slack Bot go to https://api.slack.com/apps and click Create New App:
Find and click on OAuth & Permissions:
Search for the chat:public.write scope and add it:
Scroll back up to the top of the page and install the app into your Slack Workspace:
Copy the generated token to the mario.py file as a string:
token = "xoxb-00000000-0000000000-00000000000"
Import slack and instantiate an instance of WebClient:
from slack import WebClient
client = WebClient(token=token)
Send a message to a Slack channel with the chat_postMessage method:
client.chat_postMessage(channel="mario", text=f"{price} - {title}")
Re-factor everything to make it look like:
# mario.py
from gazpacho import get, Soup
from slack import WebClient
token = "xoxb-00000000-00000000-000000000000000"
client = WebClient(token=token)
url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)
title = soup.find('h1', {'class': 'productName'}).text
price = soup.find("div", {"class": "price"})
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))
client.chat_postMessage(channel="mario", text=f"${price} - {title}")
You really shouldn't expose your token as plaintext but, whatever...
Run the script at the command line to make sure it works:
python mario.py
Scheduling this script to every 30 seconds with hickory is dead simple. Just use hickory schedule at the command line:
hickory schedule mario.py --every=30s
To check the status of the schedule run:
hickory status
# ID FILE STATE RUNS INTERVAL
# ea6b74 mario.py waiting 0 30
And to kill the schedule:
hickory kill mario.py
Scheduling this Price Monitoring Slack Bot to execute every day at 9:00am is probably more realistic:
hickory schedule mario.py --every=@9:00am
And that's it! 🤑
If you've built something cool with gazpacho and/or hickory get in touch, I'd love to hear about it!
Top comments (0)