How to never miss out on your favourite band’s sale using Python

A fun example of how to utilise python and your programming knowledge to automate your life

6 min readApr 12, 2021

Checkout the video I created for this tutorial: https://youtu.be/s9LKWm6uxIM

Why bother?

Being a software developer these days is pretty useful, everyone knows it. Besides giving you great career opportunities and the ability to work in almost any field possible, there are so many things in your life you could automate, or at least utilise in your favour.

I’m still at the beginning of my software engineering journey, and I still have a lot to learn. So, I’ve decided to start working on some small, fun projects, that aren’t necessarily related to my 9–5 job as a backend developer, to keep learning and have fun!

This is the first one and if you liked it lookout for more to come!

So, I used to be a dancer. Yes. I moved from being a full-time dancer to being a backend developer. More on that probably in another post. The thing is, I never stopped loving dance, yoga, or working out, and I still do this regularly. My favourite yoga/workout clothing brand is Lululemon. Whoever tried their stuff would know what I’m talking about! The problem is, it’s pretty pricy! So I’ve decided to utilise my Python skills and run a web scraper that will notify me whenever my favourite leggings are on sale. Sounds cool? Let’s go!

This automation idea can be executed to track anything you want! This use case is just a fun example.

You can also find the code on Github here: https://github.com/gilatb/lululemon-price-tracker

The plan

Project setup
Get the link of the product we want to monitor
Scrape the webpage and find the product info in it
Set condition for the alert
Implement email notification and send test
Create a dev email account and set privacy or use Google’s API to send email notifications
Create a scheduler using crontab to run the script every desirable amount of time

1. Project setup

This project will contain only two files: main.py and scheduler.py. Let’s create the skeleton of our project before starting to code:

In main.py:

class Crawler:

    def scan_price(self) -> list:
        pass    def alert_prices(self, prices) -> None:
        passclass Emailer:
    def _create_email_msg(self):
        pass    def send_email(self):
        passif __name__ == '__main__':
    crawler = Crawler()
    prices = crawler.scan_price()
    crawler.alert_price(prices)

And let’s keep scheduler.py empty for now, we will fill it out in the next post.

Another thing we need to do is run our project in a new virtual env. I like using pyenv and virtualenv, but it doesn't really matter. Here are the steps:

In the terminal (on mac) run this to install pyenv and pyenv-virtualenv:

$ brew install pyenv pyenv-virtualenv

Then install the right python version on pyenv:

$ pyenv install python 3.9.0

Now in the directory of the project create the venv, call it lulu (or anything else) and attach it to this project’s directory:

$ pyenv virtualenv 3.9.0 lulu
$ pyenv local lulu

If local doesn’t work, try: pyenv activate lulu

2. Get the link

The link I’m going to use is as follows, attached to a new variable called URL:

URL = 'https://www.lululemon.de/de-de/p/lululemon-align%E2%84%A2-leggings-mit-superhohem-bund-71%C2%A0cm/prod9200552.html?dwvar_prod9200552_color=47824'

3. Scaping the Link

We will use HTMLSession from requests_html to make the web-page HTML data into accessible python code.

First, we’ll go to the link in the browser and click right-click and inspect.

There, in the console's Elements tab, we can inspect the structure of the code and find the elements we are interested in. In this particular case, I’m interested in the markdown prices that appear in .markdown-prices class. I start by choosing the page content that is under the .product-detail-content class.

The code:

def scan_price(self) -> list:
    session = HTMLSession()
    page = session.get(URL)
    content = page.html.find('.product-detail-content', first=True)
    return content.find('.markdown-prices')

4. Set the condition for the alert

Let’s say you’d want to be alerted whenever the price is below 70 euros. Then we will set this as the condition to our alert, in a new function called: alert_prices:

def alert_prices(self, prices) -> None:
    for price in prices:
        if int(price.text[0:2]) < 70:
            print('send email notification')
            return

Here we loop over the prices (could be more than one in the discount section) and if we found one that stands in our condition (below 70) we will alert once. For the time being, we don’t actually send the alert, but only log as if we would send it.

5. Implement email notification and send test

After figuring out how to get the information from the webpage and how to set the condition upon sending the notification, now we will go about implementing the notification itself.

To send the emails we will use Google’s SMTP server (smtp.gmail.com). For that, we would need to create another dev account and reduce its security level. The other option is to use Google’s email API (or other available options) but for simplicity's sake, I will go with the first option.

Go ahead and create a new google account here. Then, go to your new Google account settings and click on security. reduce security level in the Less secure app access panel, click Turn on access. It’s definitely not a best practice, but good enough for us to move forward easily in the stage of this small script. Then, we will put the address and credentials in the from field of our email sender.

Let’s create a class called Emailer with two methods: _create_email_msg and send_email. We would also need to have the following properties in our class to be able to send the email:

class Emailer:
    subject: str
    from_address: str = 'my_dev_account@gmail.com'
    from_pass: str = "my_dev_account_password"
    to_address: str = 'my_usual_account@gmail.com'

    SMTP_SERVER: str = "smtp.gmail.com"
    PORT = 465

    def __init__(self, subject):
        self.subject = subject    def _create_email_msg(self):
        pass
 
    def send_email(self):
        pass

We will use two libraries to create and send the emails:

email — to create the email (docs)
smtplib — to send the email (docs)

Now let’s fill in the details about the msg in the _create_email_msg method using EmailMessage class from the email library:

def _create_email_msg(self):
    msg = EmailMessage()
    
    msg['Subject'] = self.subject
    msg['From'] = self.from_address
    msg['To'] = self.to_address
    
    content = f'Don\'t miss it here: {URL}'
    msg.set_content(content)
    
    return msg

Then, we could use the function we just created to construct an email message. Then, let’s create a context manager with our SMTP server, log in to our newly created dev account, and send the message.

def send_email(self):    msg = self._create_email_msg()

    # take password from user input
    password = input("Type your password and press enter: ")

    # Create a secure SSL context
    context = ssl.create_default_context()

    with smtplib.SMTP_SSL(self.SMTP_SERVER, self.PORT, context=context) as server:
        server.login("gb.dev1000@gmail.com", password)
        server.send_message(msg)
        server.quit()

We could also use the debugging server to test the email by logging it in the terminal like this:

# Send the message via our own SMTP debugging server.
server = smtplib.SMTP('localhost:1025')
server.send_message(msg)
server.quit()

In the terminal tun this:

python -m smtpd -n -c DebuggingServer localhost:1025

Now, let’s use the Emailer class we have created and send the email using in the alert_price function.

def alert_price(self, prices) -> None:
    for price in prices:
        if int(price.text[0:2]) < 70:
            print('send email notification')
            Emailer(subject=f'Your favourite lulu leggings cost only {price.text[0:4]} euros now!').send_email()
            print('email sent successfully!')
            return

That’s basically it! Now you could set up a cron job to run this as a scheduled job periodically. On this and more in the next post! Feel free to add fancy stuff like adding the image to the email or make a more complex condition to alert by.

Thanks for reading and until next time!