Basic website monitoring using cronjobs and ntfy.sh

There are many reasons to monitor a website, but for very simple use cases like checking for a keyword, the existing tools are overkill.

In my homelab i use a bash script and a cronjob to monitor some sites, and ntfy.sh for notifications.

Here are some examples:

Monitoring availability of articles in online-shops

I’m reading Dune and am waiting for the updated german translation of the fifth entry. So this command checks if “pre-order” is found in the html and sends me a notification if not - meaning the book is now available.

curl -s https://www.buchhaus.ch/de/buecher/taschenbuch/fantasy/detail/ISBN-9783453320918 | grep -q "vorbestellen" || curl -d "Dune available!" ntfy.sh/foo

This simple pattern is already very useful, despite no dynamic information being passed to the notification.

If instead of watching for the absence of a string you want to send a notification if a string is present, replace || with &&.

Monitoring HackerNews

Another more complex example would be to search for posts from a website (say lwn.net) on the HackerNews frontpage:

This example is very artificial, HN has an RSS feed and if you really want to monitor for posts, you should use that feed and a RSS reader.

curl -s https://news.ycombinator.com | grep "sitestr\">lwn.net" | sed -E 's/.*[^\(]<a href=\"([^"]*)\"[^<]*>([^<]*)<\/a>.*/\2\t\1/g' | while IFS=$'\t' read -r title url; do curl -d "$title" -H "Actions: view, Open Post, $url" ntfy.sh/foo; done

This makes use of some more complex parsing of the HTML respone and dynamically sends the filtered posts to ntfy.sh. It also uses actions to add a link to the notification.

Steam Item Price monitoring using the Steam API

For even more complex scripts i use python, for example to monitor the price of CSGO Cases. Since i’m scheduling these commands on my Unraid server, which does not have python installed by default, i use the python docker image.

docker run --rm --name csgo-case-notify -v /mnt/user/appdata/python/scripts:/usr/src -w /usr/src python python csgocasenotify.py

The python script itself isn’t really the focus of this post, but for completeness sake:

csgocasenotify.py

import urllib.request
import json
import time
from datetime import date

class Case:
    def __init__(self, name: str, item_id: int, count: int):
        self.fees = 0.45
        self.name = name
        self.item_id = item_id
        self.count = count
        self.price = 0

    def refresh_price(self) -> int:
        url = "https://steamcommunity.com/market/itemordershistogram?country=CH&language=english&currency=4&item_nameid=" + str(self.item_id) + "&two_factor=0"
        response = urllib.request.urlopen(url)
        data = json.load(response)
        highest_buy_order = int(data['highest_buy_order']) * 0.01
        self.price = highest_buy_order

    @property
    def total(self) -> float:
        return self.price * self.count * (1-self.fees)

    def __str__(self):
        return "{}x {} @ CHF {:.2f} => CHF {:.2f}".format(self.count, self.name, self.price, self.total)

if __name__ == "__main__":
    cases = [
        Case("Prisma Case", 176042493, 100),
        Case("Chroma 3 Case", 149865785, 100),
        Case("Horizon Case", 175999886, 100),
        Case("Spectrum 2 Case", 175917239, 100),
        Case("Clutch Case", 175966708, 100),
        Case("Gamma 2 Case", 165027636, 100),
        Case("Danger Zone Case", 176024744, 100),
        Case("Prisma 2 Case", 176118270, 100),
    ]

    for i, case in enumerate(cases):
        if i > 0:
            time.sleep(30)  # don't get rate limited
        case.refresh_price()

    total_sum = sum(case.total for case in cases)

    with open("price_history.txt", "a+") as file:
        file.seek(0)
        lines = file.readlines()
        if lines:
            old_total = float(lines[-1].strip().split(";")[1])
            change_percent = (total_sum - old_total) / old_total * 100
            title = "Case Update - Total Netto: CHF {:.2f} ({:+.2f}%)".format(total_sum, change_percent)
        else:
            title = "Case Update - Total Netto: CHF {:.2f}".format(total_sum)
        file.write(date.today().isoformat() + ";" + str(total_sum)+ "\n")

    text = "\n".join(str(case) for case in cases)
    print(title)
    print(text)

    req = urllib.request.Request("https://ntfy.sh/foo", data=text.encode(encoding='utf-8'))
    req.add_header("Title", title)
    urllib.request.urlopen(req)