home

Running code snippets embedded in markdown with Jekyll

While playing around with Jupyter Notebooks i had the idea of running code snippets embedded in markdown and inserting their output into the generated document.

To achieve this i made a small Jekyll plugin that registers a pre_render hook to execute a python script for every post.

Jekyll::Hooks.register :posts, :pre_render do | post, payload |
    if post.path.end_with?("md", "markdown")
        puts "Processing " + post.path
        post.content = `python3 "./_plugins/parse_md.py" "#{post.path}"`
    end
end

The python script then reads the files and replaces any code blocks that are preceded by <!-- exec --> with the output of the snippet.

I’ve decided to use this syntax because the markdown is still valid and will just render code blocks containing the snippets instead of their output in normal markdown processors (provided they support HTML comments that is).

# copy the locals before anything else to avoid interference between the code
# thats to be executed and this script
_locals = locals().copy()

import sys
import re
from io import StringIO
from contextlib import redirect_stdout


file_path = sys.argv[1]

def exec_match(match):
    f = StringIO()
    with redirect_stdout(f):
        exec(match.group(1), {}, _locals)
    return f.getvalue()

with open(file_path, "r", encoding="utf-8") as raw:
    code_regex = "<!--\s+exec\s+-->\n```python\n(.+?)```"
    
    # the front matter has already been parsed by jekyll when this script is executed
    # we have to delete it to prevent it from being interpreted as actual content
    processed = re.sub("^---.+?---\n?", "", raw.read(), flags=re.DOTALL)
    processed = re.sub(code_regex, exec_match, processed, flags=re.DOTALL)

    # write directly to stdout instead of using print() to avoid encoding issues
    sys.stdout.buffer.write(processed.encode('utf8'))

These little snippets allow me to inject “dynamic” content that gets updated everytime the site gets built.

As a little demonstration, below the following snipped will get run:

import datetime
print("The site was last built at **" + datetime.datetime.now().isoformat() + "**")

The site was last built at 2023-12-21T06:42:32.578353

To read more about this website, visit the about page.