kwaamfan's notes

back to personal site home

automatically updating an atom or rss feed from a markdown file with bash

6 min read

this is a small tutorial on how i write my site update posts in markdown, then add it to my atom.xml using pandoc and a simple bash script. should work with rss feeds too, although you’ll need to tweak the pandoc template yourself. you’ll need the following:

  • the ability to run bash scripts (sorry windows users… you might be able to run it by using git bash or converting it to a .bat file. if you know powershell you could apply the same logic and just rewrite the syntax)
  • pandoc installed on your system
  • an already set up atom or rss feed
  • your markdown files to be named in YYYY-MM-DD.md format (not necessary, but you’ll have to tweak the bash script yourself if you have a different naming system)

the markdown file

here’s the minimum amount of info needed for the pandoc template:

---
feedTitle:
url:
updated:
---
 
put your post here

feedTitle is the title of each entry which goes inside of <title></title>. url is the page on your site you want your update to be linked to which goes inside of <link>; when a subscriber clicks on the post’s title in their feed reader, it’ll take them to this url. updated is the date and time your post is published in ISO 8601 format without fractional seconds and goes inside of <published></published>.

i write my updates in obsidian, where updated is a property i use on all markdown files. it gets automatically updated by the linter plugin whenever i edit a file. if you don’t use obsidian, some other options for getting the date time in the right format are:

  • write your posts in vscode then use the insert date string extension
  • pass $(date -Iseconds) as an argument to pandoc (i’ll show you how to do this at the end)

the pandoc file

in ~/.pandoc/templates, i have the following stored in the file atom-main.html:

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
 
  <title>ราวเหมือนดั่งความฝัน</title>
  <subtitle>it's like a dream...</subtitle>
  <link rel="alternate" type="text/html" href="https://kwaamfan.neocities.org/" />
  <link rel="self" type="application/atom+xml" href="https://kwaamfan.neocities.org/atom.xml" />
  <updated>$updated$</updated>
  <link href="https://kwaamfan.neocities.org/" />
  <author>
    <name>kwaamfan</name>
  </author>
  <id>https://kwaamfan.neocities.org/atom.xml</id>
 
  <entry>
    <title>$feedTitle$</title>
    <link href="$url$" />
    <id>https://kwaamfan.neocities.org/?$updated$</id>
    <updated>$updated$</updated>
    <content type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        $body$
      </div>
    </content>
  </entry>

of course you will need to tweak stuff like the title and the links to your site and its feed to match your existing file. also note how many lines the header takes up (the part before the first <entry>, not including the empty line). for me this is 13.

the parts wrapped in dollar signs are variables that are read either from the input file’s front matter or passed to pandoc when it is run. $body$ is the content of the markdown file, not including the front matter, which gets converted to html by pandoc.

the bash file

here’s what my bash script looks like:

#!/bin/bash
 
date="${1:-$(date -I)}" # set the date to the first argument. if not provided, use today's date in yyyy-mm-dd format
input="$HOME/Documents/Obsidian/040 projects/neocities/atom/$date.md"
atom="$HOME/Documents/neocities/public/atom.xml"
atomTemplate="$HOME/.pandoc/templates/atom-main.html"
 
# if the input file does not exist, see if there is a file for yesterday
if [ ! -f "$input" ]; then
    input="$HOME/Documents/Obsidian/040 projects/neocities/microblog/$(date --date="yesterday" -I).md"
 
    # if there is no file for yesterday, exit
    if [ ! -f "$input" ]; then
        echo "no input file found"
        exit 1
    fi
 
fi
 
# remove the first 13 lines of the atom.xml file
tail -n +13 "$atom" > "$atom".tmp
mv "$atom".tmp "$atom"
 
# update the atom feed
pandoc --template="$atomTemplate" --wrap=none "$input" | cat - "$atom" | sponge "$atom"

you’ll need to change input, atom, and atomTemplate to match the locations on your computer where you keep your markdown files with updates, atom.xml, and atom feed template file, respectively.

i name my update files in YYYY-MM-DD format to make it easy to default to using today’s date as the input. i also like to, if the file for today doesn’t exist, check if one exists for yesterday’s date, in case i start writing an update but don’t publish it until after midnight (🙈) or the next day.

the next part that uses tail is to remove the header of the atom feed. every time that we update an atom feed, we need to update the <updated> part of the header, not just add in the new post. so we can have our pandoc template include both the new post and updated header, then just stick that on top of our old feed with its header removed. if your feed’s header is not 13 lines like mine is, change 13 to the number of lines your header is.

the final line is what converts the markdown file with the update to atom format, then combines it with the old feed whose header we removed with the last command.

if you need to add the updated argument to pandoc, change the final line to:

pandoc --template="$atomTemplate" -V "updated:$(date -Iseconds)" --wrap=none "$input" | cat - "$atom" | sponge "$atom"

the -V argument lets you add additional key:value pairs. $(date -Iseconds) is a bash command that returns the date and time in ISO 8601 format to the seconds.

without any output specified, pandoc outputs to stdout. then we take our new header and post and print it out along with our old posts and </feed> tag back to stdout. finally sponge reads all of that and writes it to the original atom.xml file.

extending the script: removing an old version of today’s post

while i don’t use this method for this script, the script i use when building my microblog has some logic to check if a post has already been made today and asks the user if they’d like to remove the old post before adding a new one. i don’t really see the need to make this script more complicated—unlike my microblog, which i sometimes build then decide to tweak and build again before publishing, my main site’s updates posts aren’t very complicated so i’m usually pretty confident that they’re good to go—but if you’d like to implement such a feature, you can see how i did so in my microblog script here on this site’s github.