Using GNU Recutils to Generate a Playlist Feed

Tue, 26 Apr 2022

In my previous post about building a shinobi site, “Going Dark (Building a Shinobi Site)”, I had explained that one of my reasons for coming up with my own process instead of using Bradley Taunt’s was that I wanted to be able to create my own video playlists without having a YouTube account. In this post I’ll be documenting my method for generating such a playlist using GNU recutils, shell scripting, and GNU make.

As with “Going Dark”, I’m writing this mainly for my own reference. You can use this post to create your own playlist in an Atom feed, too, if you have some familiarity with Unix-like systems or are willing and able to search for the appropriate documentation to figure it out on your own. My email address is at the end of this post in case you have questions.

A Recfile for Playlists

I started out using the same recfile format that I used for posts, but eventually decided on something more ambitious.

# -*- mode: rec -*-

%rec: Playlist
%mandatory: Artist Title Album Year Video Created Updated
%key: Id
%auto: Id
%sort: Id
%type: Created date
%type: Updated date
%type: Sort int
%type: Video line
%type: Description line
%doc: 
+ A plain-text database that can be used to generate HTML index,
+ RSS feeds, and XML sitemaps. Used with GNU recutils.

Using this format, we create entries like so:

Artist: Baroness
Title: Take My Bones Away
Album: Yellow & Green
Year: 2012
Video: 4V0N1x675FQ
Created: 2022-04-26T16:41:55-05:00
Updated: 2022-04-26T16:41:55-05:00

Unlike my other recfiles, I don’t include full URLs in my playlist recfile. Instead, I only pull the video ID strings YouTube uses. This is important because it gives me more flexibility when creating my entry template for the playlist feed.

<entry>
  <title>{{Artist}}: "{{Title}}"</title>
  <link href="https://yewtu.be/watch?v={{Video}}" rel="alternate"/>
  <link href="https://yewtu.be/vi/{{Video}}/maxres.jpg" rel="related"/>
  <id>https://yewtu.be/watch?v={{Video}}</id>
  <updated>{{Updated}}</updated>
  <published>{{Created}}</published>
  <summary type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml">
      <p>
        <img src="https://yewtu.be/vi/{{Video}}/maxres.jpg"
             alt="preview image for &ldquo;{{Title}}&rdquo; by {{Artist}}"
             width="640"></img>
      </p>
      <ul>
        <li>Track: &ldquo;{{Title}}&rdquo;</li>
        <li>Artist: {{Artist}}</li>
        <li>Album: <i>{{Album}}</i></li>
        <li>Year: {{Year}}</li>
      </ul>
    </div>
  </summary>
</entry>

By using YouTube’s video IDs instead of using YouTube’s URLs as is I gain the ability to do the following:

  1. I can link to a reliable Invidious instance like yewtu.be instead of directly to YouTube.
  2. I can embed videos in the Atom feed’s content as escaped XHTML, which supports the <iframe> element.

However, most feed readers will strip out <script> and <iframe> for security reasons. This is perfectly reasonable, so I’ve updated my template to pull a static preview image instead.

Also, unlike in HTML5 where you don’t need a closing tag for <img> or even a ‘/’ before the ‘>’ at the end of the <img> declaration, XHTML seems to demand a closing </img> tag. Or, at least, tidy-html5 seems to when checking over Atom feeds.

Building the Atom Feed

The script for building my playlist’s Atom feed is quite similar to the others. The only real difference is that I’m not using the usual “atom.templ” template with recfmt, but the “playlist.templ” shown earler.

#!/usr/bin/env sh

FILE="playlist"
DATE=$(date +"%Y-%m-%dT%H:%M:%S-05:00")
YEAR=$(date +"%Y")
AUTHOR="Matthew Graybosch"
EMAIL="contact@"
DOMAIN="matthewgraybosch.com"
TITLE="Matthew's Playlist"
SUBTITLE="music that ${AUTHOR} enjoys and recommends"
ICON="icon.png"
RIGHTS=${YEAR} ${AUTHOR}, all rights reserved"

echo "<?xml version=\"1.0\" encoding=\"utf-8\"?>
<feed xmlns=\"http://www.w3.org/2005/Atom\">
    <title>${TITLE}</title>
    <subtitle>${SUBTITLE}</subtitle>
    <icon>https://${DOMAIN}/${ICON}</icon>
    <link href=\"https://${DOMAIN}/feeds/${FILE}.xml\" rel=\"self\"/>
    <updated>${DATE}</updated>
    <author>
    <name>${AUTHOR}</name>
    <email>${EMAIL}${DOMAIN}</email>
    </author>
    <rights>${RIGHTS}</rights>
    <generator>POSIX shell and GNU recutils</generator>
    <id>https://${DOMAIN}/feeds/${FILE}.xml</id>";

recsel ${FILE}.rec | recfmt -f playlist.templ

echo "</feed>";

Generating Feeds with GNU Make

Invoking this is pretty easy. Here’s the build target from my makefile.

.PHONY: build
build: posts.xml fiction.xml bookmarks.xml playlist.xml

# I've snipped the other build targets for clarity. --M.G.

playlist.xml: playlist.rec playlist.templ
    recfix --auto playlist.rec
    recfix --sort playlist.rec
    ./playlist.sh | sed -e 's/\ \&\ /\ \&amp;\ /g' | tidy -q -i -w -utf8 -xml > feeds/playlist.xml

Another difference from the other feeds is that I make a point of using sed4 to escape any ampersands I’ve put in my recfile so that they don’t foul up the resulting XML. Yes, I could just use “&” instead of “&” in the recfile itself, but I’m only human and might forget. The computer won’t forget; it’ll pipe the output of playlist.sh into sed and pipe sed’s output into tidy every single time, and do it the same way every time, as long as I don’t alter the makefile.

Why use tidy? It not only pretty-prints my XML in case I’m bored enough to ever want to read it myself, but does local validation to ensure that my script is generating well-formed XML and that anything that should be escaped gets escaped. If it throws warnings or outright errors, then I screwed up and have some debugging to do.

Where’s the Playlist?

It’s available at https://matthewgraybosch.com/feeds/playlist.xml. You might want to wear headphones and turn the volume down; my playlist is mostly heavy metal and it can get loud.

I no longer provide this feed.

Why Invidious?

Invidious uses YouTube’s API to access video hosted on Google’s service, but appears to be more privacy-friendly.