From 9f1b37fa346a7dcf77c1b6963a6d2e4b871fe5ed Mon Sep 17 00:00:00 2001 From: Stonewall Jackson Date: Sun, 22 Jan 2023 09:56:17 -0500 Subject: initial commit --- src/blog/desktop-linux-with-nfs-homedirs/index.md | 226 ++++++++++++++++++++ src/blog/index.md | 9 + src/blog/makefile-based-blogging/index.md | 249 ++++++++++++++++++++++ 3 files changed, 484 insertions(+) create mode 100644 src/blog/desktop-linux-with-nfs-homedirs/index.md create mode 100644 src/blog/index.md create mode 100644 src/blog/makefile-based-blogging/index.md (limited to 'src/blog') diff --git a/src/blog/desktop-linux-with-nfs-homedirs/index.md b/src/blog/desktop-linux-with-nfs-homedirs/index.md new file mode 100644 index 0000000..e01581a --- /dev/null +++ b/src/blog/desktop-linux-with-nfs-homedirs/index.md @@ -0,0 +1,226 @@ +--- +title: Desktop Linux with NFS Home Directories +date: January 19, 2023 +subtitle: Something no one does anymore, apparently. +description: Issues you'll face with NFS-mounted homedirs, and some workarounds. +--- + +I manage multiple [Rocky Linux](https://rockylinux.org/) workstations that automount +users' home directories via kerberized NFS. Unfortunately, I don't think this is a common +setup anymore--I encountered a few bugs and performance issues that needed non-obvious +workarounds. + +## Problems + +### 1. Things break when you log in from two places at once + +If you can somehow restrict your users to a single GNOME session at any given time, +you'll probably be fine. However, as soon as someone leaves his desktop running and +logs into another workstation, strange things begin to happen. Here are some oddities +I've observed: + + - GNOME settings on one machine are clobbered by the other (this may or may not be desirable). + + - Firefox refuses to run, because the profile directory is already in use. + + - `gnome-keyring` freaks out and creates many login keyrings under `~/.local/share/keyrings`, + losing previously stored secrets in the process! + + - Sound quits working (I suspect this is due to `~/.config/pulse/cookie` being clobbered). + + - Flatpak apps completely blow up (each app stores its state in `~/.var`, and + [this is nonconfigurable](https://github.com/flatpak/flatpak/issues/1651)). Running + multiple instances of `signal-dekstop` instantly corrupts the sqlite database. + + - `goa-daemon` generates thousands of syslog messages per minute (I am unsure if this is + due to `~/.config/goa-1.0/accounts.conf` getting clobbered, or a symptom of + [this bug](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/issues/32)). + I have no idea what `goa-daemon` does, nor do I want to. I have been victimized by + [the bazaar](http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/) + enough for one lifetime. + +### 2. It's slow + +I/O-heavy tasks, like compiling and grepping, will be much slower over NFS than the local +disk. Browser profiles stored on NFS (`~/.mozilla`, `~/.cache/chromium`, etc.) provide +a noticeably poor experience. + +File browsing is also painful if you have lots of images or videos. Thumbnails for +files stored on NFS will be cached in `~/.cache/thumbnails`, which is **also** stored +on NFS! + +## Solution: Move stuff to local storage + +The [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) +lets you change the default locations of `~/.cache`, `~/.config`, and the like by setting +some environment variables in the user's session. We can solve most of these problems +by moving the various XDG directories to the local disk. + +### Automatically provision local home directories + +First, let's write a script that automatically provisions a _local_ home directory +whenever someone logs in: + +````bash +#!/bin/bash + +# /usr/local/sbin/create-local-homedir.sh + +# Log all output to syslog. +exec 1> >(logger -s -t $(basename "$0")) 2>&1 + +PAM_UID=$(id -u "${PAM_USER}") + +if (( PAM_UID >= 1000 )); then + install -o "${PAM_USER}" -g "${PAM_USER}" -m 0700 -d "/usr/local/home/${PAM_USER}" +fi +```` + +Of course, it needs to be executable: + +````bash +chmod 755 /usr/local/sbin/create-local-homedir.sh +```` + +Next, we modify the PAM configuration to execute our script whenever anyone logs in +via GDM or SSH: + +````diff +--- /etc/pam.d/gdm-password ++++ /etc/pam.d/gdm-password +@@ -1,5 +1,6 @@ + auth [success=done ignore=ignore default=bad] pam_selinux_permit.so + auth substack password-auth ++auth optional pam_exec.so /usr/local/sbin/create-local-homedir.sh + auth optional pam_gnome_keyring.so + auth include postlogin + +--- /etc/pam.d/sshd ++++ /etc/pam.d/sshd +@@ -15,3 +15,4 @@ + session optional pam_motd.so + session include password-auth + session include postlogin ++session optional pam_exec.so /usr/local/sbin/create-local-homedir.sh +```` + +
+A note on SELinux + +If you're using SELinux, you'll need a separate copy of the `create-local-homedir` script +for use with GDM, labeled with `xdm_unconfined_exec_t`: + +````bash +ln /usr/local/sbin/create-local-homedir{,-gdm}.sh +semanage fcontext -a -t xdm_unconfined_exec_t /usr/local/sbin/create-local-homedir-gdm.sh +restorecon -v /usr/local/sbin/create-local-homedir-gdm.sh +```` + +Be sure to modify `/etc/pam.d/gdm-password` appropriately. + +
+ +### Set XDG Environment Variables + +We need to tell the user's applications to use the new local home directory +for storage. We have to do this early in the PAM stack for GDM, because `$XDG_DATA_HOME` +must be set before `gnome-keyring` gets executed. + +Edit your PAM files again, adding one more line: + +````diff +--- /etc/pam.d/gdm-password ++++ /etc/pam.d/gdm-password +@@ -1,6 +1,7 @@ + auth [success=done ignore=ignore default=bad] pam_selinux_permit.so + auth substack password-auth + auth optional pam_exec.so /usr/local/sbin/create-local-homedir.sh ++auth optional pam_env.so conffile=/etc/security/pam_env_xdg.conf + auth optional pam_gnome_keyring.so + auth include postlogin + +--- /etc/pam.d/sshd ++++ /etc/pam.d/sshd +@@ -16,3 +16,4 @@ + session include password-auth + session include postlogin + session optional pam_exec.so /usr/local/sbin/create-local-homedir.sh ++session optional pam_env.so conffile=/etc/security/pam_env_xdg.conf +```` + +Then, create the corresponding `pam_env.conf(5)` file: + +````default +# /etc/security/pam_env_xdg.conf + +XDG_DATA_HOME DEFAULT=/usr/local/home/@{PAM_USER}/.local/share +XDG_STATE_HOME DEFAULT=/usr/local/home/@{PAM_USER}/.local/state +XDG_CACHE_HOME DEFAULT=/usr/local/home/@{PAM_USER}/.cache +XDG_CONFIG_HOME DEFAULT=/usr/local/home/@{PAM_USER}/.config +```` + +### Hacks for Non-XDG-Compliant Apps + +Unfortunately, since a majority of open source developers follow the +[CADT model](https://www.jwz.org/doc/cadt.html), there are many apps that ignore the +XDG specification. Sometimes these apps have their own environment variables +for specifying their storage locations. Otherwise, symlinks can provide us with an escape +hatch. + +Create a script in `/etc/profile.d` for these workarounds. Scripts in this directory +are executed within the context of the user's session, so we can freely write inside +his NFS home directory using his UID (and kerberos ticket, if applicable). + +````bash +# /etc/profile.d/local-homedirs.sh + +if (( UID >= 1000 )); then + # Building code is *much* faster on the local disk. Modify as needed: + export PYTHONUSERBASE="/usr/local/home/${USER}/.local" # python + export npm_config_cache="/usr/local/home/${USER}/.npm" # nodejs + export CARGO_HOME="/usr/local/home/${USER}/.cargo" # rust + export GOPATH="/usr/local/home/${USER}/go" # golang + + # Firefox doesn't provide an environment variable for setting the default profile + # path, so we'll just symlink it to /usr/local/home. + mkdir -p "/usr/local/home/${USER}/.mozilla" + ln -sfn "/usr/local/home/${USER}/.mozilla" "${HOME}/.mozilla" + + # Flatpak hardcodes ~/.var, so symlink it to /opt/flatpak. + ln -sfn "/opt/flatpak/${USER}" "${HOME}/.var" +fi +```` + +If you use any Flatpak apps, each user will need his own local Flatpak directory. +The Flatpak runtime appears to shadow the entire `/usr` using mount namespaces, +so any `/usr/local/home` symlinks will disappear into the abyss. Luckily, `/opt` +appears to be undefiled. Modify your original script like so: + +````diff +--- /usr/local/sbin/create-local-homedir.sh ++++ /usr/local/sbin/create-local-homedir.sh +@@ -6,4 +6,5 @@ + + if (( PAM_UID >= 1000 )); then + install -o "${PAM_USER}" -g "${PAM_USER}" -m 0700 -d "/usr/local/home/${PAM_USER}" ++ install -o "${PAM_USER}" -g "${PAM_USER}" -m 0700 -d "/opt/flatpak/${PAM_USER}" + fi +```` + +## Closing Thoughts + +Most of my users are nontechnical, so I'm pleased that these workarounds do not require +any manual intervention on their part. + +I am sad that `$XDG_CONFIG_HOME` can't be shared between multiple workstations reliably. +When I change my desktop background or add a new password to `gnome-keyring`, it only +affects the local machine. + +Initially, I tried symlinking various subdirectories of `~/.config` to the local disk +individually as I encountered different bugs (e.g. `~/.config/pulse`). Unfortunately this +proved brittle, as I was constantly playing whack-a-mole with apps that abused `$XDG_CONFIG_HOME` +for storing local state. In the end, it was less of a headache to just dump the whole thing +onto the local disk. + +I suppose if you verified an app behaved properly with multiple simultaneous NFS clients, +you could always symlink `/usr/local/home/$USER/.config/$APP` **back** onto NFS! diff --git a/src/blog/index.md b/src/blog/index.md new file mode 100644 index 0000000..1c35f0a --- /dev/null +++ b/src/blog/index.md @@ -0,0 +1,9 @@ +--- +title: sacredheartsc blog +header: Blog +description: Carolina-grown articles about self-hosting, privacy, unix, and more. +--- + +::: bloglist +__BLOG_LIST__ +::: diff --git a/src/blog/makefile-based-blogging/index.md b/src/blog/makefile-based-blogging/index.md new file mode 100644 index 0000000..9244a0d --- /dev/null +++ b/src/blog/makefile-based-blogging/index.md @@ -0,0 +1,249 @@ +--- +title: Makefile-Based Blogging +date: December 12, 2022 +subtitle: Yet another static site generator using `pandoc(1)` and `make(1)`. +description: Building a markdown-based static site generator using pandoc and make. +--- + +A few days ago, I got the gumption to start blogging again. The last time I wrote +with any frequency, I lovingly hand-crafted each HTML file before `rsync`ing it to +my web server. This time, I wanted a more efficient workflow. + +I surveyed the [vast number](https://github.com/myles/awesome-static-generators) +of static site generators available on GitHub, but most of them seemed like +overkill for my humble website. I figured that by the time I wrapped by head +around one of them, I could have just written a Makefile. + +Finally, I came across [pandoc-blog](https://github.com/lukasschwab/pandoc-blog), +which gave me inspiration and showed me the ideal pandoc incantations for +generating HTML from markdown files. And thus, my +[Makefile-based static site generator](https://git.sacredheartsc.com/www/about/) +was born. You're reading the inaugural post! + +## Generating the HTML + +The workhorse of this thing is [pandoc](https://pandoc.org), which is a ubiquitous +open-source document converter. Transforming markdown into HTML is as simple as: + +```bash +pandoc document.md -o document.html +``` + +Simple! But to generate an entire website, we'll need some of pandoc's additional +features: custom templates and document metadata. + +### Custom Templates + +The layout of pandoc's output document is determined by the +[template](https://pandoc.org/MANUAL.html#templates) in use. Pandoc includes +default templates for a variety of document formats, but you can also specify +your own. + +A very simple HTML template might look something like this: + +```html + + + + + + +

$title$

+$body$ + + +``` + +[My pandoc template](https://git.sacredheartsc.com/www/tree/templates/default.html) +is what generates the navigation bar at the top of this page. + +The variable `$body$` is replaced by the content of your markdown document when +pandoc renders the template. The other variables are replaced by their +corresponding values from the document's metadata. + +### Document Metadata + +Each pandoc source document can have associated metadata values. There are three +ways of specifying metadata: the `--medatata` [flag](https://pandoc.org/MANUAL.html#option--metadata), +a dedicated [metadata file](https://pandoc.org/MANUAL.html#option--metadata-file), or +a [YAML metadata block](https://pandoc.org/MANUAL.html#extension-yaml_metadata_block) +embedded within the document itself. We'll be using the embedded metadata blocks. + +Each markdown document for my website starts with a YAML metadata block. The +metadata for the post you're +[currently reading](https://git.sacredheartsc.com/www/tree/src/blog/makefile-based-blogging/index.md) +looks like this: + + +```yaml +--- +title: Makefile-Based Blogging +date: December 12, 2022 +subtitle: Yet another static site generator using `pandoc(1)` and `make(1)`. +description: Building a markdown-based static site generator using pandoc and make. +--- +``` + +You can put whatever YAML you like in your markdown files, as long as the metadata +starts and ends with three hyphens. + +## Automating pandoc with make + +Using a Makefile, we can automatically invoke pandoc to convert each markdown +file in our blog to HTML. In addition, `make` will keep track of which source +files have changed since the last run and rebuild them accordingly. + +First, lets describe the project layout: + +- **src/**: the source files of our blog, including markdown files and static + assets (CSS, images, etc). The subdirectory structure is entirely up to you. + +- **public/**: the output directory. After running `make`, the contents of this + directory can be `rsync`'d straight to your web server. + +- **scripts/**: helper scripts for generating the blog artifacts. Currently there + are only two: + + - [bloglist.py](https://git.sacredheartsc.com/www/tree/scripts/bloglist.py) + generates a markdown-formatted list of all your blog posts, sorted by the + `date` field in the YAML metadata block. + + - [rss.py](https://git.sacredheartsc.com/www/tree/scripts/rss.py) generates + an RSS feed for your blog. + +- **templates/**: pandoc templates which generate HTML from markdown files + (currently, there is only one). + +The Makefile used to build this website is located [here](https://git.sacredheartsc.com/www/tree/Makefile). +I've reproduced a simplified version below, to make it easier to step through. + +```makefile +###################### +# Variable definitions +###################### + +# These variables are used to generate the RSS feed +URL = https://www.sacredheartsc.com +FEED_TITLE = sacredheartsc blog +FEED_DESCRIPTION = Carolina-grown articles about self-hosting, privacy, unix, and more. + +# The number of blog posts to show on the homepage +BLOG_LIST_LIMIT = 5 + +# File extensions (other than .md) that should be included in public/ directory +STATIC_REGEX = .*\.(html|css|jpg|jpeg|png|xml|txt) + +# Pandoc template used to generate HTML +TEMPLATE = templates/default.html + +# List of subdirectories to create +SOURCE_DIRS := $(shell find src -mindepth 1 -type d) + +# List of source markdown files +SOURCE_MARKDOWN := $(shell find src -type f -name '*.md' -and ! -name .bloglist.md) + +# List of static assets +SOURCE_STATIC := $(shell find src \ + -type f \ + -regextype posix-extended \ + -iregex '$(STATIC_REGEX)') + +# List of all blog posts (excluding the main blog page) +BLOG_POSTS := $(shell find src/blog \ + -type f \ + -name '*.md' \ + -and ! -name .bloglist.md \ + -and ! -path src/blog/index.md) + +# Subdirectories to create under public/ +OUTPUT_DIRS := $(patsubst src/%, public/%, $(SOURCE_DIRS)) + +# .html files under public/, corresponding to each .md file under src/ +OUTPUT_MARKDOWN := $(patsubst src/%, public/%, $(patsubst %.md, %.html, $(SOURCE_MARKDOWN))) + +# Static file targets under public/ +OUTPUT_STATIC := $(patsubst src/%, public/%, $(SOURCE_STATIC)) + +# Script to generate RSS feed +RSSGEN = scripts/rss.py \ + src/blog \ + --title="$(FEED_TITLE)" \ + --description="$(FEED_DESCRIPTION)" \ + --url=$(URL) \ + --blog-path=/blog \ + --feed-path=/blog/rss/feed.xml + + +###################### +# File Targets +###################### + +# Default target: convert .md to .html, copy static assets, and generate RSS +public: \ + $(OUTPUT_DIRS) \ + $(OUTPUT_MARKDOWN) \ + $(OUTPUT_STATIC) \ + public/blog/feed.xml + +# Homepage (/) +public/index.html: src/index.md src/.bloglist.md $(TEMPLATE) + sed $$'/__BLOG_LIST__/{r src/.bloglist.md\nd}' $< \ + | pandoc --template=$(TEMPLATE) --output=$@ + +# Markdown list of 5 most recent blog posts +src/.bloglist.md: $(BLOG_POSTS) scripts/bloglist.py + scripts/bloglist.py src/blog $(BLOG_LIST_LIMIT) > $@ + +# The main blog listing (/blog/) +public/blog/index.html: src/blog/index.md src/blog/.bloglist.md $(TEMPLATE) + sed $$'/__BLOG_LIST__/{r src/blog/.bloglist.md\nd}' $< \ + | pandoc --template=$(TEMPLATE) --output=$@ + +# Markdown list of _all_ blog posts +src/blog/.bloglist.md: $(BLOG_POSTS) scripts/bloglist.py + scripts/bloglist.py src/blog > $@ + +# Convert all other .md files to .html +public/%.html: src/%.md $(TEMPLATE) + pandoc --template=$(TEMPLATE) --output=$@ $< + +# Catch-all: copy static assets in src/ to public/ +public/%: src/% + cp --preserve=timestamps $< $@ + +# RSS feed +public/blog/feed.xml: $(BLOG_POSTS) scripts/rss.py + $(RSSGEN) > $@ + + +###################### +# Phony Targets +###################### + +.PHONY: serve rsync clean + +# Run a local HTTP server in the output directory +serve: public + cd public && python3 -m http.server + +# Deploy the site to your webserver +rsync: public + rsync -rlphv --delete public/ webserver.example.com:/var/www/html + +clean: + rm -rf public + rm -f src/.bloglist.md + rm -f src/blog/.bloglist.md +``` + +## Closing Thoughts + +I admit, there is a small amount of hackery involved. You obviously can't generate +a time-sorted list of blog posts using pure markdown, so I'm generating the +markdown list using a Python script in an intermediate step. I then (ab)use `sed` +to shove that list into the markdown source on the fly. This means that changing +the look of the [blog list](/blog/) requires hacking up the Python code. + +But overall, I've been quite happy with this little project. There's just something +about writing paragraphs in `vi` and typing `:!make` that warms my soul with +memories of simpler times. -- cgit