Cleaning up broken snapshots (snapper + btrfs)

The combination of btrfs + snapper is a great solution for the Linux desktop. Perhaps even the best thing since sliced bread. Once properly set-up, you can rollback any file that you may accidentally damage at some point. I’ve found it invaluable during software upgrades/migrations (oops, are all your desktop panels gone after upgrading? don’t worry, just roll back) or when running into bugs (oops, the Digikam library got corrupted? don’t worry, just roll back).

Configuring snapper involves letting systemd activate it regularly using systemd timers. This works well, although you may end up having corrupt / incomplete snapshots if your computer crashes in the middle of a snapper operation.

Having broken snapshots will be made known to you in the journal with events such as:
:1: parser error : Document is empty

This message indicates that you have work to do to clean up broken snapshots. The following Bash oneliner may help you do this:

IFS=$'\n';
for x in $(grep -hr SUBVOLUME /etc/snapper/configs | cut -d '"' -f 2); do
  for y in "$x/.snapshots/"*; do
    z="$y/info.xml";
    if ! [ -s "$z" ]; then echo "***$y***"; ls -lah "$y";
      read -p "Delete? (y/n)" R; if ! [ "$R" = "y" ]; then continue; fi;
      set -x; btrfs subvol del "$y/snapshot"; rm -rf "$y"; set +x;
fi; done; done; unset IFS
  • I’ve broken the oneliner over multiple lines for this post, but just merge them together for use in a shell.
  • This is just a oneliner and not a real program. The way I do it here, is not the recommended way to loop in Bash though it should work fine for this use case (the alternative, using read + while, won’t work here due to a nested read). Refactoring would make it more complex, at which point I’d suggest to just make it a Python program.
  • It needs to run as root.
  • As always, have back-ups. Caveat emptor.

Apache conditional websockets

If you have Apache >= 2.4 and enabled proxy_wstunnel, then this example may help to show how to redirect websockets connections to the right place. (Note that there are easier ways if you want to redirect all traffic to the websocket location. This example is for if you want to have the normal HTTP side-by-side with the websockets.)

[...]
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteRule /(.*) ws://localhost:1234/$1 [P,L]
[...]

Search and replace in .docx using Python

A simple PoC to get you going:

#!/usr/bin/env python2
import zipfile as zf
import StringIO as si
import sys

candidates = ["word/document.xml"]

sio = si.StringIO()
with zf.ZipFile(sio, "w", compression=zf.ZIP_DEFLATED) as zio:
  with zf.ZipFile(sys.argv[1], "r") as za:
    for infile in za.namelist():
      indata = za.read(infile)
      if infile in candidates:
        indata = indata.replace("token", "replacement")
      zio.writestr(infile, indata)

with open("modified_{}".format(sys.argv[1]), "w") as oufile:
  oufile.write(sio.getvalue())

Spam level headers in Thunderbird

Spamassassin adds several special headers to every email that it scans. One of these is X-Spam-Level, that is, the degree of “spaminess” that the email scored.

thunderbird-headers

When training your spam filters, it can be useful to see the score of each scanned email. In Thunderbird, this can easily be done by changing a setting using the config editor. In the field mailnews.headers.extraExpandedHeaders add the value X-Spam-Level.

thunderbird-setextra

As a bonus, to see the user agent string, you can also use the config editor by setting mailnews.headers.showUserAgent to value true. This simple flag renders a separate plugin for this functionality superfluous.

Gmail-style email address aliases with Postfix

As some might know, Google offers alternative ways to refer to Gmail email addresses.

For example, you can use the “plus” technique to make distinctions between different organizations that you share you email address with. For instance, if you own example@gmail.com, then example+nospam still delivers to the same address. Indeed, everything after the plus symbol is ignored (Gmail is simply adhering to the standard here).

One advantage of this technique is that you can hand out different email aliases to different organizations. If you start receiving spam on any of these aliases then you can trace this back to the organization which leaked your email address to spammers.

The Postfix mail transfer agent also supports such a feature. If you set

recipient_delimiter = +

then Postfix tries to deliver to example+nospam, and if this doesn’t exist, will deliver to example.

Of course, you can change this parameter. This is especially interesting if you own the whole domain name and do not need to consider other users. Changing the parameter to a dot symbol is not a bad idea, since many web forms tend to inadvertently block out the plus symbol.