🍙

Thaura Work: checking in

Just a social update. I spent a lot of time playing in OpenCode and Thaura. OC has these free models (basically promoting the big silicon companies): Ling, Muse Spark, Nemotron, MiMo. It’s nice getting a feel for them. Thaura uses GLM-4.5.

I started working a little more on KRUpload, an old web-based file uploader program that’s become this monstrosity.

Right now, in my mind, it’s “Google Drive with rsync”. Last month, it was “Google Drive with some PostgreSQL tables, and canned methods, with icons.”

I 👍🏼 Rsync

rsync is great for transferring files that are:

  • short
  • text files that change often
  • large files that don’t change much
  • archives (great for mirrors)

It’s not good for:

  • enormous files that change often, like database files
  • directories full of files that change rapidly, or aren’t regular files
  • 3-way synchronization with multiple masters (guess what, nothing is good at this)
  • backups, unless you use it within backup scripts

I set up an rsync server to expose KRUpload shared directories as shared directories on an rsync server.

So you can write a script to sync a local directory with this shared directory on the server.

It was easy with docker compose:

  rsyncd:
    image: alpinelinux/rsyncd:latest
    container_name: rsyncd
    restart: unless-stopped
    ports:
      - "873:873"
    volumes:
      - ../data/kru-uploads:/var/uploads:rw
      - ./rsyncd/modules.d:/etc/rsyncd.modules.d:ro
      - ./rsyncd/secrets.d:/etc/rsyncd.secrets.d:rw
      - ./rsyncd/rsyncd.conf:/etc/rsyncd.conf:ro
      - ./rsyncd/log:/var/log
    environment:
      - RSYNC_MODULES=backups
    networks:
      krupload_net:
        ipv4_address: 172.28.0.11

The section for the web server starts like this:

  web:
    build: .
    ports:
      - "8080:80"
    volumes:
      - ./src:/var/www
      - ../data/kru-uploads:/var/uploads
      - ../data/kru-config:/var/lib/krupload/config
      - ../data/kru-logs:/var/log/krupload
      # BackupShare writes per-share rsyncd module configs + secrets here. These
      # are the SAME host dirs the rsyncd service reads (rsyncd/modules.d and
      # rsyncd/secrets.d), so a share created via the UI lands where rsyncd loads
      # it on next restart. Mounted rw into web at writable paths because the
      # PHP worker runs as www-data (uid 33) and can't write to /etc or /var.
      - ./rsyncd/modules.d:/var/lib/krupload/rsync-modules:rw
      - ./rsyncd/secrets.d:/var/lib/krupload/rsync-secrets:rw

What wasn’t easy: dynamically creating shares and passwords.

The app could make the config files, but the server didn’t pick up the changes. In a normal system, you can send a SIGHUP to the rsyncd process, and force a re-read. This, however, was made incredibly difficult because the app is inside one container, and rsyncd was in another container.

Eventually, I settled on a somewhat non-secure, and kludgy solution:

  1. The app creates config files for rsync modules (their term for shares) in rsyncd.modules.d/, and rsync usernames and passwords in rsyncd.secrets.d/
  2. When the files change, an external program restarts the docker container (with the command “docker compose restart rsyncd”).

The other possibilities were:

  • Get the container process to respond to signals. This sounded good – but it’s hard to do.
  • Run s6 or another supervisor inside the container, and send commands to the supervisor, and the supervisor sends the SIGHUP. (Cleaner, but it breaks the general pattern to run a single process inside a container.)
  • These didn’t solve the problem of sending a signal from within a container. I considered a message queue, or using Redis publish/subscribe, but that didn’t solve the problem of turning the rsyncd container into a subscriber. (Other interprocess communication, like named pipes or sockets, had the same problems.)

So, the least complex solution was a C program to watch the config folders, and then run a shell command to restart the container. Here’s its help:

Usage: watchreload/watchreload [options] <debounce_seconds> <path_to_watch> <shell_command...>

Options:
  --pidfile=/path/to/my.pid   Where to write this process's PID
                              (default: /tmp/watchreload.pid).

Examples:
  watchreload/watchreload 5 ./rsyncd docker compose restart rsyncd
  watchreload/watchreload --pidfile=./watchreload.pid 5 ./rsyncd docker compose restart rsyncd

I integrated it into the environment through a justfile. (I had to do this because the LLM clobbered my original Makefile when it generated the watchreload.c code.)

# Compile the rsyncd config watcher (C) from source.
watcher-build:
    cd watchreload && make

# Watch ./rsyncd for config changes in the FOREGROUND (blocks until Ctrl+C).
# Usage: just watcher [debounce_seconds]
watcher debounce="5":
    ./watchreload/watchreload {{debounce}} ./rsyncd "docker compose restart rsyncd"

# Stop the background rsyncd watcher started by `just up`.
watcher-stop:
    test -f /tmp/watchreload.pid || { echo "no pidfile; watcher not running?"; exit 0; }
    kill "$(cat /tmp/watchreload.pid)" && rm -f /tmp/watchreload.pid
    echo "rsyncd watcher stopped"

The code the LLM wrote was pretty huge. The original version was in PHP, and used polling and a “touch file” to record the last time a change was detected. I had it recode in C and use inotify, which is more efficient. I should have checked it for unused code, and also ditched anything related to looking at non-directory files.

/*
 * watchreload.c — RSYNCD Config Watcher (C, inotify-based)
 *
 * C port of watchreload.php. Watches a directory tree for changes to
 * config files using the Linux inotify filesystem watcher (recursive), and
 * on change waits a debounce period, runs a shell command (e.g.
 * "docker compose restart rsyncd"), then advances a touch-file baseline so
 * the just-consumed changes don't refire.
 *
 * The touch-file baseline mirrors the PHP version: a hidden ".last_restart"
 * marker holds the timestamp of the most recent restart. A file counts as
 * changed only if its mtime is strictly newer than that marker. The marker
 * itself and other dotfiles are ignored, so our own bookkeeping never
 * triggers a change. After each restart we bump the marker to "now", which
 * sits ahead of every config file, so quiet periods detect nothing.
 *
 * Why keep the mtime-baseline scan alongside inotify? inotify tells us WHEN
 * something happened but not WHICH file, and it can miss events under load
 * or race with editors that write-then-rename. The cheap recursive stat()
 * sweep against the baseline is the source of truth; inotify is merely the
 * event-driven wake-up that replaces blind polling. We also fall back to a
 * periodic poll so a missed inotify event can never stall the watcher.
 *
 * Usage:
 *   ./watchreload [options] <debounce_seconds> <path_to_watch> <shell_command...>
 *
 * Options:
 *   --pidfile=/path/to/my.pid   Where to write this process's PID (default:
 *                               /tmp/watchreload.pid). Removed on clean exit.
 *
 * Example:
 *   ./watchreload 5 ./rsyncd docker compose restart rsyncd
 *   ./watchreload --pidfile=./watchreload.pid 5 ./rsyncd docker compose restart rsyncd
 */

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <signal.h>
#include <time.h>
#include <limits.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/wait.h>
#include <sys/inotify.h>
#include <sys/select.h>
#include <utime.h>
#include <dirent.h>

static volatile sig_atomic_t g_running = 1;

static void on_signal(int sig) {
    (void)sig;
    g_running = 0;
}

static const char *timestamp(void) {
    static __thread char buf[64];
    time_t t = time(NULL);
    struct tm tmv;
    localtime_r(&t, &tmv);
    strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S", &tmv);
    return buf;
}

/* Write our PID to path. Returns 0 on success, -1 on failure (with a message). */
static int write_pidfile(const char *path) {
    FILE *f = fopen(path, "w");
    if (!f) {
        fprintf(stderr, "[%s] ERROR: cannot write pidfile '%s': %s\n",
                timestamp(), path, strerror(errno));
        return -1;
    }
    fprintf(f, "%d\n", getpid());
    fclose(f);
    printf("[%s] PID file:             %s\n", timestamp(), path);
    return 0;
}

/* Remove the pidfile on clean shutdown. Missing file is not an error. */
static void remove_pidfile(const char *path) {
    unlink(path); /* ignore ENOENT */
}

/* Sleep up to `secs`, returning early if a signal sets g_running == 0. */
static void interruptible_sleep(double secs) {
    const double step = 0.05; /* 50 ms granularity, like the PHP usleep(0.5s) loop */
    double elapsed = 0.0;
    while (g_running && elapsed < secs) {
        double s = step;
        if (elapsed + s > secs) s = secs - elapsed;
        struct timespec ts = { (long)(s), (long)((s - (long)s) * 1e9) };
        nanosleep(&ts, NULL);
        elapsed += s;
    }
}

/* ---- inotify setup ----------------------------------------------------- */

struct watch_ctx {
    int fd;                       /* inotify instance descriptor            */
    char root[PATH_MAX];          /* absolute path being watched            */
};

/* Recursively add watches for dir and all subdirectories beneath it. */
static void add_recursive_watches(struct watch_ctx *ctx, const char *dir) {
    DIR *d = opendir(dir);
    if (!d) {
        if (errno != ENOENT)
            fprintf(stderr, "[%s] WARN: cannot open dir '%s': %s\n",
                    timestamp(), dir, strerror(errno));
        return;
    }
    struct dirent *ent;
    while ((ent = readdir(d)) != NULL) {
        if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
            continue;
        /* skip any dir named "log" or "logs" */
        if (strcmp(ent->d_name, "log") == 0 || strcmp(ent->d_name, "logs") == 0)
            continue;
        /* skip any dir named "run" or "var" */
        if (strcmp(ent->d_name, "run") == 0 || strcmp(ent->d_name, "var") == 0)
            continue;
        
        char full[PATH_MAX];
        snprintf(full, sizeof(full), "%s/%s", dir, ent->d_name);
        struct stat st;
        if (lstat(full, &st) != 0)
            continue;
        if (S_ISDIR(st.st_mode)) {
            int wd = inotify_add_watch(ctx->fd, full, IN_CREATE | IN_MOVED_TO |
                                       IN_DELETE | IN_MODIFY | IN_ATTRIB);
            if (wd < 0 && errno != EACCES && errno != EPERM)
                fprintf(stderr, "[%s] WARN: inotify_add_watch('%s'): %s\n",
                        timestamp(), full, strerror(errno));
            add_recursive_watches(ctx, full);
        }
    }
    closedir(d);
}

static int init_inotify(const char *root, struct watch_ctx *out) {
    out->fd = inotify_init1(IN_NONBLOCK);
    if (out->fd < 0) {
        fprintf(stderr, "[%s] ERROR: inotify_init1 failed: %s\n",
                timestamp(), strerror(errno));
        return -1;
    }
    strncpy(out->root, root, PATH_MAX - 1);
    out->root[PATH_MAX - 1] = '\0';

    int wd = inotify_add_watch(out->fd, root, IN_CREATE | IN_MOVED_TO |
                               IN_DELETE | IN_MODIFY | IN_ATTRIB);
    if (wd < 0) {
        fprintf(stderr, "[%s] ERROR: inotify_add_watch('%s') failed: %s\n",
                timestamp(), root, strerror(errno));
        close(out->fd);
        return -1;
    }
    add_recursive_watches(out, root);
    return 0;
}

/* Drain any pending inotify events (non-blocking). Returns 1 if at least one
 * relevant event arrived, 0 otherwise. Used purely as a wake-up hint. */
static int drain_events(int fd) {
    unsigned char buf[(sizeof(struct inotify_event) + NAME_MAX + 1) * 64];
    ssize_t len = read(fd, buf, sizeof(buf));
    if (len <= 0)
        return 0;
    const struct inotify_event *ev = (const struct inotify_event *)buf;
    int interesting = 0;
    for (ssize_t off = 0; off < len;) {
        ev = (const struct inotify_event *)((char *)buf + off);
        if (ev->mask & (IN_CREATE | IN_MOVED_TO | IN_DELETE | IN_MODIFY | IN_ATTRIB))
            interesting = 1;
        off += sizeof(struct inotify_event) + ev->len;
    }
    return interesting;
}

/* Re-scan the tree for new subdirectories (handles newly created dirs whose
 * first inotify event may have raced before we added their watch). */
static void refresh_watches(struct watch_ctx *ctx) {
    add_recursive_watches(ctx, ctx->root);
}

/* ---- baseline / change detection --------------------------------------- */

static long get_baseline(const char *marker_path) {
    struct stat st;
    if (stat(marker_path, &st) == 0 && st.st_mtime > 0)
        return (long)st.st_mtime;
    return (long)time(NULL);
}

/* Touch the marker: create it if absent, then stamp both timestamps to now. */
static void set_baseline_now(const char *marker_path) {
    FILE *f = fopen(marker_path, "a");
    if (f) fclose(f);
    struct utimbuf tb;
    time_t now = time(NULL);
    tb.actime = now;
    tb.modtime = now;
    utime(marker_path, &tb);
}

/* Dynamic string vector. */
typedef struct {
    char **items;
    size_t count;
    size_t cap;
} strvec;

static void sv_push(strvec *v, const char *s) {
    if (v->count == v->cap) {
        v->cap = v->cap ? v->cap * 2 : 16;
        v->items = realloc(v->items, v->cap * sizeof(char *));
        if (!v->items) { perror("realloc"); exit(1); }
    }
    v->items[v->count++] = strdup(s);
}

/* Recursively walk abs_dir. For every regular file whose basename is not the
 * marker and does not begin with '.', record "<rel_prefix>/<name>" when its
 * mtime is strictly newer than baseline. rel_prefix is "" at the top level. */
static void find_changed_files(const char *abs_dir, const char *rel_prefix,
                               long baseline, const char *marker_base,
                               strvec *out) {
    DIR *d = opendir(abs_dir);
    if (!d) return;
    struct dirent *ent;
    while ((ent = readdir(d)) != NULL) {
        if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
            continue;
        char abs_full[PATH_MAX], rel_full[PATH_MAX];
        snprintf(abs_full, sizeof(abs_full), "%s/%s", abs_dir, ent->d_name);
        if (rel_prefix[0])
            snprintf(rel_full, sizeof(rel_full), "%s/%s", rel_prefix, ent->d_name);
        else
            snprintf(rel_full, sizeof(rel_full), "%s", ent->d_name);

        struct stat st;
        if (lstat(abs_full, &st) != 0)
            continue;
        if (S_ISDIR(st.st_mode)) {
            find_changed_files(abs_full, rel_full, baseline, marker_base, out);
        } else if (S_ISREG(st.st_mode)) {
            if (strcmp(ent->d_name, marker_base) == 0) continue;
            if (ent->d_name[0] == '.') continue;
            if ((long)st.st_mtime > baseline)
                sv_push(out, rel_full);
        }
    }
    closedir(d);
}

/* ---- restart action ---------------------------------------------------- */

/* Run a shell command string (e.g. "docker compose restart rsyncd") via
 * /bin/sh -c and return its exit code (-1 on fork/exec failure). */
static int run_shell_command(const char *cmd) {
    pid_t pid = fork();
    if (pid < 0) {
        fprintf(stderr, "[%s] ERROR: fork failed: %s\n", timestamp(), strerror(errno));
        return -1;
    }
    if (pid == 0) {
        execl("/bin/sh", "sh", "-c", cmd, (char *)NULL);
        perror("execl");
        _exit(127);
    }
    int status = 0;
    waitpid(pid, &status, 0);
    if (WIFEXITED(status))
        return WEXITSTATUS(status);
    return -1;
}

static void do_restart(const char *command,
                       const char *marker_path, int debounce_secs,
                       const char *const *changed_list, int n_changed) {
    printf("[%s] Change detected in rsyncd config directory:\n", timestamp());
    for (int i = 0; i < n_changed; i++)
        printf("[%s]   %s\n", timestamp(), changed_list[i]);
    printf("[%s] Waiting %ds debounce...\n", timestamp(), debounce_secs);
    fflush(stdout);

    interruptible_sleep((double)debounce_secs);
    if (!g_running)
        return;

    printf("[%s] Running: %s\n", timestamp(), command);
    fflush(stdout);

    int rc = run_shell_command(command);

    if (rc == 0) {
        printf("[%s] Command completed successfully\n", timestamp());
    } else {
        printf("[%s] ERROR: command exited with code %d\n", timestamp(), rc);
    }
    fflush(stdout);

    set_baseline_now(marker_path);
}

/* ---- main -------------------------------------------------------------- */

static void usage(const char *prog) {
    fprintf(stderr,
            "Usage: %s [options] <debounce_seconds> <path_to_watch> <shell_command...>\n"
            "\n"
            "Options:\n"
            "  --pidfile=/path/to/my.pid   Where to write this process's PID\n"
            "                              (default: /tmp/watchreload.pid).\n"
            "\n"
            "Examples:\n"
            "  %s 5 ./rsyncd docker compose restart rsyncd\n"
            "  %s --pidfile=./watchreload.pid 5 ./rsyncd docker compose restart rsyncd\n",
            prog, prog, prog);
    exit(2);
}

#define PIDFILE_OPT "--pidfile="
#define PIDFILE_OPT_LEN (sizeof(PIDFILE_OPT) - 1)

/* Pull any leading --pidfile= option out of argv. Returns the positional arg
 * count after stripping it and stores the chosen path in *pidfile_out. The
 * flag may appear anywhere before the positionals; only one is accepted. */
static int strip_pidfile_opt(int argc, char **argv, char **pidfile_out) {
    *pidfile_out = strdup("/tmp/watchreload.pid");
    if (!*pidfile_out) { perror("strdup"); exit(1); }

    char **pos = argv + 1;          /* first positional slot (after prog name) */
    int npos = 0;                   /* count of positionals kept */
    for (int i = 1; i < argc; i++) {
        if (strncmp(argv[i], PIDFILE_OPT, PIDFILE_OPT_LEN) == 0) {
            const char *val = argv[i] + PIDFILE_OPT_LEN;
            if (*val == '\0') {
                fprintf(stderr, "ERROR: --pidfile requires a path argument\n");
                usage(argv[0]);
            }
            free(*pidfile_out);
            *pidfile_out = strdup(val);
            if (!*pidfile_out) { perror("strdup"); exit(1); }
            continue;               /* drop the option from the positional list */
        }
        *pos++ = argv[i];
        npos++;
    }
    return npos;                    /* number of remaining positional args */
}

int main(int argc, char **argv) {
    /* Extract --pidfile= (optional) so it isn't swallowed into the shell command. */
    char *pidfile_path;
    int pos_count = strip_pidfile_opt(argc, argv, &pidfile_path);

    if (pos_count < 3)
        usage(argv[0]);

    /* Write our PID now, early, so supervisors can track us immediately. */
    if (write_pidfile(pidfile_path) != 0)
        return 1;

    /* Positional args are now argv[1..]: debounce, path, then the command. */
    /* Parse debounce */
    char *endp;
    long debounce = strtol(argv[1], &endp, 10);
    if (*endp != '\0' || debounce < 0) {
        fprintf(stderr, "ERROR: invalid debounce seconds '%s'\n", argv[1]);
        remove_pidfile(pidfile_path);
        usage(argv[0]);
    }

    /* Path to watch */
    const char *watch_arg = argv[2];
    char resolved[PATH_MAX];
    if (!realpath(watch_arg, resolved)) {
        fprintf(stderr, "ERROR: watch path '%s' does not exist or is not resolvable (%s)\n",
                watch_arg, strerror(errno));
        remove_pidfile(pidfile_path);
        return 1;
    }
    struct stat dst;
    if (stat(resolved, &dst) != 0 || !S_ISDIR(dst.st_mode)) {
        fprintf(stderr, "ERROR: '%s' is not a directory\n", resolved);
        remove_pidfile(pidfile_path);
        return 1;
    }

    /* Positional layout after stripping: argv[1]=debounce, argv[2]=path,
     * argv[3..pos_count]=shell command words. Join those into one command. */
    size_t cmd_len = 0;
    for (int i = 3; i <= pos_count; i++) cmd_len += strlen(argv[i]) + 1;
    char *command = malloc(cmd_len + 1);
    if (!command) { perror("malloc"); return 1; }
    command[0] = '\0';
    for (int i = 3; i <= pos_count; i++) {
        if (i > 3) strcat(command, " ");
        strcat(command, argv[i]);
    }

    /* Marker file inside the watched dir (buffer widened for the suffix) */
    char marker_path[PATH_MAX + 32];
    snprintf(marker_path, sizeof(marker_path), "%s/.last_restart", resolved);
    const char *marker_base = ".last_restart";

    /* Install signal handlers */
    struct sigaction sa;
    memset(&sa, 0, sizeof(sa));
    sa.sa_handler = on_signal;
    sigemptyset(&sa.sa_mask);
    sigaction(SIGINT, &sa, NULL);
    sigaction(SIGTERM, &sa, NULL);
    sigaction(SIGHUP, &sa, NULL);

    printf("[%s] RSYNCD Config Watcher starting (inotify)...\n", timestamp());
    printf("[%s] Monitoring directory: %s\n", timestamp(), resolved);
    printf("[%s] Baseline marker:      %s\n", timestamp(), marker_path);
    printf("[%s] Debounce period:      %lds\n", timestamp(), debounce);
    printf("[%s] Command to run:       %s\n", timestamp(), command);
    printf("[%s] Press Ctrl+C to stop\n\n", timestamp());
    fflush(stdout);

    /* Set up inotify */
    struct watch_ctx ctx;
    if (init_inotify(resolved, &ctx) != 0)
        return 1;

    /* Initial restart with current config, then establish baseline. */
    const char *initial[] = { "(initial start)" };
    do_restart(command, marker_path, (int)debounce, initial, 1);
    set_baseline_now(marker_path);

    /* Main loop: block on inotify OR a 1-second timeout, whichever comes
     * first. On wake (event or timeout), sweep for changed files vs baseline.
     * This gives event-driven responsiveness with a safety-net poll. */
    const long POLL_FALLBACK_SECS = 1;

    while (g_running) {
        fd_set rfds;
        FD_ZERO(&rfds);
        FD_SET(ctx.fd, &rfds);
        int maxfd = ctx.fd + 1;
        struct timeval tv = { POLL_FALLBACK_SECS, 0 };

        int sel = select(maxfd, &rfds, NULL, NULL, &tv);
        if (sel < 0) {
            if (errno == EINTR) continue; /* interrupted by signal */
            fprintf(stderr, "[%s] ERROR: select: %s\n", timestamp(), strerror(errno));
            break;
        }

        if (sel > 0 && FD_ISSET(ctx.fd, &rfds)) {
            drain_events(ctx.fd);
            /* New directories may have appeared; refresh watches. */
            refresh_watches(&ctx);
        }

        if (!g_running)
            break;

        long baseline = get_baseline(marker_path);
        strvec changed = {0};
        find_changed_files(resolved, "", baseline, marker_base, &changed);

        if (changed.count > 0) {
            do_restart(command, marker_path, (int)debounce,
                       (const char *const *)changed.items, (int)changed.count);
        }

        for (size_t i = 0; i < changed.count; i++) free(changed.items[i]);
        free(changed.items);
    }

    close(ctx.fd);
    remove_pidfile(pidfile_path);
    printf("\n[%s] Watcher stopped.\n", timestamp());
    free(command);
    free(pidfile_path);
    return 0;
}

So I ended up having a chat:

analyze the watchreload.c code and tell me if there's unused code.

-- it said all the code is used - that was a kind of static analysis

does it scan over all the files? is that necessary?

-- yes, and explained the way inotify is typically used - inotify fires when the contents of a directory change. the program scans the directory to find the file that changed.

I think a full scan isn't necessary. I do nothing with the changed files. All I want to know is that a change happened, so I can run the command to restart the container.

-- it replaced the full scan with code to "snapshot" the entire tree, to handle the risks of uncaught events. 🤐 It literally replaced polling with more polling. Then it built it, and it worked.

😮‍💨

I swear, I think this program should be under 200 lines long. It’d be even shorter in Bash.

I also found this feature that may have rendered this program superfluous.

admin
Author: admin

This is the server’s system administrator. This site is undergoing some changes.