Using pcsk to Supervise mythbackend

If mythbackend has ever crashed on you, odds are that you didn't know and you missed out on recording a favourite programme. Here is a nifty rc script I wrote using the default /etc/init.d/mythtv-backend script in Ubuntu 7.04 (Feisty) as a base, but modified to have the following improvements:

1) The Process Creation's Swiss-Army Knife (PCSK) is used to supervise the mythbackend process and respawn it if it dies. Alternatives to pcsk include daemontools and psmon, but I like pcsk as it is small and simple and does exactly what I need.
2) The script is more clever about whether a process is actually running, rather than just checking for the existence of a .pid file (which will never work if the process crashed instead of being deliberately stopped).
3) You can simply run /etc/init.d/mythtv-backend start and know that no matter what the status was previously, mythbackend will be running afterwards (useful if another process or script is used to start mythbackend).
4) Non-root users stand a good chance of being able to determine the status of mythbackend without requiring sudo.
5) Additional status information is provided using Andrew Ruthven's mythtv-status perl script. This isn't essential, but is used if you have it installed.
6) Behaviour of force-reload is clearer

A disadvantage for you may be that this method loses the ability to 'nice' the mythbackend process (eg using /etc/default/mythtv-backend). I wasn't using nice, so it was no problem for me.

PCSK

First, download, compile and install pcsk from http://www.nix.hu/projects/pcsk/. Something along the lines of:

cd
mkdir pcsk
cd pcsk
wget http://downloads.nix.hu/downloads/pcsk/pcsk-0.0.5.tar.bz2
tar xjf pcsk-0.0.5.tar.bz2
cd pcsk-0.0.5
make install

mythtv-backend.pcsk

Then, using your favourite text editor, create the following script in the appropriate place for your distro. For my Ubuntu installation I chose /etc/init.d/mythtv-backend.pcsk.

/etc/init.d/mythtv-backend.pcsk

#! /bin/sh
### BEGIN INIT INFO
# Provides:          mythtv-backend
# Required-Start:    $local_fs
# Required-Stop:     $local_fs
# Default-Start:     24
# Default-Stop:      S
# Short-Description: Start/Stop the MythTV server.
### END INIT INFO
#
# modified mythtv-backend rc script
# v0.2 by DJK 28th Nov 2007:
#       Added pcsk supervision to restart if it dies
#       Had to lose NICE for now, sorry.  Might be able to use renice sometime in the future?
#       Call mythtv-status script for extra details
#

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/mythbackend
NAME="mythbackend"
DESC="MythTV server"

# Check that the mythbackend binary exists
if ! [ -x "$DAEMON" ] ; then
        echo "Could not find $DAEMON binary"
        exit 1
fi

# Check that the pcsk binary exists somewhere in the path (I choose /usr/local/bin)
if ! PCSKBINARY="`which pcsk`" > /dev/null 2>&1; then
        echo "Could not find pcsk binary to supervise $NAME.  Try www.nix.hu/projects/pcsk..."
        exit 1
fi

# Arguments to give to the mythtv-status script
MYTHTVSTATUSARGS="-c"

# Check that the  mythtv-status perl script exists somewhere in the path (I choose /usr/local/bin)
if ! MYTHTVSTATUSBINARY="`which mythtv-status`" > /dev/null 2>&1; then
        # It's not disasterous if it's not there - just let the user know where to find it if they try status
        MYTHTVSTATUSBINARY="echo mythtv-status perl script not found.  Try www.etc.gen.nz/projects/mythtv/mythtv-status.html"
        MYTHTVSTATUSARGS=
fi

set -e

. /lib/lsb/init-functions

USER=mythtv
RUNDIR=/var/run/mythtv
# mythbackend arguments - I've removed --daemon since pcsk will daemonise it for me
ARGS=" --logfile /var/log/mythtv/mythbackend.log --pidfile $RUNDIR/$NAME.pid"
EXTRA_ARGS=""
NICE=0

# The pid and log files for pcsk to use
PCSKPIDFILE="$RUNDIR/$NAME-pcsk.pid"
PCSKLOGFILE="/var/log/mythtv/mythbackend-pcsk.log"

# Command line options for pcsk:
# -r: restart the program if it dies
# -e: log stderr, if there is any
# -o: log stdout, if there is any
# -0: don't restart if the exit code was 0
# -w3: wait 3 sec before restart (default in original script)
# -i10: increment each wait between restarts by 10s
# -c10: restart up to 10 times if the restart is unsuccessful, before giving up
# -m60: restart wasn't successful if the program exits within 60 sec
# -p $PIDFILE: use that file for the pid
# -l $PCSKLOGFILE: log file to keep track of things - good to set up logrotate
PCSKOPTS="-reo0 -w3 -i10 -c10 -m60 -p $PCSKPIDFILE -l $PCSKLOGFILE"
# Check for alternate options for mythtv-backend
if [ -f /etc/default/mythtv-backend ]; then
  . /etc/default/mythtv-backend
fi

ARGS="$ARGS $EXTRA_ARGS"

PCSKARGS="$PCSKOPTS $DAEMON -- $ARGS"

# If the run directory doesn't exist, then create it if we can and chown it to $USER
# I'm assuming if it's already there its permissions and ownership are OK (cheat...)
# This makes it easier for non-root users to still run this script to get status info
if ! [ -d "$RUNDIR" ] ; then
        if [ "`id -un`" = "root" ] ; then
                mkdir -p "$RUNDIR"
                chown -R "$USER" "$RUNDIR"
        else
                echo "Could not create $RUNDIR.  You need to be root to do that.  Try sudo."
                exit 1
        fi
fi

unset DISPLAY
unset SESSION_MANAGER

case "$1" in
  start)
        [ "`id -un`" = "root" ] || {
                echo "Failed.  You need to be root to $1 $DESC.  Try sudo."
                exit 1
        }
        # Check if pcsk is already supervising this process
        # by checking for the existance of a pidfile and that it's a currently running process
        PCSKPID=
        DAEMONPID=
        if [ -e "$PCSKPIDFILE" ] ; then
                read PCSKPID < "$PCSKPIDFILE"
                if [ -n "$PCSKPID" ] && [ -d "/proc/$PCSKPID" ] ; then
                        # pcsk is running, what about our daemon?
                        if [ -e "$RUNDIR/$NAME.pid" ] ; then
                                read DAEMONPID < "$RUNDIR/$NAME.pid"
                                if [ -n "$DAEMONPID" ] && [ -d "/proc/$DAEMONPID" ] ; then
                                        # daemon is running too
                                        echo "pcsk($NAME) (pid $PCSKPID) is already running, supervising $NAME (pid $DAEMONPID) "
                                        echo "Use restart if you need."
                                        exit 1
                                fi
                                # daemon isn't running at the moment (pcsk might be waiting to respawn it)
                                # anyway, take the opportunity to restart everything now
                                echo "pcsk($NAME) (pid $PCSKPID) is running but $NAME is currently stopped.  Restarting now..."
                                $0 restart
                                exit $?
                        fi
                fi
        fi
        echo -n "Starting $DESC: pcsk($NAME) "
        # NICE gets ignored because pcsk doesn't support it and start-stop-daemon would just nice pcsk, not the daemon.
        # Maybe we could somehow use renice?  Later...
        if ! start-stop-daemon --start --pidfile "$PCSKPIDFILE" --chuid $USER --exec "$PCSKBINARY" -- $PCSKARGS ; then
                echo "failed (start-stop-daemon exited with code $?)"
                exit 1
        fi
        echo "."
        ;;
  stop)        [ "`id -un`" = "root" ] || {
                echo "Failed.  You need to be root to $1 $DESC.  Try sudo."
                exit 1
        }
        echo -n "Stopping $DESC: pcsk($NAME) "
        start-stop-daemon --stop --oknodo --pidfile "$PCSKPIDFILE" --chuid $USER --retry 5 --exec "$PCSKBINARY" -- $PCSKARGS
        [ -e "$PCSKPIDFILE" ] && rm -f "$PCSKPIDFILE"
        [ -e "$RUNDIR/$NAME.pid" ] && rm -f "$RUNDIR/$NAME.pid"
        echo "."
        ;;
  restart)
        $0 stop
        sleep 3
        $0 start
        ;;
  force-reload)
        echo "force-reload not supported, restarting instead..."
        $0 restart
        ;;
  status)
        # We have an advantage in this mess that we know what the pidfiles (and therefore pids) are
        # Poor-man's status can be had by checking for the existance of a pidfile
        # and then that it's a currently running process by looking for that directory in /proc
        PCSKPID=
        DAEMONPID=
        EXITBAD=FALSE
        if [ -e "$PCSKPIDFILE" ] ; then
                read PCSKPID < "$PCSKPIDFILE"
                if [ -n "$PCSKPID" ] && [ -d "/proc/$PCSKPID" ] ; then
                        echo "pcsk($NAME) (pid $PCSKPID) is running..."
                else
                        echo "pcsk($NAME) is dead but pid file exists"
                        EXITBAD=TRUE
                fi
        else
                echo "pcsk($NAME) is stopped"
                EXITBAD=TRUE
        fi
        if [ -e "$RUNDIR/$NAME.pid" ] ; then
                read DAEMONPID < "$RUNDIR/$NAME.pid"
                if [ -n "$DAEMONPID" ] && [ -d "/proc/$DAEMONPID" ] ; then
                        echo "$NAME (pid $DAEMONPID) is running..."
                else
                        echo "$NAME is dead but pid file exists"
                        EXITBAD=TRUE
                fi
        else
                echo "$NAME is stopped"
                EXITBAD=TRUE
        fi
        [ "$EXITBAD" = "TRUE" ] && exit 1
        $MYTHTVSTATUSBINARY $MYTHTVSTATUSARGS
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|status|force-reload(unsupported)}" >&2
        exit 1
        ;;
esac

exit 0

Stop mythbackend first.

sudo /etc/init.d/mythtv-backend stop

Keep a copy of the original script in case you need it, but make the new one the default.

cd /etc/init.d
sudo mv mythtv-backend mythtv-backend.original
sudo ln -s mythtv-backend.pcsk mythtv-backend

Start mythbackend again.

sudo mythtv-backend start