Mutual Restart Recipe

One common pattern in cfengine work is to have cfengine ensure that cron is running, and to have a cron job that ensures cfengine is running. That way, if cfengine dies, cron will restart it, and vice versa. I didn't find anything similar documented in Puppet, so here's my first attempt. If others want to make it more portable, or if you want to point me at where this has already been written up, that's fine.

  cron { restart-puppetmaster:
    command => "if [ -e /var/run/puppetmasterd.pid ]; then ps uw -p `cat /var/run/puppetmasterd.pid` | grep -q ' ruby /usr/sbin/puppetmasterd' || (rm /var/run/puppetmasterd.pid; /etc/init.d/puppetmaster start); else /etc/init.d/puppetmaster start; fi",
    user => root,
    minute => 0
  }
  cron { restart-puppet:
    command => "if [ -e /var/run/puppetd.pid ]; then ps uw -p `cat /var/run/puppetd.pid` | grep -q ' ruby /usr/sbin/puppetd' || (rm /var/run/puppetd.pid; /etc/init.d/puppet start); else /etc/init.d/puppet start; fi",
    user => root,
    minute => 0
  }
  service { cron:
    ensure => running,
    enable => true
  }

Example:

gold:~# ps aux | egrep 'puppet|cron'
root      1199  0.0  0.5   2208   668 ?        Ss   May05   0:00 /usr/sbin/cron
puppet   22383  0.4 25.8  44488 33860 ?        Ssl  15:34   0:36 ruby /usr/sbin/puppetmasterd
root     23074  0.0 14.5  31800 19068 ?        Ssl  17:22   0:00 ruby /usr/sbin/puppetd --server gold --factsync
gold:~# kill -9 1199
gold:~# puppetd -vt --factsync --server gold
info: Retrieving facts
info: Loading fact virtual
info: Caching configuration at /etc/puppet/localconfig.yaml
info: /: Mounted /
notice: Starting configuration run
notice: //gold/cae-host/service=cron/ensure: ensure changed 'stopped' to 'running'
notice: Finished configuration run in 1.23 seconds
gold:~# crontab -l
# HEADER This file was autogenerated at Fri May 18 17:19:45 -0500 2007 by puppet.  While it
# HEADER can still be managed manually, it is definitely not recommended.
# HEADER Note particularly that the comments starting with 'Puppet Name' should
# HEADER not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: restart-puppetmaster
0 * * * * if [ -e /var/run/puppetmasterd.pid ]; then ps uw -p `cat /var/run/puppetmasterd.pid` | grep -q ' ruby /usr/sbin/puppetmasterd' || (rm /var/run/puppetmasterd.pid; /etc/init.d/puppetmaster start); fi
# Puppet Name: restart-puppet
0 * * * * if [ -e /var/run/puppetd.pid ]; then ps uw -p `cat /var/run/puppetd.pid` | grep -q ' ruby /usr/sbin/puppetd' || (rm /var/run/puppetd.pid; /etc/init.d/puppet start); fi
gold:~# kill -9 22383 23074
gold:~# if [ -e /var/run/puppetmasterd.pid ]; then ps uw -p `cat /var/run/puppetmasterd.pid` | grep -q ' ruby /usr/sbin/puppetmasterd' || (rm /var/run/puppetmasterd.pid; /etc/init.d/puppetmaster start); fi
Starting puppet configuration management tool master server.
gold:~# if [ -e /var/run/puppetd.pid ]; then ps uw -p `cat /var/run/puppetd.pid` | grep -q ' ruby /usr/sbin/puppetd' || (rm /var/run/puppetd.pid; /etc/init.d/puppet start); fi
Starting puppet configuration management tool.
gold:~# ps aux | egrep 'puppet|cron'
root     24417  0.0  0.6   2208   888 ?        Ss   17:42   0:00 /usr/sbin/cron
puppet   24444  0.2 20.8  38256 27380 ?        Ssl  17:43   0:00 ruby /usr/sbin/puppetmasterd
root     24468  0.4 18.3  35184 24124 ?        Ssl  17:44   0:00 ruby /usr/sbin/puppetd --server gold --factsync