Puppet: System Administration Automated

Puppet Training Schedule
Next Class July 27-29
New York, New York
Discount before July 1st

Zabbix Agent recipe

Problem

Ensure that Zabbix agent is installed on a host and is configured properly.

Solution

$zabbix_user_parameters = []

class zabbix-agent {
    $zabbix_server = "XXX.XXX.XXX.XXX"
    $zabbix_config_dir = "/etc/zabbix"
    $zabbix_agent_conf = "$zabbix_config_dir/zabbix_agent.conf"
    $zabbix_agentd_conf = "$zabbix_config_dir/zabbix_agentd.conf"

    package {
        "zabbix-agent":
            ensure => installed
    }

    service {
        "zabbix-agent":
            enable => true,
            ensure => running,
            hasstatus => true,
            require => Package["zabbix-agent"]
    }

    file {
        $zabbix_config_dir:
            ensure => directory,
            owner => root,
            group => root,
            mode => 0755,
            require => Package["zabbix-agent"];

        $zabbix_agent_conf:
            owner => root,
            group => root,
            mode => 0644,
            content => template("zabbix_agent_conf.erb"),
            require => Package["zabbix-agent"];

        $zabbix_agentd_conf:
            owner => root,
            group => root,
            mode => 0644,
            content => template("zabbix_agentd_conf.erb"),
            require => Package["zabbix-agent"];
    }
}

Here is zabbix_agent_conf.erb:

# This is config file for zabbix_agent
# To get more information about ZABBIX, 
# go http://www.zabbix.com

# IP address of ZABBIX server
# Connections from other hosts will be denied

Server=<%= zabbix_server %>

# Spend no more than Timeout seconds on processing
# Must be between 1 and 30

Timeout=3

####### USER-DEFINED MONITORED PARAMETERS #######
# Format: UserParameter=<key>,<shell command>
# Note that shell command must not return empty string or EOL only
#UserParameter=system.test,who|wc -l
### Set of parameter for monitoring MySQL server (v3.23.42 and later)
### Change -u<username> and add -p<password> if required
#UserParameter=mysql.ping,mysqladmin -uroot ping|grep alive|wc -l
#UserParameter=mysql.uptime,mysqladmin -uroot status|cut -f2 -d":"|cut -f1 -d"T"
#UserParameter=mysql.threads,mysqladmin -uroot status|cut -f3 -d":"|cut -f1 -d"Q"
#UserParameter=mysql.questions,mysqladmin -uroot status|cut -f4 -d":"|cut -f1 -d"S"
#UserParameter=mysql.slowqueries,mysqladmin -uroot status|cut -f5 -d":"|cut -f1 -d"O"
#UserParameter=mysql.qps,mysqladmin -uroot status|cut -f9 -d":"
#UserParameter=mysql.version,mysql -V

<% zabbix_user_parameters.each do |zabbix_user_parameter| %>UserParameter=<%= zabbix_user_parameter %>
<% end %>

And here is zabbix_agentd_conf.erb:

# This is config file for zabbix_agentd
# To get more information about ZABBIX, go http://www.zabbix.com

############ GENERAL PARAMETERS #################

# List of comma delimited IP addresses (or hostnames) of ZABBIX servers. 
# No spaces allowed. First entry is used for sending active checks.
# Note that hostnames must resolve hostname->IP address and
# IP address->hostname.

Server=<%= zabbix_server %>

# Server port for sending active checks

#ServerPort=10051

# Unique hostname. Required for active checks.

Hostname=<%= hostname %>

# Listen port. Default is 10050

#ListenPort=10050

# IP address to bind agent
# If missing, bind to all available IPs

#ListenIP=127.0.0.1

# Number of pre-forked instances of zabbix_agentd.
# Default value is 5
# This parameter must be between 1 and 16

StartAgents=5

# How often refresh list of active checks. 2 minutes by default.

#RefreshActiveChecks=120

# Disable active checks. The agent will work in passive mode listening server.

#DisableActive=1

# Enable remote commands for ZABBIX agent. By default remote commands disabled.

#EnableRemoteCommands=1

# Specifies debug level
# 0 - debug is not created
# 1 - critical information
# 2 - error information
# 3 - warnings (default)
# 4 - for debugging (produces lots of information)

DebugLevel=3

# Name of PID file

PidFile=/var/run/zabbix-agent/zabbix_agentd.pid

# Name of log file.
# If not set, syslog will be used

LogFile=/var/log/zabbix-agent/zabbix_agentd.log

# Spend no more than Timeout seconds on processing
# Must be between 1 and 30

Timeout=3

####### USER-DEFINED MONITORED PARAMETERS #######
# Format: UserParameter=<key>,<shell command>
# Note that shell command must not return empty string or EOL only
#UserParameter=system.test,who|wc -l
### Set of parameter for monitoring MySQL server (v3.23.42 and later)
### Change -u<username> and add -p<password> if required
#UserParameter=mysql.ping,mysqladmin -uroot ping|grep alive|wc -l
#UserParameter=mysql.uptime,mysqladmin -uroot status|cut -f2 -d":"|cut -f1 -d"T"
#UserParameter=mysql.threads,mysqladmin -uroot status|cut -f3 -d":"|cut -f1 -d"Q"
#UserParameter=mysql.questions,mysqladmin -uroot status|cut -f4 -d":"|cut -f1 -d"S"
#UserParameter=mysql.slowqueries,mysqladmin -uroot status|cut -f5 -d":"|cut -f1 -d"O"
#UserParameter=mysql.qps,mysqladmin -uroot status|cut -f9 -d":"
#UserParameter=mysql.version,mysql -V

<% zabbix_user_parameters.each do |zabbix_user_parameter| %>UserParameter=<%= zabbix_user_parameter %>
<% end %>

Including Automatic Update to Zabbix Server

I enhanced this solution to also update the Zabbix Server with the new machine information. The solution uses curl (wget could work, but I was using custom certificates that wget didn't seem to support).

Node Configuration

First, you need to specify the required parameters globally for the system:

node example.node.com {
    $shortname = "example" # the name Zabbix Server uses for this node
    $zabbix_server = "XXX.XXX.XXX.XXX"
    $zabbixport = 443
    $zabbixadmin = "ADMINUSER"
    $zabbixpassword = "ADMINPASSWORD"
    include zabbix-agent
}

init.pp changes

Next, the Zabbix Agent init.pp needs to create a script to update the system:

$zabbix_user_parameters = []

class zabbix-agent {
    $zabbix_config_dir = "/etc/zabbix"
    $zabbix_agentd_conf = "$zabbix_config_dir/zabbix_agentd.conf"
    $zabbix_register_sh = "$zabbix_config_dir/zabbix_register.sh"

    package {
        "zabbix-agent":
            ensure => installed;
        "curl":
            ensure => installed,
            before => File[$zabbix_register_sh];
    }

    service {
        "zabbix-agent":
            enable => true,
            ensure => running,
            hasstatus => true,
            require => Package["zabbix-agent"]
    }
    
    file {
        $zabbix_config_dir:
            ensure => directory,
            owner => root,
            group => root,
            mode => 0755,
            require => Package["zabbix-agent"];    

        $zabbix_register_sh:
            owner => root,
            group => root,
            mode => 0700,
            content => template("zabbix/zabbix_register.sh.erb"),
            require => Package["zabbix-agent"];

        $zabbix_agent_conf:
            owner => root,
            group => root,
            mode => 0644,
            content => template("zabbix_agent_conf.erb"),
            require => Package["zabbix-agent"];

        $zabbix_agentd_conf:
            owner => root,
            group => root,
            mode => 0644,
            content => template("zabbix/zabbix_agentd.conf.erb"),
            require => Package["zabbix-agent"];
    }
    
    exec { $zabbix_register_sh:  }
   
}

Script to notify Zabbix

Now the template for the script, which I put in modules/zabbix-agent/templates:

#!/bin/bash

##"curl --insecure..." is used by me because the server is using a self signed
##certificate and I'm not worried about MITM etc. attacks.

HOST=<%= shortname %>
IP=<%= ipaddress %>

BASE=/tmp/zabpush
COOKIE=$BASE.cookies.txt
OUT1=$BASE.login.html
OUT2=$BASE.host.html
HEADERS=$BASE.headers.txt

USERNAME=<%= zabbixadmin %>
PASSWORD=<%= zabbixpassword %>

URL=https://<%= zabbixserver %>:<%= zabbixport %>

curl --insecure -c $COOKIE -D $HEADERS -d "login=1&form_refresh=1&name=$USERNAME&password=$PASSWORD&enter=Enter" $URL/index.php > $OUT1

SESSION=`awk '/<%= zabbixserver %>/ { print $7 }' $COOKIE`
SID=`echo $SESSION | cut -c 17-`

# Normally 1 long line -- split for readability
FORM="sid=$SID&form=Create+Host&form_refresh=2&config=0&groupid=0&groups%5B5%5D=5&templates%5B10001%5D=Template_Linux&ipmi_port=623&ipmi_privilege=2&ipmi_username="
FORM="$FORM&ipmi_password=&devicetype=&name=&os=&serialno=&tag=&macaddress=&hardware=&software=&contact=&location=&notes=&ext_host_profiles%5Bdevice_alias%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_type%5D=&ext_host_profiles%5Bdevice_chassis%5D=&ext_host_profiles%5Bdevice_os%5D=&ext_host_profiles%5Bdevice_os_short%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_hw_arch%5D=&ext_host_profiles%5Bdevice_serial%5D=&ext_host_profiles%5Bdevice_model%5D=&ext_host_profiles%5Bdevice_tag%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_vendor%5D=&ext_host_profiles%5Bdevice_contract%5D=&ext_host_profiles%5Bdevice_who%5D=&ext_host_profiles%5Bdevice_status%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_app_01%5D=&ext_host_profiles%5Bdevice_app_02%5D=&ext_host_profiles%5Bdevice_app_03%5D=&ext_host_profiles%5Bdevice_app_04%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_app_05%5D=&ext_host_profiles%5Bdevice_url_1%5D=&ext_host_profiles%5Bdevice_url_2%5D=&ext_host_profiles%5Bdevice_url_3%5D="
FORM="$FORM&ext_host_profiles%5Bdevice_networks%5D=&ext_host_profiles%5Bdevice_notes%5D=&ext_host_profiles%5Bdevice_hardware%5D=&ext_host_profiles%5Bdevice_software%5D="
FORM="$FORM&ext_host_profiles%5Bip_subnet_mask%5D=&ext_host_profiles%5Bip_router%5D=&ext_host_profiles%5Bip_macaddress%5D=&ext_host_profiles%5Boob_ip%5D="
FORM="$FORM&ext_host_profiles%5Boob_subnet_mask%5D=&ext_host_profiles%5Boob_router%5D=&ext_host_profiles%5Bdate_hw_buy%5D=&ext_host_profiles%5Bdate_hw_install%5D="
FORM="$FORM&ext_host_profiles%5Bdate_hw_expiry%5D=&ext_host_profiles%5Bdate_hw_decomm%5D=&ext_host_profiles%5Bsite_street_1%5D=&ext_host_profiles%5Bsite_street_2%5D="
FORM="$FORM&ext_host_profiles%5Bsite_street_3%5D=&ext_host_profiles%5Bsite_city%5D=&ext_host_profiles%5Bsite_state%5D=&ext_host_profiles%5Bsite_country%5D="
FORM="$FORM&ext_host_profiles%5Bsite_zip%5D=&ext_host_profiles%5Bsite_rack%5D=&ext_host_profiles%5Bsite_notes%5D=&ext_host_profiles%5Bpoc_1_name%5D="
FORM="$FORM&ext_host_profiles%5Bpoc_1_email%5D=&ext_host_profiles%5Bpoc_1_phone_1%5D=&ext_host_profiles%5Bpoc_1_phone_2%5D=&ext_host_profiles%5Bpoc_1_cell%5D="
FORM="$FORM&ext_host_profiles%5Bpoc_1_screen%5D=&ext_host_profiles%5Bpoc_1_notes%5D=&ext_host_profiles%5Bpoc_2_name%5D=&ext_host_profiles%5Bpoc_2_email%5D="
FORM="$FORM&ext_host_profiles%5Bpoc_2_phone_1%5D=&ext_host_profiles%5Bpoc_2_phone_2%5D=&ext_host_profiles%5Bpoc_2_cell%5D=&ext_host_profiles%5Bpoc_2_screen%5D="
FORM="$FORM&ext_host_profiles%5Bpoc_2_notes%5D=&host=$HOST&newgroup=&dns=&ip=$IP&useip=1&port=10050&proxy_hostid=0&status=0&save=Save"

curl --insecure -c $COOKIE -D $HEADERS -b zbx_sessionid=$SESSION -d "$FORM" $URL/hosts.php > $OUT2

rm $BASE*

Discussion

5/2/2009 - Updated and verified to work with Debian Sid.