Setup and configure Nagios Client (NSCA) on CentOS/RHEL 6.3

NSCA (Nagios Service Check Acceptor) is a Linux/Unix daemon allows you to integrate passive alerts and checks from remote machines and applications with Nagios. Useful for processing security alerts, as well as redundant and distributed Nagios setups.

This article will guide you through the installation and configuration steps of Nagios Client – NSCA on CentOS 6.3.

Server side Nagios core and plugin package version:

   Nagios Core: nagios-4.0.0
   Nagios Plugin: nagios-plugins-1.4.16

To install Nagios please see my Nagios Installation Docs.

For testing purpose we have setup following machines:

IP Address Hostname
Nagios Server 10.0.1.10 mon001
Nagios Client 10.0.1.20 haproxy001

1) Prerequisite

  • Nagios server in working condition.

  • Following dependent package to compile and install.

   yum install gcc glibc glibc-common xinetd 

Create nagios user and group by which we will be installing NSCA

   useradd -m nagios
password nagios

2) Installation

2.1) Create a directory where you would download nsca source:

   mkdir /usr/local/src
   cd /usr/local/src

2.2) First of all we need to Download/Untar/Compile all necessary files for NSCA-Plugin:

   wget http://downloads.sourceforge.net/project/nagios/nsca-2.x/nsca-2.9.1/nsca-2.9.1.tar.gz
   tar -xzvf nsca-2.9.1.tar.gz
   cd nsca-2.9.1
   ./configure --with-nsca-user=nagios --with-nsca-grp=nagios
   make all

2.2) Installing nsca binary to there respective directory and fixing permission:

   cp sample-config/nsca.cfg sample-config/send_nsca.cfg /usr/local/nagios/etc/
   cp src/send_nsca src/nsca /usr/local/nagios/bin/
   chown nagios:nagios /usr/local/nagios/bin/nsca /usr/local/nagios/bin/send_nsca
   chown nagios.nagcmd /usr/local/nagios/etc/nsca.cfg /usr/local/nagios/etc/send_nsca.cfg
   chmod g+r /usr/local/nagios/etc/nsca.cfg

3) Configuring NSCA
Apart for setting “server_address” and “debug” options there is no need to change any other configuration parameter.
The server address option lets you specific an IP to bind to. This is used when there is more than one network interface card. We will not change this entry in our setup as we only have one NIC card.

   #server_address=192.168.1.207  # My local IP address

Set the NSCA debug level to 1 to check NSCA daemon is working fine or not. NSCA writes it logs to the standard syslog facility “(i.e. /var/log/messages)”

   vi /usr/local/nagios/etc/nsca.cfg
   debug=1

4) Validation
The next step would be to start up NSCA

   /usr/local/nagios/bin/nsca -c /usr/local/nagios/etc/nsca.cfg

Check for nsca process is running and tcp socket is open

   ps -ef | grep -v grep | grep -i nsca 
   netstat -planet | grep 5667 

Now we will configure NSCA as a Service in Xinetd

Add the following line to your /etc/services file

   vi /etc/services
   nsca    5667/tcp   # NSCA

Copy “nsca.xinetd” file to xinetd config directory.

   cp sample-config/nsca.xinetd /etc/xinetd.d/nsca

Kill the NSCA daemon process.

   Kill < /var/run/nsca.pid
   rm /var/run/nsca.pid

Replace the ipaddress fields with the IP addresses of hosts which are allowed to connect to the NSCA daemon and restart xinetd

   vi /etc/xinetd.d/nsca
   only_from       = "ipaddress"
   /etc/rc.d/init.d/xinetd restart

Check for nsca process is running and tcp socket is open

   netstat -planet | grep 5667 

4) Testing
As now NSCA is running now we need to send some data to Nagios via NSCA. The format for a service check packet using NSCA is

   hostname[tab]svc_description[tab]return_code[tab]plugin_output[newline].

Now create a sample test file and run nsca.

   vi /tmp/tmp
   localhost       TestMessage     0       This is a test message.

   /usr/local/nagios/bin/send_nsca localhost -c /usr/local/nagios/etc/send_nsca.cfg < /tmp/tmp

If it sent, then we should get a message saying “1 data packet(s) sent to host successfully.” and log message will look like

   tail -f /var/log/messages
   Feb 23 17:10:05 bhdhcp06941 nsca[24490]: Time difference in packet: 0 seconds for host localhost
   Feb 23 17:10:05 bhdhcp06941 nsca[24490]: SERVICE CHECK -> Host Name: 'localhost', Service Description: 'TestMessage', Return Code: '0', Output: 'This is a test message.'
   Feb 23 17:10:05 bhdhcp06941 nsca[24490]: Attempting to write to nagios command pipe
   Feb 23 17:10:05 bhdhcp06941 nsca[24490]: End of connection...
   Feb 23 17:10:05 bhdhcp06941 nagios: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;TestMessage;0;This is a test message.
   Feb 23 17:10:05 bhdhcp06941 nagios: Warning:  Passive check result was received for service 'TestMessage' on host 'localhost', but the service could not be found!

5) Nagios Configuration

If everything is running smoothly so far, the final step would be to create the service to process your passive checks in Nagios. We are going to use the check_dummy as our check_command in the service we must define that command as well.

   vi /usr/local/nagios/etc/objects/commands.cfg

   define command{
       command_name    check_dummy
       command_line    $USER1$/check_dummy $ARG1$
   }

Next, we will create a service template for the passive checks.

   vi /usr/local/nagios/etc/objects/services.cfg

   define service{
       use                                     generic-service
       name                                    passive_service
       active_checks_enabled                   0
       passive_checks_enabled                  1 # We want only passive checking
       flap_detection_enabled                  0
       register                                0 # This is a template, not a real service
       is_volatile                             0
       check_period                            24x7
       max_check_attempts                      1
       normal_check_interval                   5
       retry_check_interval                    1
       check_freshness                         0
       contact_groups                          admins
       check_command                           check_dummy!0
       notification_interval                   120
       notification_period                     24x7
       notification_options                    w,u,c,r
       stalking_options                        w,c,u
   }

After, we can create actual services to match our service checks being passed by NSCA. Keep in mind that the service_description must match the svc_description received in the nsca packet, in our above example using send_nsca, the svc-description was “TestMessage”. I will continue building a service check using that example:

   vi /usr/local/nagios/etc/objects/services.cfg

   define service{
       use                                     passive_service
       service_description                     TestMessage
       host_name                               localhost
   }

Verify the configuration and restart the Nagios daemon so that it loads the updates in our config file.

   /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
   /etc/init.d/nagios restart

You should now be able to see the service listed on the nagios web interface. Note that the service is PENDING until it receives it’s first result. It has no scheduled updates because it is a passive service. We should be able to send a packet with our message now using send_nsca and have it processed and displayed on the web interface.

Repeat the same steps as our last send:

   /usr/local/nagios/bin/send_nsca localhost -c /usr/local/nagios/etc/send_nsca.cfg < /tmp/tmp

Check log it should show something like (Now logs will not report for service defination not defined)

   tail -f /var/log/messages
   Feb 23 17:41:00 bhdhcp06941 nsca[25293]: SERVICE CHECK -> Host Name: 'localhost', Service Description: 'TestMessage', Return Code: '0', Output: 'This is a test   message.'
   Feb 23 17:41:00 bhdhcp06941 nsca[25293]: Attempting to write to nagios command pipe
   Feb 23 17:41:00 bhdhcp06941 nsca[25293]: End of connection...
   Feb 23 17:41:00 bhdhcp06941 xinetd[25000]: EXIT: nsca status=0 pid=25293 duration=0(sec)
   Feb 23 17:41:00 bhdhcp06941 nagios: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;TestMessage;0;This is a test message.
   Feb 23 17:41:03 bhdhcp06941 nagios: PASSIVE SERVICE CHECK: localhost;TestMessage;0;This is a test message.

Check the Web UI as well now the service should show up in Web UI something like

   TestMessage        OK     02-24-2012 17:14:58     0d 0h 0m 40s     1/1     This is a test message.

Now generate one more test error message via passive check

   vi /tmp/tmp2
   localhost       TestMessage     2       This is a Test Error.

   /usr/local/nagios/bin/send_nsca localhost -c /usr/local/nagios/etc/send_nsca.cfg < /tmp/tmp2

Try sending again and the result should be a red ERROR under status. In addition, this should have triggered the notification check and send an email to the members of your admin contact group.

   Feb 23 17:45:14 bhdhcp06941 nsca[25376]: SERVICE CHECK -> Host Name: 'localhost', Service Description: 'TestMessage', Return Code: '2', Output: 'This is a Test Error.'
   Feb 23 17:45:14 bhdhcp06941 nsca[25376]: Attempting to write to nagios command pipe
   Feb 23 17:45:14 bhdhcp06941 nsca[25376]: End of connection...
   Feb 23 17:45:14 bhdhcp06941 xinetd[25000]: EXIT: nsca status=0 pid=25376 duration=1(sec)
   Feb 23 17:45:14 bhdhcp06941 nagios: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;TestMessage;2;This is a Test Error.
   Feb 23 17:45:23 bhdhcp06941 nagios: PASSIVE SERVICE CHECK: localhost;TestMessage;2;This is a Test Error.
   Feb 23 17:45:23 bhdhcp06941 nagios: SERVICE ALERT: localhost;TestMessage;CRITICAL;HARD;1;This is a Test Error.
   Feb 23 17:45:23 bhdhcp06941 nagios: SERVICE NOTIFICATION: sachinnagiosadmin;localhost;TestMessage;CRITICAL;notify-service-by-email;This is a Test Error.
   Feb 23 17:45:23 bhdhcp06941 nagios: SERVICE NOTIFICATION: nagiosadmin;localhost;TestMessage;CRITICAL;notify-service-by-email;This is a Test Error.

Check the Web UI, now the service should show up in Web UI something like

   TestMessage     CRITICAL     02-24-2012 11:48:53     0d 5h 25m 26s     1/1     CRITICAL: Didn't not got the response from Passive Check (Please Check)

6) Issue with Passive check
Problem with Passive check is that the alert remain in same state and we didn’t would get any alert if the check is running successfully or not. So to overcome this we need to configure Nagios in different way.

   vi /usr/local/nagios/etc/objects/commands.cfg

    define command{
       command_name    check_dummy
       command_line    $USER1$/check_dummy $ARG1$ $ARG2$
    }

   vi /usr/local/nagios/etc/objects/services.cfg

    define service{
       use                                     generic-service
       name                                    passive_service
       active_checks_enabled                   0
       passive_checks_enabled                  1 # We want only passive checking
       flap_detection_enabled                  0
       register                                0 # This is a template, not a real service
       is_volatile                             0
       check_period                            24x7
       max_check_attempts                      1
       normal_check_interval                   5
       retry_check_interval                    1
       check_freshness                         0
       contact_groups                          admins
       check_command                           check_dummy!0!"Initial OK"
       notification_interval                   60
       notification_period                     24x7
       notification_options                    w,u,c,r
       stalking_options                        w,c,u
    }

    define service{
       use                                     passive_service
       service_description                     TestMessage
       host_name                               localhost
       check_freshness                         1
       freshness_threshold                     600 # Time in second it will recheck and if not get result will alert as Critical
       check_command                           check_dummy!2!"Didn't not got the response from Passive Check (Please Check)"
    }

The following option will enable freshness of a service and will execute the command mentioned after every second specified in configuration

   check_freshness                         1
   freshness_threshold                     600 # Time in second it will recheck and if not get result will alert as Critical
   check_command                           check_dummy!2!"Didn't not got the response from Passive Check (Please Check)"

Verify Nagios checks and restart the Nagios service

   /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
   /etc/init.d/nagios restart

Nagios Log message will look like if time is experied (freshness time).

   tail -f /var/log/messages
   Feb 23 18:20:27 bhdhcp06941 nagios: Warning: The results of service 'TestMessage' on host 'localhost' are stale by 0d 0h 0m 28s (threshold=0d 0h 2m 30s).  
   I'm forcing an immediate check of the service.


Related Posts:

Installing Nagios Server (4.0.0) on CentOS/RHEL 6.3
Setup and configure Nagios Client (NRPE) on CentOS/RHEL 6.3
NagiosGraph – Graphs in Nagios on CentOS/RHEL 6.3