Traphandling for Vortex GDT8x14RZ Raidcontroller
================================================

Author: 
------- 

Martin Fuerstenau, OCE Printing Sytems GmbH
martin.fuerstenau@ops.de
	
Prerequisities:
---------------

- Installed snmptrapd
- Installed and configured snmptt
- Installed Nagios with installed eventhandlers 
  (With Suse the eventhandlers are part of the development or the
  source package)

How it works:
-------------

The srcd demon running on the host conatining the ICP Vortex controller will send
a snmptrap in cause of trouble to the monitor host.

On the monitor host the trap ist received by snmptrapd and passed to snmptt.
snmptt will process the trap as defined (see Installation and Appendix A.).

Instead of passing the trapvalues directly via submit_check_result to nagios they are passed
to several wrapper scripts. Do this provides several benefits like testing the Nagios
construct without the need to send traps, filtering of values/characters and several
other things.

There are three types of traps:
- MAJOR, wich will cause a warning in my case (Returncode 1)
- INFO, which will only change the plugin-output but stays green (Returncode 0)
- CRITICAL, which is an actual alarm and causes red (Returncode 2)

The wrapper script will address the host, the service definition , the returncode and the 
payload (i.e. the information) from the trap.

Doing it this way makes it very easy to use it in Nagios. All major definitions are done
in a template (see Appendix B). For every host you need a service definition (see Appendix C)
Please modify it to your need.

Seperate it in 3 services has the benefit that you see what happened. If all types
of alert will be in one service definition can cause that you will not take notice of a potential
problem because every plugin output is replaced with a next. 


Installation:
-------------

1. snmptt configuration:
------------------------

The config file is derived from the MIB deliverd by Vortex. Please include it
in your snmptt.ini like this:

  [TrapFiles]
  # A list of snmptt.conf files (this is NOT the snmptrapd.conf file).  The COMPLETE path
  # and filename.  Ex: '/etc/snmp/snmptt.conf'
  snmptt_conf_files = <<END
  /etc/snmp/snmptt/snmptt.conf.apx
  /etc/snmp/snmptt/snmptt.conf.vortex
  END

The config file itself is located in Appendix a.

2. Nagios
---------

2.1 General
- - - - - - 
Nagios must be configured to accept passive check result (mostly standard).

Verify the all pathes in the shellscript submit_check_result which is
part of the eventhandlers are correct, especially the localtion of the pipe
nagios.cmd.

Attention!
Beware of the access rights. snmptrapd and snmptt are normally running as root.
Therefore all scripts started by snmptt are running as root.

If your Nagios is running by a special user (i.e. user nagios, group nagios) and your apache 
is running as user apache, group apache, the directory containing the pipe nagios.cmd is owned by user
nagios group apache for example. In this case a trap will not be processed.

2.2 Plugins
- - - - - -
Copy the wrapper scripts to a appropriate directory. It is unwise to use the same directory for official
plugins from the nagios plugin project and for addidional or selfwritten plugins.

2.3 Service definitions
- - - - - - - - - - - -

Add the service definitions to the appropriate directory/file and modify them to your needs.


3. Handling
-----------

the only thing to do is to reset traps via "submit passive check result"


Appendix A - snmptt.conf.vortex:
--------------------------------

MIB: ICP-CONTROLLER-TRAP-MIB (file:/usr/share/snmp/mibs/srctrap.mib) converted on Fri Nov 24 18:02:26 2006 using snmpttconvertmib v1.1
#
#
#
EVENT evInformationEvent .1.3.6.1.4.1.1483.1.2.0.100 "Status Events" INFORMATIONAL
EXEC /usr/lib/nagios/ops_plugins/vortex_info_trap_wrapper $r "$4"
FORMAT $4
SDESC
An event with informational character.
Variables:
  1: trpEvtCtrlNo
     Syntax="INTEGER"
     Descr="The Controller number to which the event belongs to."
  2: trpEvtID
     Syntax="INTEGER"
     Descr="The internal event ID."
  3: trpEvtTmStamp
     Syntax="INTEGER"
     Descr="Time stamp of event. It is the number of seconds since
                         midnight January 1, 1970 GMT"
  4: trpEvtDescription
     Syntax="OCTETSTR"
     Descr="Description of event."
EDESC
#
#
#
EVENT evWarningEvent .1.3.6.1.4.1.1483.1.2.0.101 "Status Events" MAJOR
FORMAT $4
EXEC /usr/lib/nagios/ops_plugins/vortex_warning_trap_wrapper $r "$4"
SDESC
An event with warning character.
Nothing critical has happened, but the concerning
disk array controller and drive should be examined.
Variables:
  1: trpEvtCtrlNo
     Syntax="INTEGER"
     Descr="The Controller number to which the event belongs to."
  2: trpEvtID
     Syntax="INTEGER"
     Descr="The internal event ID."
  3: trpEvtTmStamp
     Syntax="INTEGER"
     Descr="Time stamp of event. It is the number of seconds since
                         midnight January 1, 1970 GMT"
  4: trpEvtDescription
     Syntax="OCTETSTR"
     Descr="Description of event."
EDESC
#
#
#
EVENT evCriticalEvent .1.3.6.1.4.1.1483.1.2.0.102 "Status Events" CRITICAL
FORMAT $4
EXEC /usr/lib/nagios/ops_plugins/vortex_critical_trap_wrapper $r "$4"
SDESC
An event with critical character. Immediate reaction
is absolutely necessary.
Variables:
  1: trpEvtCtrlNo
     Syntax="INTEGER"
     Descr="The Controller number to which the event belongs to."
  2: trpEvtID
     Syntax="INTEGER"
     Descr="The internal event ID."
  3: trpEvtTmStamp
     Syntax="INTEGER"
     Descr="Time stamp of event. It is the number of seconds since
                         midnight January 1, 1970 GMT"
  4: trpEvtDescription
     Syntax="OCTETSTR"
     Descr="Description of event."
EDESC


Appendix B - the trap template:
-------------------------------

define service{
	name				VORTEX-TRAP
	register			0
	passive_checks_enabled		1
	active_checks_enabled		0
	notifications_enabled		1
        is_volatile			1
        check_period			none
	check_freshness			1
        max_check_attempts		1
        normal_check_interval		1
	flap_detection_enabled		0
        retry_check_interval		1
        contact_groups			linux-admins,linux-admins_sms
        notification_interval		31536000
        notification_period		24x7
        notification_options		c,w,r
	check_command                   check-host-alive
        }

define service{
	name				VORTEX-RAID-TRAP-WARNING
	use 				VORTEX-TRAP
	register			0
        service_description		VORTEX-RAID-WARNING
        notification_options		w,r
        }

define service{
	name				VORTEX-RAID-TRAP-INFO
	use 				VORTEX-TRAP
	register			0
        service_description		VORTEX-RAID-INFO
        notification_options		n
        }

define service{
	name				VORTEX-RAID-TRAP-CRITICAL
	use 				VORTEX-TRAP
	register			0
        service_description		VORTEX-RAID-CRITICAL
        notification_options		c,r
        }


Appendix C - the service definition:
------------------------------------

define service{
	use				VORTEX-RAID-TRAP-WARNING
        host_name			linux1
        }

define service{
	use				VORTEX-RAID-TRAP-INFO
        host_name			linux1
        }

define service{
	use				VORTEX-RAID-TRAP-CRITICAL
        host_name			linux1
        }