munin で dell open manage

次は dell で Open Manage 経由で munin でグラフを作ってみる。

OpenManager

dellftpにあがっているので適当にダウンロードして解凍する。

mkdir tmp
cd tmp
wget http://ftp.dell.com/sysman/OM_6.2.0_ManNode_A00.tar.gz
tar zxvf OM_6.2.0_ManNode_A00.tar.gz


インストーラーを動かすと動作しないので・・・

sh setup.sh

怒られる。

     Unrecognized Operating System. This script cannot continue with
     the installation. Select rpms from the OS folder in the media that
     closely matches this Operating System to continue with the manual
     install.

ちょっと改造する。

vim /etc/redhat-release
-------------------------------------------------------------
CentOS release 5.6 (Final)
↓
CentOS release 5.6 (Final Tikanga)
-------------------------------------------------------------

RHEL5 のリリース名称が Tikanga で、OpenManage インストーラーはこの文字列でOSを判定しているらしい。
困ったもんだ。



もう一度実行。
>||
sh setup.sh

                  DELL SOFTWARE LICENSE AGREEMENT


以下略

何入れるの〜って聞かれるので、とりあえず全部 All ぢゃ〜 って答える

##############################################

  Server Administrator Custom Install Utility

##############################################

  Components for Server Administrator Managed Node Software:

    [ ] 1. Server Administrator Web Server
    [ ] 2. Server Instrumentation
    [ ] 3. Storage Management
    [ ] 4. Remote Enablement
    [ ] 5. Remote Access Core Components
    [ ] 6. Remote Access SA Plugin Components
    [ ] 7. All

  Enter the number to select a component from the above list.
  Enter q to quit.

  Enter : 7


これでええのんか?と聞かれるので、 i でインストール。

##############################################

  Server Administrator Custom Install Utility

##############################################

  Selected Options
   - All

  Dependencies
   - Server Administrator Web Server
   - Server Instrumentation
   - Storage Management
   - Remote Enablement
   - Remote Access Core Components
   - Remote Access SA Plugin Components

  Components for Server Administrator Managed Node Software:

  Installed Components:
    [x] Server Instrumentation

  Add Components
    [x] 1. Server Administrator Web Server
    [x] 2. Server Instrumentation
    [x] 3. Storage Management
    [x] 4. Remote Enablement
    [x] 5. Remote Access Core Components
    [x] 6. Remote Access SA Plugin Components
    [x] 7. All

  Enter the number to select a component from the above list.
  Enter c to copy selected components to destination folder.
  Enter i to install the selected components.
  Enter r to reset selection and start over.
  Enter q to quit.

  Enter : i

すると、壮大にずっこける。
だいたいこんな感じ。

警告: srvadmin-fsa-6.2.0-1.6.el3.i386.rpm: ヘッダ V3 DSA signature: NOKEY, key ID 23b66a9d
エラー: 依存性の欠如:
        libcmpiCppImpl.so.0 は srvadmin-itunnelprovider-6.2.0-1.6.el5.i386 に必要とされて います
        libcmpiCppImpl0 >= 2.0.0 は srvadmin-itunnelprovider-6.2.0-1.6.el5.i386 に必要とされています
        openwsman-server >= 2.1.5 は srvadmin-itunnelprovider-6.2.0-1.6.el5.i386 に必要と されています
        sblim-sfcb >= 1.3.2 は srvadmin-itunnelprovider-6.2.0-1.6.el5.i386 に必要とされて います
        sblim-sfcc >= 2.1.0 は srvadmin-itunnelprovider-6.2.0-1.6.el5.i386 に必要とされて います

警告: srvadmin-cm-6.2.0-677.i386.rpm: ヘッダ V3 DSA signature: NOKEY, key ID 23b66a9d
エラー: 依存性の欠如:
        libstdc++.so.5 は srvadmin-rac5-components-6.2.0-677.i386 に必要とされています
        libxslt.so.1 は srvadmin-xmlsup-6.2.0-1.17.el5.i386 に必要とされています

 -   Installation of some of the packages have failed.
 -   Please refer to 'readme_ins.txt' file for more details.
 -   The readme_ins.txt can be found in srvadmin/docs/readme folder


依存パッケージをちくちく入れていく。
ネットから落とせるものはネットから入れる。その他のよくわかんないのは、dell OpenManage Installer についてくるのになぜか勝手に入れてくれない付属パッケージを入れておく。

yum -y install compat-libstdc++-33 libxslt curl
rpm -Uvh linux/RPMS/supportRPMS/opensource-components/RHEL5/*Dell*.rpm


もう一回。

sh setup.sh


すると、今回はインストールがちゃんと行われる。
やったー。

   1:srvadmin-jre           ########################################### [  7%]
   2:srvadmin-smweb         ########################################### [ 14%]
   3:srvadmin-racdrsc5      ########################################### [ 21%]
   4:srvadmin-racadm5       ########################################### [ 29%]
   5:srvadmin-fsa           ########################################### [ 36%]
   6:srvadmin-sysfsutils    ########################################### [ 43%]
   7:srvadmin-storelib      ########################################### [ 50%]
   8:srvadmin-storage       ########################################### [ 57%]
   9:srvadmin-megalib       ########################################### [ 64%]
  10:srvadmin-itunnelprovide########################################### [ 71%]
  11:srvadmin-iws           ########################################### [ 79%]
  12:srvadmin-storage-popula########################################### [ 86%]
  13:srvadmin-storelib-libpc########################################### [ 93%]
  14:srvadmin-storelib-sysfs########################################### [100%]

最後に、デーモン実行してもいいんか?みたいなことを聞かれるから y で実行する。

   Do you want the Server Administrator services started?

   Note: If this is an upgrade, services that were running prior
   to upgrade are already started. Enter 'y' to start the services
   that are in 'Stop' state.

   Press ('y' for yes | 'Enter' to exit): y
Starting Systems Management Device Drivers:
Starting dell_rbu:                                         [  OK  ]
Starting ipmi driver: Already started                      [  OK  ]
Starting Systems Management Data Engine:
Starting dsm_sa_datamgrd:                                  [  OK  ]
Starting dsm_sa_eventmgrd:                                 [  OK  ]
Starting DSM SA Shared Services:                           [  OK  ]

Starting DSM SA Connection Service:                        [  OK  ]

できた。
これで、 omreport コマンドが使えるはずだ!!

使ってみよう。。。あれ?

[root@hogehofe tmp]# omreport

-bash: omreport: command not found


なんか、知らないけど、 /opt/dell/srvadmin/bin/omreport というところにインストールされていた。
これではめんどいから、適当にシンボリックリンクでも貼っとく。

ln -s /opt/dell/srvadmin/bin/omreport /usr/bin/omreport

今度は動くよ!

[root@hogehofe tmp]# omreport

omreport         Reports component properties.

Usage: omreport <command...>  [name=value...] ... [option...]

以下略

OpenManageで遊ぼう

状態一覧 omreport chassis

omreport chassis

------------------------------------------------------------
Health

Main System Chassis

SEVERITY : COMPONENT
Ok       : Fans
Ok       : Intrusion
Ok       : Memory
Ok       : Power Supplies
Ok       : Processors
Ok       : Temperatures
Ok       : Voltages
Ok       : Hardware Log
Ok       : Batteries

For further help, type the command followed by -?
------------------------------------------------------------


個別にもっと詳細が見れる。


パーワーサプライ(電源) omreport chassis pwrsupplies

omreport chassis pwrsupplies

------------------------------------------------------
Power Supplies Information

---------------------------------------
Main System Chassis Power Supplies : Ok
---------------------------------------

Power Supply Redundancy : Ok

Attribute : Redundancy Status
Value     : Full


Individual Power Supply Elements
Index                    : 0
Status                   : Ok
Location                 : PS 1 Status
Type                     : AC
Rated Input Wattage      : [No Value]
Maximum Output Wattage   : 670 W
Online Status            : Presence Detected
Power Monitoring Capable : No

Index                    : 1
Status                   : Ok
Location                 : PS 2 Status
Type                     : AC
Rated Input Wattage      : [No Value]
Maximum Output Wattage   : 670 W
Online Status            : Presence Detected
Power Monitoring Capable : No
------------------------------------------------------


ファン omreport chassis fans

omreport chassis fans

-----------------------------------------------------
Fan Probes Information

----------------------------
Main System Chassis Fans: Ok
----------------------------

Fan Redundancy : Ok

Attribute : Redundancy Status
Value     : Full

Probe List
Index                     : 0
Status                    : Ok
Probe Name                : System Board FAN MOD 1A RPM
Reading                   : 6675 RPM
Minimum Warning Threshold : [N/A]
Maximum Warning Threshold : [N/A]
Minimum Failure Threshold : 1575 RPM
Maximum Failure Threshold : [N/A]

Index                     : 1
Status                    : Ok
Probe Name                : System Board FAN MOD 1B RPM
Reading                   : 6525 RPM
Minimum Warning Threshold : [N/A]
Maximum Warning Threshold : [N/A]
Minimum Failure Threshold : 1575 RPM
Maximum Failure Threshold : [N/A]

以下略


温度 omreport chassis temps

omreport chassis temps

-----------------------------------------------------
Temperature Probes Information

------------------------------------
Main System Chassis Temperatures: Ok
------------------------------------

Index                     : 0
Status                    : Ok
Probe Name                : System Board Ambient Temp
Reading                   : 24.0 C
Minimum Warning Threshold : 8.0 C
Maximum Warning Threshold : 42.0 C
Minimum Failure Threshold : 3.0 C
Maximum Failure Threshold : 47.0 C
-----------------------------------------------------

次はRAIDの監視。
omreport storage controller

omreport storage controller

-----------------------------------------------------
Controllers
ID                                            : 0
Status                                        : Ok
Name                                          : PERC 6/i Integrated
Slot ID                                       : Embedded
State                                         : Ready
Firmware Version                              : 6.3.0-0001
Minimum Required Firmware Version             : Not Applicable
Driver Version                                : 00.00.04.31-RH1

以下略
-----------------------------------------------------


ファームウェアのバージョンが低いと、Status が Non-Critical になるらしい。
面倒だから、別にいいんじゃね・・・って思ってしまうんだけど、上げたい人は上げればいいんじゃないの?
知らないけど。

-----------------------------------------------------
Status                                        : Non-Critical
Name                                          : PERC 6/i Integrated
Slot ID                                       : Embedded
State                                         : Degraded
Firmware Version                              : 6.1.1-0047
Minimum Required Firmware Version             : 6.2.0-0012
-----------------------------------------------------

仮想ディスク omreport storage vdisk controller=0
RAIDコントローラ上に作った RAIDを提供するディスクさんですね。

omreport storage vdisk controller=0

-----------------------------------------------------
ID                       : 0
Status                   : Ok
Name                     : Virtual Disk 0
State                    : Ready
HotSpare Policy violated : Not Applicable

以下略
-----------------------------------------------------


物理ディスク omreport storage pdisk controller=0
仮想ディスクを構築するために使った物理的な実ディスクさんですね。

omreport storage pdisk controller=0

-----------------------------------------------------
ID                        : 0:0:0
Status                    : Ok
Name                      : Physical Disk 0:0:0
State                     : Online
以下略
-----------------------------------------------------

muninで監視する。

ここまでできたら、こいつらを munin で監視してみましょう!!
ただ、 munin exchange にあるプラグインはちょっと1つを除き微妙だったので、 hppower などの HP監視 plugin を改造して作ってみました。
何が微妙なのかというと、ファンの回転数をグラフにするプラグインはあるんだけど、私たちが知りたいのは、ファンが動いているか、それとも壊れたかどうかなわけで、、、回転数を教えてもらっても、あんまり嬉しくないんですね。


電源 dell_omreport_power

#!/usr/bin/env perl
#
# dell_omreport_power - Munin plugin for Dell server power supplies
#
# Copyright (C) 2011 rti (hiroyuki fujie)
#
# Orignal Source Code.
# hppower
#       Copyright (C) 2010 Magnus Hagander
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

=head1 NAME

dell_omreport_power - Munin plugin to monitor Dell server fans

=head1 CONFIGURATION

=head1 INTERPRETATION

The graph will show information about how many power supplies are ok and howmany are degratded.
=head1 INTERPRETATION

The graph will show information about how many power supplies are ok and how
many are degratded.

=head1 MAGIC MARKERS

 #%# family=contrib

=head1 AUTHOR

Magnus Hagander <magnus@hagander.net>
Hiroyuki Fujie <super.rti@gmail.com>

=head1 LICENSE

GPLv2

=cut

use strict;
use warnings;

#use Munin::Plugin;

if ( $ARGV[0] and $ARGV[0] eq "autoconf") {
   print "no\n";
   exit(0);
}

if ( $ARGV[0] and $ARGV[0] eq "config") {
   print "graph_title Dell Power Supplies\n";
   print "graph_category system\n";
   print "ok.label Power Supplies OK\n";
   print "ok.type GAUGE\n";
   print "ok.min 0\n";
   print "degraded.label Power Supplies degraded\n";
   print "degraded.type GAUGE\n";
   print "degraded.min 0\n";
   print "degraded.critical 1\n";
   exit(0);
}

open(DELLOPMANAGE, "omreport chassis pwrsupplies |") || die "Could not run omreport\n";
my $ok=0;
my $degraded=0;

while (<DELLOPMANAGE>) {
   next if /^\s*$/;
   next if ! /^\s*Status/;
   if (/\s+(Ok)\s+/) {
      $ok++;
   }
   elsif (/\s+Absent\s+/) {
      # unknown, we don't track that
   }
   else {
      $degraded++;
   }
}
close(DELLOPMANAGE);

print "ok.value $ok\n";
print "degraded.value $degraded\n";

'



ファン dell_omreport_fan

#!/usr/bin/env perl
#
# dell_omreport_fan - Munin plugin for Dell server power supplies
#
# Copyright (C) 2011 rti (hiroyuki fujie)
#
# Orignal Source Code.
# hpfan
#       Copyright (C) 2010 Magnus Hagander
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

=head1 NAME

dell_omreport_fan - Munin plugin to monitor Dell server fans

=head1 CONFIGURATION


=head1 INTERPRETATION

The graph will show information about how many fan supplies are ok and how
many are degratded.

=head1 MAGIC MARKERS

 #%# family=contrib

=head1 AUTHOR

Magnus Hagander <magnus@hagander.net>
Hiroyuki Fujie <super.rti@gmail.com>

=head1 LICENSE

GPLv2

=cut

use strict;
use warnings;

#use Munin::Plugin;

if ( $ARGV[0] and $ARGV[0] eq "autoconf") {
   print "no\n";
   exit(0);
}

if ( $ARGV[0] and $ARGV[0] eq "config") {
   print "graph_title Dell Fans\n";
   print "graph_category system\n";
   print "ok.label Fans OK\n";
   print "ok.type GAUGE\n";
   print "ok.min 0\n";
   print "degraded.label Fan degraded\n";
   print "degraded.type GAUGE\n";
   print "degraded.min 0\n";
   print "degraded.critical 1\n";
   exit(0);
}

open(DELLOPMANAGE, "omreport chassis fans |") || die "Could not run omreport\n";
my $ok=0;
my $degraded=0;

while (<DELLOPMANAGE>) {
   next if /^\s*$/;
   next if ! /^\s*Status/;
   if (/\s+(Ok)\s+/) {
      $ok++;
   }
   elsif (/\s+Absent\s+/) {
      # unknown, we don't track that
   }
   else {
      $degraded++;
   }
}
close(DELLOPMANAGE);

print "ok.value $ok\n";
print "degraded.value $degraded\n";

'


RAID dell_omreport_acu

#!/usr/bin/env perl
#
# acu - Munin plugin for Dell Raid Array Controllers
#
# Copyright (C) 2011 Hiroyuki Fujie(rti)
# Copyright (C) 2010 Magnus Hagander

#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

=head1 NAME

dell_omreport_acu - Munin plugin to monitor Dell Raid Array Controllers

=head1 CONFIGURATION

=head1 INTERPRETATION

The graph will show information about number of controllers,
logical drives and physical drives that are ok or degraded.

=head1 MAGIC MARKERS

 #%# family=contrib

=head1 AUTHOR

Magnus Hagander <magnus@hagander.net>
Hiroyuki Fujie <super.rti@mail.com> 

=head1 LICENSE

GPLv2

=cut

use strict;
use warnings;

#use Munin::Plugin;

if ( $ARGV[0] and $ARGV[0] eq "autoconf") {
   print "no\n";
   exit(0);
}

if ( $ARGV[0] and $ARGV[0] eq "config") {
   print "graph_title Dell Hardware Raid\n";
   print "graph_category disk\n";
   print "dell_omreport_acu_logicaldrives_ok.label Logical Drives OK\n";
   print "dell_omreport_acu_logicaldrives_ok.type GAUGE\n";
   print "dell_omreport_acu_logicaldrives_ok.min 0\n";
   print "dell_omreport_acu_logicaldrives_deg.label Logical Drives degraded\n";
   print "dell_omreport_acu_logicaldrives_deg.type GAUGE\n";
   print "dell_omreport_acu_logicaldrives_deg.min 0\n";
   print "dell_omreport_acu_logicaldrives_deg.critical 1\n";
   print "dell_omreport_acu_physicaldrives_ok.label Physical Drives OK\n";
   print "dell_omreport_acu_physicaldrives_ok.type GAUGE\n";
   print "dell_omreport_acu_physicaldrives_ok.min 0\n";
   print "dell_omreport_acu_physicaldrives_deg.label Physical Drives degraded\n";
   print "dell_omreport_acu_physicaldrives_deg.type GAUGE\n";
   print "dell_omreport_acu_physicaldrives_deg.min 0\n";
   print "dell_omreport_acu_physicaldrives_deg.critical 1\n";
   exit(0);
}

my @slots;
open(DELLOPMANAGE, "omreport storage controller |") || die "Could not run omreport\n";
while (<DELLOPMANAGE>) {
   if (/ID *\: *(\d+)/) {
      push @slots, $1;
   }
}
close(DELLOPMANAGE);

my $logicaldrives_ok = 0;
my $logicaldrives_degrad = 0;
my $physicaldrives_ok = 0;
my $physicaldrives_degrad = 0;
my $logicaldrives_msg = '';
my $physicaldrives_msg = '';

foreach my $slot (@slots) {
   # Check logical drives
   open(DELLOPMANAGE, "omreport storage vdisk controller=$slot |") || die "Could not run omreport\n";
   while (<DELLOPMANAGE>) {
      if (/Status *\: *(.*)/) {
         if ($1 eq "Ok") {
            $logicaldrives_ok++;
         } else {
            $logicaldrives_degrad++;
            $logicaldrives_msg .= $_;
         }
      }
   }

   # Finally, check physical drives
   open(DELLOPMANAGE, "omreport storage pdisk controller=$slot |") || die "Could not run omreport\n";
   while (<DELLOPMANAGE>) {
      if (/Status *\: *(.*)/) {
         if ($1 eq "Ok") {
            $physicaldrives_ok++;
         } else {
            $physicaldrives_degrad++;
            $physicaldrives_msg .= $_;
         }
      }
    }
   close(DELLOPMANAGE);
}

$logicaldrives_msg =~ s/\n/ :: /g;
$physicaldrives_msg =~ s/\n/ :: /g;

print "dell_omreport_acu_logicaldrives_ok.value $logicaldrives_ok\n";
print "dell_omreport_acu_logicaldrives_deg.value $logicaldrives_degrad\n";
print "dell_omreport_acu_logicaldrives_deg.extinfo $logicaldrives_msg\n" if ($logicaldrives_msg);
print "dell_omreport_acu_physicaldrives_ok.value $physicaldrives_ok\n";
print "dell_omreport_acu_physicaldrives_deg.value $physicaldrives_degrad\n";
print "dell_omreport_acu_physicaldrives_deg.extinfo $physicaldrives_msg\n" if ($physicaldrives_msg);

遊び方

cd /usr/share/munin/plugins
wget "http://rtilabs.net/files/2011_08_15/dell_omreport_acu"
wget "http://rtilabs.net/files/2011_08_15/dell_omreport_fan"
wget "http://rtilabs.net/files/2011_08_15/dell_omreport_power"

chmod +x dell_omreport_*

ln -s /usr/share/munin/plugins/dell_omreport_acu /etc/munin/plugins/dell_omreport_acu 
ln -s /usr/share/munin/plugins/dell_omreport_fan /etc/munin/plugins/dell_omreport_fan 
ln -s /usr/share/munin/plugins/dell_omreport_power /etc/munin/plugins/dell_omreport_power

/etc/init.d/munin-node restart