SmartOS 的监控

简单好用(难用),不过缺少一个轻量级的图形化管理界面,尤其是监控界面,作为一个奇葩的存在,并且我又入了这个坑,有必要好好监控一下,不然服务器运行情况简直一抹黑, 监控的资料很少, 基于 Solaris,常规的监控方案都用不上,Beats 安装不了,只能再想想办法。

先看看 pkgin 源里面有没有现成的监控工具吧。

[root@INFINITBYTE ~]# pkgin search monitor
top-3.6.1nb1         The ubiquitous system monitoring utility
tcp_wrappers-7.6.1nb4  Monitor and filter incoming requests for network services
postgresql91-monitoring-9.1.9  PostgreSQL monitoring tools
postgresql91-monitoring-9.1.4  PostgreSQL monitoring tools
nrsysmond-1.3.0      New Relic Server Monitor
nagios-base-3.3.1nb2  Network monitor
mytop-1.6nb3         Console-based tool for monitoring the threads and performance of MySQL
-node-2.0.7nb1 = System monitoring tool, client version
-node-2.0.0 >   System monitoring tool, client version
-master-2.0.7nb1  System monitoring tool, master version
-master-1.4.5   System monitoring tool, master version
mtop-0.6.6           Curses-based MySQL server monitor
innotop-1.7.2        MySQL and InnoDB monitor program
ganglia-monitor-core-3.1.2nb12  Ganglia cluster monitor, monitoring daemon

=: package is installed and up-to-date
<: package is installed but newer version is available
>: installed package has a greater version than available package

居然有 New Relic 的 nrsysmond,果然够全。不过我没有账号,注册试用,居然提示我验证码需要更新浏览器,立马弃坑。

munin 比较轻量级,就它了。

首先,需要更新 pkg_admin,每次重启都需要,否则 pkgin 无法正常工作。

pkg_admin rebuild
pkgin -y up

然后下载安装

pkgin search munin
pkgin install munin-node

提示错误:

The following users need to be created for munin-node-2.0.7nb1:

    munin: munin, /nonexistent, /usr/bin/false

原来需要提前创建用户,先卸载 /etc/shadow 才能创建用户:

umount /etc/shadow; cp /usbkey/shadow /etc/shadow; 

创建用户,修改用户属性:

useradd munin; 
cat /etc/passwd 
munin:x:100:1::/nonexistent:/usr/bin/false

持久化到配置文件:

cp /etc/shadow /usbkey/shadow

现在可以安装了。

[root@INFINITBYTE /opt/local]# pkgin install munin-node

calculating dependencies... done.



nothing to upgrade.

1 packages to be installed: munin-node-2.0.7nb1 (1128K to download, 0B to install)



proceed ? [Y/n] y

downloading packages...

munin-node-2.0.7nb1.tgz                                        100%  271KB 135.3KB/s  30.5KB/s   00:02    

installing packages...

installing munin-node-2.0.7nb1...

munin-node-2.0.7nb1: copying /opt/local/share/examples/munin/munin-node.conf to /opt/local/etc/munin/munin-node.conf

===========================================================================

The following files should be created for munin-node-2.0.7nb1:



    /etc/rc.d/munin-node (m=0755)

        [/opt/local/share/examples/rc.d/munin-node]



===========================================================================

//////////////////////////////////////////////////////////////////////////////



This package is SMF enabled, which means you can use SMF to 'enable',

'disable' or 'restart' the persistent daemon process, e.g.:



  svcadm enable munin:default



The SMF manifest was automatically imported now.



See our wiki on what's SMF and how to use it to your advantage:



  http://wiki.joyent.com/display/smart/About+the+Service+Management+Facility



//////////////////////////////////////////////////////////////////////////////

===========================================================================

$NetBSD: MESSAGE,v 1.3 2006/12/05 14:53:32 xtraeme Exp $



Plugins can be auto-configured with "munin-node-configure --shell | sh"



You may also want to add an entry such as



/var/log/munin/munin-node.log   munin:munin 644 7 *    24   Z



to /etc/newsyslog.conf to have the node log rotated.



===========================================================================

pkg_install warnings: 0, errors: 0

reading local summary...

processing local summary...

updating database: 100%

marking munin-node-2.0.7nb1 as non auto-removable

安装好了之后,可以增加一些 SmartOS 的监控插件。

cd /opt/local/lib/munin/plugins
curl --insecure -O https://raw.github.com/munin-monitoring/contrib/master/plugins/zfs/zfs_usage_
curl --insecure -O https://raw.githubusercontent.com/munin-monitoring/contrib/0.2013.02.12/plugins/system/solaris-memstat
curl --insecure -O https://raw.githubusercontent.com/munin-monitoring/contrib/0.2013.02.12/plugins/system/zones_cpu
curl --insecure -O https://raw.githubusercontent.com/munin-monitoring/contrib/0.2013.02.12/plugins/system/zones_mem

chmod +x /opt/local/lib/munin/plugins/zfs_usage_ 
chmod +x /opt/local/lib/munin/plugins/solaris-memstat
chmod +x /opt/local/lib/munin/plugins/zones_cpu
chmod +x /opt/local/lib/munin/plugins/zones_mem

cd /opt/local/etc/munin/plugins
ln -s /opt/local/lib/munin/plugins/zones_cpu zones_cpu
ln -s /opt/local/lib/munin/plugins/zones_mem zones_mem
ln -s /opt/local/lib/munin/plugins/zfs_usage_ zfs_usage_zones
ln -s /opt/local/lib/munin/plugins/solaris-memstat solaris-memstat

然后,设置 munin 节点名称和允许服务端连接,SmartOS 是28,服务端 ip 是 213。

vi /opt/local/etc/munin/munin-node.conf

host_name infinitbyte
allow ^192\.168\.3\.213$
host *
 host 192.168.3.28

下一步,安装服务,服务配置文件:

mkdir -p /opt/custom/smf
cat /opt/custom/smf/munin-node.xml

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
  <service name='network/munin-node' type='service' version='0'>
    <create_default_instance enabled='true'/>
    <single_instance/>
    <exec_method name='start' type='method' exec='/opt/local/sbin/munin-node' timeout_seconds='0'>
      <method_context/>
    </exec_method>
    <exec_method name='stop' type='method' exec=':kill' timeout_seconds='0'>
      <method_context/>
    </exec_method>
  </service>
</service_bundle>

导入服务

svccfg -v import /opt/custom/smf/munin-node.xml
svcadm disable network/munin-node
svcadm enable network/munin-node
svcadm restart network/munin-node
netstat -ano|grep 4949

安装服务端,Centos 服务器,SmartOS 的一个 zone。

 yum install munin
/etc/init.d/munin-node restart
chkconfig munin-node on
netstat -ano|grep 4949

修改配置,添加远程节点

vi /etc/munin/munin.conf 

[infinitbyte]

    address 192.168.3.28

    use_node_name yes

添加 nginx 配置,方便访问。

vi /etc/nginx/conf.d/munin.conf 

server {

    listen       80;

    server_name  munin.example.org;

    location / {

        root   /var/www/html/munin/;

        index  index.html index.htm;

    }

}

现在通过munin.example.org就能访问 SmartOS 的监控信息了。

Snip20180224_2

Snip20180224_4

不过,如果能够实现一个 Metricbeat 插件,来收集 munin 的数据,存放到 es 里面就更好了。

相关链接:

https://github.com/munin-monitoring/contrib/tree/master/plugins

How to add Remote Linux host on Munin server to monitor

https://qiita.com/timwata/items/41bb56ea2325c289c392

https://serverfault.com/questions/461817/how-to-run-munin-node-on-smartos-global-zone

https://www.claudiokuenzler.com/blog/434/get-real-memory-usage-statistics-physical-smartos-global#.WpCxOpNuafR