集群監視軟體Ganglia

集群監視軟體Ganglia

什麼是ganglia
Ganglia監控軟體主要是用來監控系統性能的軟體，如：cpu 、mem、硬碟利用率， I/O負載、網路流量情況等，通過曲線很容易見到每個節點的工作狀態，對合理調整、分配系統資源，提高系統整體性能起到重要作用。

Ganglia的組成
ganglia 是分散式的監控系統，有兩個Daemon, 分別是：客戶端Ganglia Monitoring Daemon (gmond)和服務端Ganglia Meta Daemon (gmetad)

什麼是PHP Web Frontend
Php Web Frontend 是Ganglia的一套基於php開發和運行的web統計瀏覽程序

什麼是PHP
PHP是一個基於服務端來創建動態網站的腳本語言，您可以用PHP和HTML生成網站主頁。當一個訪問者打開主頁時，服務端便執行PHP的命令並將執行結果發送至訪問者的瀏覽器中，這類似於ASP和CoildFusion，然而PHP和他們不同之處在於PHP開放源碼和跨越平台，PHP可以運行在WINDOWS NT和多種版本的UNIX上。它不需要任何預先處理而快速反饋結果，它也不需要mod_perl的調整來使您的伺服器的內存映象減小。PHP消耗的資源較少，當PHP作為Apache Web伺服器一部分時，運行代碼不需要調用外部二進位程序，伺服器不需要承擔任何額外的負擔。

什麼是RRDtool
RRDtool是系統存放和顯示time-series (即網路帶寬、溫度、人數、伺服器負載等) ,並且它額可以繪出有用的圖表用來顯示處理的數據和數據密度。

相關資源

http://www.ganglia.info
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It relies on a multicast-based listen/announce protocol to monitor state within clusters and uses a tree of point-to-point connections amongst representative cluster nodes to federate clusters and aggregate their state. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on over 500 clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
The ganglia system is comprised of two unique daemons, a PHP-based web frontend and a few other small utility programs.
Ganglia Monitoring Daemon (gmond)

Gmond is a multi-threaded daemon which runs on each cluster node you want to monitor. Installation is easy. You don't have to have a common NFS filesystem or a database backend, install special accounts, maintain configuration files or other annoying hassles.
Gmond has four main responsibilities: monitor changes in host state, announce relevant changes, listen to the state of all other ganglia nodes via a unicast or multicast channel and answer requests for an XML description of the cluster state.

Each gmond transmits in information in two different ways: unicasting/multicasting host state in external data representation (XDR) format using UDP messages or sending XML over a TCP connection.

Ganglia Meta Daemon (gmetad)

Federation in Ganglia is achieved using a tree of point-to-point connections amongst representative cluster nodes to aggregate the state of multiple clusters. At each node in the tree, a Ganglia Meta Daemon (gmetad) periodically polls a collection of child data sources, parses the collected XML, saves all numeric, volatile metrics to round-robin databases and exports the aggregated XML over a TCP sockets to clients. Data sources may be either gmond daemons, representing specific clusters, or other gmetad daemons, representing sets of clusters. Data sources use source IP addresses for access control and can be specified using multiple IP addresses for failover. The latter capability is natural for aggregating data from clusters since each gmond daemon contains the entire state of its cluster.

Ganglia PHP Web Frontend

The Ganglia web frontend provides a view of the gathered information via real-time dynamic web pages. Most importantly, it displays Ganglia data in a meaningful way for system administrators and computer users. Although the web frontend to ganglia started as a simple HTML view of the XML tree, it has evolved into a system that keeps a colorful history of all collected data.
The Ganglia web frontend caters to system administrators and users. For example, one can view the CPU utilization over the past hour, day, week, month, or year. The web frontend shows similar graphs for Memory usage, disk usage, network statistics, number of running processes, and all other Ganglia metrics.

The web frontend depends on the existence of the gmetad which provides it with data from several Ganglia sources. Specifically, the web frontend will open the local port 8651 (by default) and expects to receive a Ganglia XML tree. The web pages themselves are highly dynamic; any change to the Ganglia data appears immediately on the site. This behavior leads to a very responsive site, but requires that the full XML tree be parsed on every page access. Therefore, the Ganglia web frontend should run on a fairly powerful, dedicated machine if it presents a large amount of data.

The Ganglia web frontend is written in the PHP scripting language, and uses graphs generated by gmetad to display history information. It has been tested on many flavours of Unix (primarily Linux) with the Apache webserver and the PHP 4.1 module.


http://www.php.net
PHP is a widely-used general-purpose scripting language that is especially suited for Web development and can be embedded into HTML.

http://www.rrdtool.org

RRD is the Acronym for Round Robin Database. RRD is a system to store and display time-series data (i.e. network bandwidth, machine-room temperature, server load average). It stores the data in a very compact way that will not expand over time, and it can create beautiful graphs. It can be used via simple shell scripts or as a perl module.

具體安裝步驟

安裝環境 Redhat AS3 update4 with Apache
安裝軟體 rrdtool-1.0.49  Ganglia3.0.1 PHP4.4.0 Ganglia-web-3.0.0-1

安裝RRDTool

gmetad需要先安裝RRDTool，默認的安裝路徑：/usr/local/rrdtool-1.0.49
Your_prompt>tar rrdtool.tar.gz
Your_prompt>cd rrdtool-1.0.49
Your_prompt>./configure
Your_prompt>make
Your_prompt>make install
更改rrdtool-1.0.49名稱為rrdtool
Your_prompt>mv rrdtool-1.0.49 rrdtool
rrd.h in /usr/local/rrdtool/include/rrd.h
librrd.a in /usr/local/rrdtool/lib/librrd.a

Server端的安裝和配置

gmetad的安裝

gmetad不是默認安裝的，安裝時需要加參數 --with-gmetad 。即rrdtool庫及其頭文件必須存在，默認的路徑是/usr/include/rrd.h和 /usr/lib/librrd.a，如果在安裝rrdtool時安在了不同的路徑下，這裡需要指明它們的路徑。
./configure CFLAGS="-I/rrd/header/path" CPPFLAGS="-I/rrd/header/path" \
LDFLAGS="-L/rrd/library/path" --with-gmetad
Your_prompt>tar –zxvf ganglia-3.0.1.tar.gz
Your_prompt>cd ganglia-3.0.1
Your_prompt>./configure CFLAGS="-I/rrd/header/usr/local/rrdtool/include/rrd.h"
CPPFLAGS="-I/rrd/header/usr/local/rrdtool/include/rrd.h "
LDFLAGS="-L/rrd/library/usr/local/rrdtool/lib/librrd.a " --with-gmetad
Your_prompt>make
Your_prompt>make install

為了保證在開始時啟動，需要將gmetad.init文件拷貝到 /etc/rc.d/init.d/
Your_prompt> cd ganglia-3.0.1/gmetad
Your_prompt> cp gmetad.init /etc/rc.d/init.d/gmetad

將配置文件拷貝到/etc目錄下
Your_prompt> cp gmetad.conf /etc/gmetad.conf
Add GMETAD to the list of programs at startup
Your_prompt> chkconfig --add gmetad
Your_prompt> chkconfig --list gmetad
GMETAD 0:off 1:off 2:on 3:on 4:on 5:on 6:off

啟動gmetad
Your_prompt>/etc/rc.d/init.d/gmetad start
Starting GANGLIA gmetad: [ OK ]
Your_prompt>telnet localhost 8651 | grep 「hostname」
就可以得到監控的各個主機的狀態。

gmetad.conf的配置
# data_source "another source" 1.3.4.7:8655 1.3.4.8
data_source "SERVER" 10 node1 node2
data_source是最重要的參量，在GMOND的Cluser name配置必須與data_source的相同，這個參量被設置為群的名字，被監測以便能監測那群狀態。如果有二個或更多監測對象，當有一對象不能被監測，將讀取data_source 配置的下一個對象

client端安裝和配置

Your_prompt>tar –zxvf ganglia-3.0.1.tar.gz
Your_prompt>cd ganglia-3.0.1
Your_prompt>./configure
Your_prompt>make
Your_prompt>make install
Your_prompt>cd gmond
Your_prompt>gmond –t > /etc/gmond.conf
Your_prompt>cp gmond.init /etc/rc.d/init.d/gmond
Your_prompt> chkconfig --add gmond
Your_prompt> chkconfig --list gmond
gmond 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Your_prompt>/etc/rc.d/init.d/gmond start
Starting GANGLIA gmond: [ OK ]
Your_prompt>telnet localhost 8649 就可以獲取機群內運行gmond的主機的信息

配置gmond.conf

vi /etc/gmond.conf
globals {
setuid = no
user = nobody
cleanup_threshold = 300 /*secs */
}
修改為
setuid = yes
user = scett #本機用戶名

cluster {
name = "unspecified" #Cluser name
}
修改監控組名稱
name = 「SERVER」
配置完成後重新啟動gmond.

安裝php4.40

tar zxvf php-4.4.0.tar.gz
cd php-4.4.0
./configure --prefix=/usr/local/php --with-config-filepath=/usr/local/php --with-apxs2=/usr/local/httpd/bin/apxs --with-openssl=/usr/local/ssl
--with-mysql=/usr/local/mysql
make
make install

修改apache配置文件
Vi /etc/httpd/conf/httpd.conf

LoadModule php4_module    extramodules/libphp4.so
AddModule mod_php4.c
   AddType  application/x-httpd-php         .php .php4 .php3 .phtml
AddType  application/x-httpd-php-source  .phps

安裝 php web frontend

rpm -Uvh ganglia-web-3.0.0-1.i386.rpm

最終通過 http://server/ganglia 訪問頁面如下

http://5iwww.vicp.net/attachment/1165555010_0.jpg

http://5iwww.vicp.net/attachment/1165555010_1.jpg

LZ的伺服器上沒有能反向解析出其他伺服器的名字，所以圖上顯示的都是IP

ganglia還可以分組，是很不錯的可視化集群監控。
當初做運維的時候就想做一個這樣的系統，但是沒做完就離開了。
現在用ganglia管理100多台機器，用起來很方便。
能再吸取一些cacti在圖的縮放上面的精華，就無人能敵了。

有一點沒看明白，你要監控的集群IP什麼的都是在哪裡寫的啊？還有就是不用開SNMP的么？

ganglia有自己的mon程序，每個節點上都需要運行一個gmond，ip不需要寫在伺服器上

就是說每個節點裝一個client？

我謝過一個文檔，論壇很多人問我要過。

這個軟體可以進行二次開發嗎?

我使用的rocks裡面帶有這個軟體,還是挺好用的.不過我希望能在自己的程序中使用它的功能,不知道它有沒有API什麼的可以供C語言調用啊?我看了好象沒有.

原帖由 fuyic 於 2007-2-8 15:16 發表於 6樓  
就是說每個節點裝一個client？

是要在每個cilent上裝一個gmond

good.!:D :D

Tags: