redhat as4 u5 配置hp cluster 問題
使用紅帽子cluster軟體配置雙機后，fence_ilo -a (ilo ip addr) -l (username) -p (password) -o (off on status reboot)
off 和status執行后結果正常，但reboot 或on 會報錯
Command without TOGGLE="Yes" attribute is ignored when host power is off.
power_on: unexpected error，求解
《解決方案》
回復 #1 jihua2005 的帖子
內置fence的現象
《解決方案》
回復 #1 jihua2005 的帖子
個人建議將ilo的firmware升級到最新。
《解決方案》
回復 #1 jihua2005 的帖子
我搜了些文章，似乎早起版本有bug，你參考下：
Reporter: Shane Bradley
Created: Thu, 21 Feb 2008 12:26:53
Updated: Thu, 16 Oct 2008 05:52:37
Key: 433864
Versions: Not provided
Environment: i686
Priority: 3
Status: In Progress
Resolution: Not provided
Original Link: https://bugzilla.redhat.com/show_bug.cgi?id=433864
Summary: RHEL5u1 /sbin/fence_ilo fails with 'failed to turn on'
Description:
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.12)
Gecko/20080208 Fedora/2.0.0.12-1.fc8 Firefox/2.0.0.12
Description of problem:
RHEL 5u1 with latest updates to all packages.
HP ilo firmware: 1.91 (tested 1.89 and get same results)
ACPI is disabled at kernel level and runlevel.
When a machine that has a fence device of type HP ilo or HP ilo2 the
machine
will not fence correctly when it is a member of the cluster. If the machine
is
not a member of the cluster, then fencing will work correctly.
The commands ran when node is member of cluster:
$ fence_ilo -a 10.10.56.56 -l Administrator -p redhatrules -o reboot
$ fence_node bl20pg2-1.gsslab.rdu.redhat.com
Node's fence defination takes default which is reboot.
<device name="ilo1"/>
The fence will fail with an error: failed to turn on
/var/log/messages will contain messages such as this below:
Feb 20 16:19:04 cluster2-3 fenced: agent "fence_ilo" reports: failed
to
turn on
Feb 20 16:19:04 cluster2-3 fenced: fence "bl20pg2-
1.gsslab.rdu.redhat.com"
failed
Feb 20 16:19:09 cluster2-3 fenced: fencing node
"bl20pg2-1.gsslab.rdu.redhat.com"
The ilo event log shows that there is an extra "OFF" in the fence action.
What
actually occurs is "OFF,ON,OFF" and when fence device requests status it is
returned "OFF" which is true because of that extra "OFF" that is ran.
------------------
ILO event log
------------------
*EXTRA off
Caution iLO 02/19/2008 15:48 02/19/2008 15:48 1 Host server powered OFF by:
*Extra off
Administrator. Informational iLO 02/19/2008 15:48 02/19/2008 15:48 4 XML
logout: Administrator. Informational iLO 02/19/2008 15:48 02/19/2008 15:48
1
Host server powered ON by: Administrator. Informational iLO 02/19/2008
15:48
02/19/2008 15:48 4 XML login: Administrator - 10.16.252.176 Caution iLO
02/19/2008 15:47 02/19/2008 15:47 1 Host server powered OFF by:
Administrator.
Informational iLO 02/19/2008 15:47 02/19/2008 15:47 4 XML logout:
Administrator.
Informational iLO 02/19/2008 15:47 02/19/2008 15:47 4 XML login:
Administrator
- 10.16.252.176
--------------------------------------------------------------------------
------
If I change the fencing section of cluster.conf so that instead of doing
reboot, it does off then on. It appears to work, however i
<device name="ilo1" action="off"/>
<device name="ilo1" action="on"/>
The fence will work, however it works with errors. It appears that the
first action "OFF" fails, however it marches on and tries "ON". On is
successful and the fence works. This would lead me to believe that "OFF" is
not returning the correct status or is mangled. This would explain the 2nd
"OFF" that happens in reboot action. The OFF is unsuccessful, then it tries
ON. ON works, but before its status is check OFF is retried. Which leads to
the power status stating that the "ON" failed.
Feb 21 14:58:54 cluster2-3 ccsd: process_get: Invalid connection
descriptor received.
Feb 21 14:58:54 cluster2-3 ccsd: Error while processing get: Invalid
request descriptor
Feb 21 14:58:54 cluster2-3 ccsd: Attempt to close an unopened CCS
descriptor (558150).
Feb 21 14:58:54 cluster2-3 ccsd: Error while processing disconnect:
Invalid request descriptor
Feb 21 14:58:54 cluster2-3 fence_node: Fence of
"bl20pg2-1.gsslab.rdu.redhat.com" was unsuccessful
Feb 21 14:58:55 cluster2-3 dhclient: DHCPREQUEST on eth1 to 10.10.56.49
port 67
Feb 21 14:59:39 cluster2-3 last message repeated 3 times
Feb 21 14:59:48 cluster2-3 dhclient: DHCPREQUEST on eth1 to 10.10.56.49
port 67
Feb 21 14:59:48 cluster2-3 fenced: fence "bl20pg2-
1.gsslab.rdu.redhat.com"
success
Feb 21 14:59:53 cluster2-3 ccsd: Attempt to close an unopened CCS
descriptor (558390).
Feb 21 14:59:53 cluster2-3 ccsd: Error while processing disconnect:
Invalid request descriptor
---sbradley
Version-Release number of selected component (if applicable):
cman-2.0.73-1.el5_1.1
How reproducible:
Always
Steps to Reproduce:
1.setup rhel5u1 machine(with fence ilo/ilo2) with cman installed.
2.add the machine to the cluster
3.start cman on that node
4.fence_node <nodename>
Actual Results:
The node that was fenced fails to reboot.
Fails with error "failed to turn on"
Expected Results:
The node should have been hard rebooted by the ilo device.
Additional info:
FWIW, someone else reported this recently as well to me on IRC.
Hi, i'm heving this problem too.
# fence_node node2.local
agent "fence_ilo" reports: failed to turn on
And then the node2.local is turned off.
acpid is turned off
iLO 2 Firmware Version: 1.43 12/12/2007
It started happen since i updated the version of rhel from 5.0 to 5.1
x86_64
Regards,
FabioSilva
fssilva@gmail.com
I have verified this workaround is valid below.
-----------------------------------------------
*Make sure you have turned off acpi at runlevel and kernel level.
$ cp /etc/cluster/cluster.conf /root
$ emacs /etc/cluster/cluster.conf
*Update version number by 1.
*Then edit the fence device section for each node for example:
<method name="1">
<device name="ilo1"/>
</method>
change to -->
<method name="1">
<device name="ilo1" action="off"/>
<device name="ilo1" action="on"/>
</method>
Do this for all cluster nodes, keep in mind the above is an example.
Please use values that are in your cluster.conf.
-------
After change is made and file is saved, update cluster with new
cluster.conf.
RHEL5
$ ccs_tool update /etc/cluster/cluster.conf
RHEL4
$ ccs_tool update /etc/cluster/cluster.conf
$ cman_tool version -r <version number>
Then run this command to verify changes are made and they have new version
number.
$ cman_tool status
Restart cluster services on both nodes(I believe a restart of fenced is
needed).
Test your fencing with fence_node command:
$ fence_node <node name>
Edited summary to match actual error message.
Fixed in commit 227fd2259db164351f4a87df11f0aaca7e8e8431
Comments:
