歡迎您光臨本站 註冊首頁

python并行包pypar在64位機器上的安裝

←手機掃碼閱讀     火星人 @ 2014-03-29 , reply:0

pypar是一個簡單小巧的python包,可用於編寫并行處理的python程序。兩年前在32位機器上,安裝很順利。但近來機群環境變化,需要在64位的Linux系統上重新編譯,需要一點小技巧:

本系統是運行在2 Intel CPU x 雙核 x 雙線程的Dell機器上的Linux:
QUOTE:
「」「
[xxia@dell3 examples]$ uname -r
2.6.9-67.0.4.ELsmp
[xxia@dell3 examples]$ cat /etc/*release*
Fedora Core release 4 (Stentz) but with CentOS 4 kernel
LSB_VERSION="1.3"
Fedora Core release 4 (Stentz) but with CentOS 4 kernel
」「」

首先,安裝MPI (lam-7.1.1)時要建立共享庫:
QUOTE:
「」「
[xxia@dell3 lam-7.1.1]$ ./configure --enable-shared --prefix=/home/xxia/usr && make && make install
」「」

否則,連接pypar時會通不過:
QUOTE:
「」「
[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -shared build/temp.linux-x86_64-2.5/mpiext.o -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -lutil -ldl -o build/lib.linux-x86_64-2.5/pypar/mpiext.so
/usr/bin/ld: /home/xxia/usr/lib/libmpi.a(abort.o): relocation R_X86_64_32 against `lam_mpi_comm_world' can not be used when making a shared object; recompile with -fPIC
/home/xxia/usr/lib/libmpi.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
「」「

其次,在pypar 1.9.3的安裝程序中,64位系統的參數寫法也不正確:
QUOTE:
「」「
if os.name == 'posix' and os.uname()[4] == 'x86_64':
#Extra flags for 64 bit architectures
#extra_compile_args = ' -fPIC -m64' #Valid for gcc
extra_compile_args = ' -fPIC -tp amd64' #Valid for pgcc
else:
extra_compile_args = None
」「」

由此會導致錯誤:
QUOTE:
「」「
[xxia@dell3 pypar_1.9.3]$ python2.4 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.4
creating build/lib.linux-x86_64-2.4/pypar
copying lib/pypar/pypar.py -> build/lib.linux-x86_64-2.4/pypar
copying lib/pypar/__init__.py -> build/lib.linux-x86_64-2.4/pypar
running build_ext
building 'pypar.mpiext' extension
creating build/temp.linux-x86_64-2.4
Traceback (most recent call last):
File "setup.py", line 161, in ?
extra_compile_args=extra_compile_args)]
File "/home/xxia/usr/lib/python2.4/distutils/core.py", line 149, in setup
dist.run_commands()
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands
self.run_command(cmd)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/install.py", line 506, in run
self.run_command('build')
File "/home/xxia/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/build.py", line 112, in run
self.run_command(cmd_name)
File "/home/xxia/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 279, in run
self.build_extensions()
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 405, in build_extensions
self.build_extension(ext)
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 470, in build_extension
depends=ext.depends)
File "/home/xxia/usr/lib/python2.4/distutils/ccompiler.py", line 699, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/home/xxia/usr/lib/python2.4/distutils/unixccompiler.py", line 112, in _compile
self.spawn(self.compiler_so + cc_args + [src, '-o', obj] +
TypeError: can only concatenate list (not "str") to list
[xxia@dell3 pypar_1.9.3]$
」「」

正確的代碼應是:
QUOTE:
「」「
if os.name == 'posix' and os.uname()[4] == 'x86_64':
#Extra flags for 64 bit architectures
#extra_compile_args = ' -fPIC -m64' #Valid for gcc
extra_compile_args = ' -fPIC -tp amd64' #Valid for pgcc

extra_compile_args = [' -fPIC -m64'] #added by xxq
#extra_compile_args = None #added by xxq
#extra_compile_args = [' -fPIC -tp -m64'] #added by xxq
else:
extra_compile_args = None
」「」

即便這們,仍有錯誤:
QUOTE:
「」「[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/xxia/usr/include -I/home/xxia/usr/include/python2.5 -c mpiext.c -o build/temp.linux-x86_64-2.5/mpiext.o -fPIC -m64
gcc: mpiext.c: No such file or directory
gcc: -fPIC -m64: No such file or directory
gcc: no input files
error: command 'gcc' failed with exit status 1
[xxia@dell3 pypar_1.9.3]$
「」「

解決辦法是將參數 "-fPIC -m64"前置,手工編譯:
QUOTE:
」「」
[xxia@dell3 pypar_1.9.3]$ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/xxia/usr/include -I/home/xxia/usr/include/python2.5 -c mpiext.c -fPIC -m64 -o build/temp.linux-x86_64-2.5/mpiext.o
「」「

然後再安裝:
QUOTE:
「」「[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -shared build/temp.linux-x86_64-2.5/mpiext.o -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -lutil -ldl -o build/lib.linux-x86_64-2.5/pypar/mpiext.so
running install_lib
creating /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/pypar.py -> /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/__init__.py -> /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/mpiext.so -> /home/xxia/usr/lib/python2.5/site-packages/pypar
byte-compiling /home/xxia/usr/lib/python2.5/site-packages/pypar/pypar.py to pypar.pyc
byte-compiling /home/xxia/usr/lib/python2.5/site-packages/pypar/__init__.py to __init__.pyc
running install_egg_info
Writing /home/xxia/usr/lib/python2.5/site-packages/Pypar-1.9.2-py2.5.egg-info
「」「

至此,大功乃成。

[火星人 ] python并行包pypar在64位機器上的安裝已經有872次圍觀

http://coctec.com/docs/linux/show-post-200893.html