Memo‎ > ‎

Install mpi4py on Amazon EC2

posted May 29, 2014, 8:00 AM by Teng-Yok Lee   [ updated May 29, 2014, 8:36 AM ]
It sounds very simple

# REF:
$ sudo pip install mpi4py

Nevertheless, as my EC2 instance use openmpi-devel installed via yum, it cannot find mpi.h. Later I searched for the installation path and found it:

# REF:
$ rpm -ql openmpi-devel | grep mpi.h

Then how to run pip with the specified mpi.h?

NOTE: mpi4py will look for pyconfig.h and Python.h, which are only available in python2.7.

As a result, build from the source is needed. The following steps are my procedure:
$ wget
$ tar zxvf mpi4py-1.3.1.tar.gz
# Edit the mpi section in mpi4py-1.3.1/mpi.cfg as follows

## define_macros        =
## undef_macros         =
## include_dirs         = %(mpi_dir)s/include
include_dirs         = /usr/include/openmpi-x86_64:/usr/include/python2.7
## libraries            = mpi
libraries            = mpi
## library_dirs         = %(mpi_dir)s/lib
library_dirs         = /usr/lib64/openmpi/lib
## runtime_library_dirs = %(mpi_dir)s/lib
runtime_library_dirs = /usr/lib64/openmpi/lib
$ python2.7 build
$ sudo python2.7 install

# Edit ~/.bashrc by adding the following line:
# export PYTHONPATH=$PYTHONPATH:/usr/lib64/python2.7/site-packages/

Meanwhile, I also use mpirun+NFS to install mpi4py to all nodes.

$ cd /mnt/share/mpi4py-1.3.1
$ mpirun -hostfile ~/hostfile -n 2 -npernode 1 sudo python2.7 install

NOTE: To make this work, root need the permission to write to the NFS (here /mnt/share). Then in /etc/exports, the permission for each drive should be (rw,no_root_squash) (REF: