Re: [ESPResSo-users] Error in python code checks

[Pierre de Buyl]

Hi Michael,

In your log, it appears that some packages are detected from a conda Python
installation. For instance, the MPI library is found with conda, which can lead
to incompatibilities.

I guess that Henri's test was with the system Python and the Ubuntu packaged
MPI.

I also run Ubuntu 16.04 on my desktop and can run with the system packages. I
don't use conda. I use the packages for boost, mpi, fftw, etc.

I use a virtual environment for Python, with cython, h5py, matplotlib, numpy,
scipy, sphinx.

>From a clean build, I first activate the Python virtual environment, then run
cmake.

Hope it helps.

P

On Thu, Mar 29, 2018 at 01:15:43PM +0200, Michael Klatt wrote:
> Dear Rudolf, Dear Henri,
> 
> Thank you very much for your e-mails!
> I appreciate your quick help very much.
> 
> Unfortunately, there seems to be still some conflict.
> 
> It is indeed my laptop (Xubuntu 16.04) where I have been trying to compile
> espresso. So, I can install software freely. My laptop has two cores and
> each core can have two threads. Could the problem be caused if pypresso
> tries to call more than four threads?
> 
> If not, here is an update on the error reports:
> 
> I used new (that is empty) build directories and libboost-mpi-dev was
> already installed.
> 
> The packages "python-h5py python-enum libhdf5-openmpi-dev" were actually
> missing. So I installed them and built espresso in a completely new
> directory.
> 
> There is still an error message (although it seems to have slightly
> changed), see the postscript.
> 
> I have attached the output of cmake. Is that what you actually meant?
> Do you need anything else?
> 
> Concerning the Docker files. I am sorry, but I am not used to Docker files.
> How should I used them?
> 
> Thank you very much once more for your help!
> 
> Best regards,
> 
> Michael
> 
> $ make check
> [  0%] Built target myconfig
> [  0%] Built target check_myconfig
> [  0%] Built target EspressoConfig
> Scanning dependencies of target link_cell_test
> [  0%] Building CXX object
> src/core/unit_tests/CMakeFiles/link_cell_test.dir/link_cell_test.cpp.o
> [  0%] Linking CXX executable link_cell_test
> [  0%] Built target link_cell_test
> Scanning dependencies of target Cache_test
> [  1%] Building CXX object
> src/core/unit_tests/CMakeFiles/Cache_test.dir/Cache_test.cpp.o
> [  1%] Linking CXX executable Cache_test
> [  1%] Built target Cache_test
> Scanning dependencies of target None_test
> [  1%] Building CXX object
> src/core/unit_tests/CMakeFiles/None_test.dir/None_test.cpp.o
> [  2%] Linking CXX executable None_test
> [  2%] Built target None_test
> Scanning dependencies of target all_compare_test
> [  3%] Building CXX object
> src/core/unit_tests/CMakeFiles/all_compare_test.dir/all_compare_test.cpp.o
> [  3%] Linking CXX executable all_compare_test
> CMakeFiles/all_compare_test.dir/all_compare_test.cpp.o: In function `bool
> Utils::Mpi::all_compare<int>(boost::mpi::communicator const&, int const&)':
> all_compare_test.cpp:(.text._ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_[_ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_]+0x3e):
> undefined reference to `boost::mpi::communicator::operator int() const'
> all_compare_test.cpp:(.text._ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_[_ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_]+0xa2):
> undefined reference to `boost::mpi::communicator::operator int() const'
> collect2: error: ld returned 1 exit status
> src/core/unit_tests/CMakeFiles/all_compare_test.dir/build.make:126: recipe
> for target 'src/core/unit_tests/all_compare_test' failed
> make[3]: *** [src/core/unit_tests/all_compare_test] Error 1
> CMakeFiles/Makefile2:1599: recipe for target
> 'src/core/unit_tests/CMakeFiles/all_compare_test.dir/all' failed
> make[2]: *** [src/core/unit_tests/CMakeFiles/all_compare_test.dir/all] Error
> 2
> CMakeFiles/Makefile2:107: recipe for target 'CMakeFiles/check.dir/rule'
> failed
> make[1]: *** [CMakeFiles/check.dir/rule] Error 2
> Makefile:186: recipe for target 'check' failed
> make: *** [check] Error 2
> 
> -------------------------------------------------------------------------
> 
> P.P.S. $ ./pypresso
> Python 2.7.14 |Anaconda, Inc.| (default, Dec  7 2017, 17:05:42)
> [GCC 7.2.0] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import espressomd
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File 
> "/home/mklatt/postdoc-working/workbench/espresso/build/src/python/espressomd/__init__.py",
> line 23, in <module>
>     import espressomd._init
> ImportError: 
> /home/mklatt/postdoc-working/workbench/espresso/build/src/core/io/writer/h5md/libH5mdCore.so:
> undefined symbol: 
> _ZN5boost3mpi6detail19packed_archive_recvEiiiRNS0_15packed_iarchiveER10MPI_Status
> >>>
> 
> 
> 
> 
> On 29.03.2018 01:57, Henri Menke wrote:
> >On Wed, 2018-03-28 at 20:55 +0200, Michael Klatt wrote:
> >>Dear all,
> >>
> >>I am new user of ESPResSo. Unfortunately, I can only run the TCL
> >>scripts, but pypresso causes errors that I cannot resolve on my own.
> >>
> >>Is there a TCL counterpart of the python minimize_energy functionality?
> >>If not may I ask for your help to resolve my problems with pypresso?
> >>Thank you very much!
> >>
> >>In the following, I summarize the error reports (full error reports are
> >>included in the postscript).
> >>
> >>I am using Xubuntu 16.04. Neither the latest release (3.3.1) of ESPResSo
> >>nor cloning the python branch on GitHub worked. Resolving the problems
> >>for any of the two would suffice.
> >
> >I just compiled ESPResSo on a naked Ubuntu 16.04 to see if I can reproduce 
> >your
> >issue, but I couldn't.  Here are my steps:
> >
> >apt-get update
> >apt-get install build-essential cmake git openmpi-bin libfftw3-dev 
> >libboost-dev libboost-serialization-dev libboost-mpi-dev 
> >libboost-filesystem-dev libboost-test-dev cython python python-numpy 
> >python-h5py python-enum libhdf5-openmpi-dev libhdf5-openmpi-10 libhdf5-10
> >git clone https://github.com/espressomd/espresso.git
> >cd espresso/
> >mkdir build
> >cd build/
> >cmake ..
> >make -j 16
> >
> >Then I can run ./pypresso
> >
> >>>>import espressomd
> >>>>print(espressomd.features())
> >['BOND_ANGLE', 'BUCKINGHAM', 'COLLISION_DETECTION', 'CONSTRAINTS', 
> >'ELECTROSTATICS', 'EXCLUSIONS', 'EXTERNAL_FORCES', 'FFTW', 'GAUSSIAN', 
> >'GHOSTS_HAVE_BONDS', 'H5MD', 'HERTZIAN', 'LANGEVIN_PER_PARTICLE', 'LATTICE', 
> >'LB', 'LB_BOUNDARIES', 'LENNARD_JONES', 'LENNARD_JONES_GENERIC', 'LJCOS', 
> >'LJCOS2', 'MASS', 'MORSE', 'NPT', 'P3M', 'PARTIAL_PERIODIC', 'SOFT_SPHERE', 
> >'TABULATED']
> >
> >
> >
> >>-------------------------------------------------------------------------
> >>
> >>(1) When I compiled the latest release (3.3.1), "import espressomd"
> >>causes no problems, but "print(espressomd.features())" throws an error:
> >>Traceback (most recent call last):
> >>File "<stdin>", line 1, in <module>
> >>AttributeError: 'module' object has no attribute 'features'
> >>
> >>-------------------------------------------------------------------------
> >>
> >>(2) Last week, I have cloned the python branch of ESPResSo and followed
> >>the instructions for installation from the ESPResSo documentation.
> >>
> >>"make install" was completed without errors.
> >>However, "make check" reported a problem:
> >>undefined reference to `boost::mpi::communicator::operator int() const'
> >>
> >>When I import espressomd in "pypresso", the following error is reported:
> >>libcluster_analysis.so.4: undefined symbol:
> >>_ZNK5boost3mpi12communicatorcviEv
> >>
> >>-------------------------------------------------------------------------
> >>
> >>Thank you very much once more for your help!
> >>
> >>Best regards,
> >>
> >>Michael
> >>
> >>-------------------------------------------------------------------------
> >>
> >>P.S. Error when invoking code checks:
> >>
> >>$ make check
> >>[ 2%] Built target myconfig
> >>[ 2%] Built target check_myconfig
> >>[ 2%] Built target EspressoConfig
> >>[ 2%] Built target link_cell_test
> >>[ 2%] Built target Cache_test
> >>[ 2%] Built target None_test
> >>[ 2%] Linking CXX executable all_compare_test
> >>CMakeFiles/all_compare_test.dir/all_compare_test.cpp.o: In function
> >>`bool Utils::Mpi::all_compare<int>(boost::mpi::communicator const&, int
> >>const&)':
> >>all_compare_test.cpp:(.text._ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12comm
> >>unicatorERKT_[_ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_]
> >>+0x3e):
> >>undefined reference to `boost::mpi::communicator::operator int() const'
> >>all_compare_test.cpp:(.text._ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12comm
> >>unicatorERKT_[_ZN5Utils3Mpi11all_compareIiEEbRKN5boost3mpi12communicatorERKT_]
> >>+0xa2):
> >>undefined reference to `boost::mpi::communicator::operator int() const'
> >>collect2: error: ld returned 1 exit status
> >>src/core/unit_tests/CMakeFiles/all_compare_test.dir/build.make:121:
> >>recipe for target 'src/core/unit_tests/all_compare_test' failed
> >>make[3]: *** [src/core/unit_tests/all_compare_test] Error 1
> >>CMakeFiles/Makefile2:1510: recipe for target
> >>'src/core/unit_tests/CMakeFiles/all_compare_test.dir/all' failed
> >>make[2]: *** [src/core/unit_tests/CMakeFiles/all_compare_test.dir/all]
> >>Error 2
> >>CMakeFiles/Makefile2:107: recipe for target 'CMakeFiles/check.dir/rule'
> >>failed
> >>make[1]: *** [CMakeFiles/check.dir/rule] Error 2
> >>Makefile:186: recipe for target 'check' failed
> >>make: *** [check] Error 2
> >>
> >>-------------------------------------------------------------------------
> >>
> >>P.P.S. Error when importing espressomd in "pypresso":
> >>
> >>Traceback (most recent call last):
> >>File "<stdin>", line 1, in <module>
> >>File "/usr/local/lib/python2.7/site-packages/espressomd/__init__.py",
> >>line 23, in <module>
> >>import espressomd._init
> >>ImportError: /home/mklatt/[...path to
> >>espresso...]/espresso/build/src/core/cluster_analysis/libcluster_analysis.so.4
> >>:
> >>undefined symbol: _ZNK5boost3mpi12communicatorcviEv
> >>
> >>
> 
> -- 
> Karlsruhe Institute of Technology (KIT)
> Institute of Stochastics
> 
> Englerstraße 2
> 76131 Karlsruhe, Germany
> 
> Phone: +49 721 608 46688
> E-mail: address@hidden
> Web: http://mklatt.org
> 
> KIT --- The Research University in the Helmholtz Association

> -- The C compiler identification is GNU 5.4.0
> -- The CXX compiler identification is GNU 5.4.0
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Found Git: /usr/bin/git (found version "2.7.4") 
> -- Config file: 
> /home/mklatt/postdoc-working/workbench/espresso/src/core/myconfig-default.hpp
> -- Performing Test result__PRETTY_FUNCTION__
> -- Performing Test result__PRETTY_FUNCTION__ - Success
> CUDA_TOOLKIT_ROOT_DIR not found or specified
> -- Could NOT find CUDA (missing:  CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE 
> CUDA_INCLUDE_DIRS) (Required is at least version "7.0")
> -- Found PythonInterp: /home/mklatt/Downloads/miniconda2/bin/python (found 
> version "2.7.14") 
> -- Found Cython version 0.23
> -- Found Cython: /usr/bin/cython (Required is at least version "0.23") 
> -- Found NumPy: 
> /home/mklatt/.local/lib/python2.7/site-packages/numpy/core/include (found 
> version "1.14.2") 
> -- NumPy ver. 1.14.2 found (include: 
> /home/mklatt/.local/lib/python2.7/site-packages/numpy/core/include)
> -- Found FFTW3: /usr/lib/x86_64-linux-gnu/libfftw3.so  
> -- Found HDF5: 
> /usr/lib/x86_64-linux-gnu/hdf5/openmpi/lib/libhdf5.so;/usr/lib/x86_64-linux-gnu/libsz.so;/home/mklatt/Downloads/miniconda2/lib/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so
>  (found suitable version "1.8.16", minimum required is "1.8") 
> -- The Python module h5py was not found. (missing:  PY_H5PY) 
> -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") 
> -- Checking for module 'scafacos'
> --   No package 'scafacos' found
> -- Found GSL: /usr/include (found version "2.1") 
> -- Found MPI_C: 
> /home/mklatt/Downloads/miniconda2/lib/libmpich.so;/home/mklatt/Downloads/miniconda2/lib/libopa.so;/home/mklatt/Downloads/miniconda2/lib/libmpl.so;/usr/lib/x86_64-linux-gnu/librt.so;/usr/lib/x86_64-linux-gnu/libpthread.so
>   
> -- Found MPI_CXX: 
> /home/mklatt/Downloads/miniconda2/lib/libmpichcxx.so;/home/mklatt/Downloads/miniconda2/lib/libmpich.so;/home/mklatt/Downloads/miniconda2/lib/libopa.so;/home/mklatt/Downloads/miniconda2/lib/libmpl.so;/usr/lib/x86_64-linux-gnu/librt.so;/usr/lib/x86_64-linux-gnu/libpthread.so
>   
> -- Boost version: 1.58.0
> -- Found the following Boost libraries:
> --   mpi
> --   serialization
> --   filesystem
> --   system
> --   unit_test_framework
> -- Found Doxygen: /usr/bin/doxygen (found version "1.8.11") 
> -- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE SPHINX_API_DOC_EXE) 
> (Required is at least version "1.6.6")
> -- writing 
> /home/mklatt/postdoc-working/workbench/espresso/build/myconfig-sample.hpp
> -- 
> -- The following features have been enabled:
> 
>  * HDF5 (required version >= 1.8) , parallel
> 
> -- The following OPTIONAL packages have been found:
> 
>  * FFTW3
>  * HDF5 (required version >= 1.8) , parallel
>  * Git
>  * PkgConfig
>  * GSL
>  * Doxygen
> 
> -- The following REQUIRED packages have been found:
> 
>  * PythonInterp
>  * Cython (required version >= 0.23)
>  * NumPy
>  * MPI
>  * Boost (required version >= 1.53.0)
> 
> -- The following OPTIONAL packages have not been found:
> 
>  * CUDA (required version >= 7.0)
>  * Sphinx (required version >= 1.6.6)
> 
> -- Configuring done
> -- Generating done
> -- Build files have been written to: 
> /home/mklatt/postdoc-working/workbench/espresso/build


-- 
-----------------------------------------------------------
Pierre de Buyl
KU Leuven - Institute for Theoretical Physics
T +32 16 3 27355
W http://pdebuyl.be/
-----------------------------------------------------------