You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

With the new-style psana1 that supports both py2 and py3 use the following commands to create a test release:

mkdir ana-4.0.5
cd ana-4.0.5
source /reg/g/psdm/etc/psconda.sh (defaults to py2, optionally add a "-py3" flag for py3)
source /cds/sw/ds/ana/conda1/manage/bin/setup_testrel
git clone git@github.com:lcls-psana/Detector.git
scons

cpo found that (for reasons I don't currently understand, but perhaps related to conda) LD_LIBRARY_PATH is not taking precedence with test releases and I was forced to use LD_PRELOAD like this (I also have the impression LD_PRELOAD_PATH didn't work):

export LD_PRELOAD=/cds/home/c/cpo/ana-4.0.12/arch/x86_64-rhel7-gcc48-opt/lib/libpsddl_pds2psana.so:/cds/home/c/cpo/ana-4.0.12/arch/x86_64-rhel7-gcc48-opt/lib/libPSXtcInput.so

To compile debug code one has to put "dbg" at end of SIT_ARCH: export SIT_ARCH=x86_64-rhel7-gcc48-dbg.  Remember to set LD_PRELOAD above to point to the dbg area (instead of opt) as well.

This variable is used in https://github.com/lcls-psana/SConsTools/blob/master/src/tools/psdm_cplusplus.py

Some Notes From Dan D. On the LD_PRELOAD Problem

Hi,

The fundamental issue is that the shared libraries are being built in a way to effectively ignore LD_LIBRARY_PATH (more details below*). To get around this I made a modified branch of SConsTools to restore the old behavior of the scons build.

These are the instructions you need to follow:

  1. Run ‘scons -c’ to clean up your test release (or just nuke the ‘arch’ and ‘build’ directories)
  2. In your test release directory clone the SConsTools and checkout the  testrel-fix branch (git clone -b testrel-fix git@github.com:lcls-psana/SConsTools.git)
  3. Now if you build all the shared libraries you built will use LD_LIBRARY_PATH.
  4. Unfortunately, the shared libraries from the base release still won’t use LD_LIBRARY_PATH. To get around this I had to add  psana, PSXtcInput, and PSEvt to my test release and rebuild them. One alternative solution if we don’t want to affect the production releases is to make say a ana.X.Y.Z-dev version of all the base releases that is built with the old behavior and use it as the ‘base’ release for development.

* SConsTools changes:

At some point a line in src/tools/psdm_cplusplus.py was changed from “env.Append(LINKFLAGS = ' ' + _ld_opt.get(opt,'') + ' -Wl,--copy-dt-needed-entries -Wl,--enable-new-dtags')” to “env.Append(LINKFLAGS = ' ' + _ld_opt.get(opt,''))”

This changed the behavior of the shared libraries/executables that we build so that they set RPATH instead of RUNPATH. The generally priority on Linux for searching is RPATH -> LD_LIBRARY_PATH -> RUNPATH. This means if a matching library is in the paths defined in RPATH it will always be found there and LD_LIBRARY_PATH isn’t used. This means that you can’t override the libraries in the base release by putting them in arch/<arch>/lib directory of the test release and adding it to LD_LIBRARY_PATH.

----------------------------------

Mikhail tried the above and it didn't work:  instead Dan suggested: 

for i in ${PWD}/arch/x86_64-rhel7-gcc48-opt/lib/*.so; do export "LD_PRELOAD=$i:$LD_PRELOAD"; done
  • No labels