...

This "dbg" variable is used in https://github.com/lcls-psana/SConsTools/blob/master/src/tools/psdm_cplusplus.py

External Packages

If working on external packages (for example, ndarray, psalg, pdsdata, python), you need to also to check out the corresponding proxy package. For example: ndarray_ext, Python, psalg_ext, pdsdata_ext, etc. The Python proxy package allows the python header files to be visible to C++ code.  Sometimes you will also get missing symbols for packages like boost and then that external package must also be checked out.

Some Notes From Dan D. On the LD_PRELOAD Problem

NOTE: as of Dec. 2022 we don't think LD_PRELOAD is needed.  A combination of the existing rpath approach plus checking out external packages (see above section) seems to be sufficient.

...

for i in ${PWD}/arch/x86_64-rhel7-gcc48-opt/lib/*.so; do export "LD_PRELOAD=$i:$LD_PRELOAD"; done

Conda LD_LIBRARY_PATH Incompatibility

• conda has it's own "sysroot" with /lib and /include
• these can conflict with OS default sys root, but conda avoids that by using RPATH in the executables
• setting LD_LIBRARY_PATH can couple these two incompatible worlds: we saw this with conda's libattr.so.1 (dependency of Qt) breaking "ls"
• but psana1 test releases need to load alternate libraries (currently we use LD_LIBRARY_PATH)
• proposed solution: use LD_PRELOAD which can be set on a library-by-library basis for psana1 test releases
• when we build a binary in a development release we don't use RPATH, so that's why we had been using LD_LIBRARY_PATH (we think)
• preliminary test-release possible proposal:
  • eliminate use of LD_LIBRARY_PATH
  • use RPATH for building development binaries
    • can be set to ONLY point to conda's sysroot (this would be just like conda)
    • could also set to first the test-release library path then conda, but it might simpler to just use LD_PRELOAD everywhere
  • use LD_PRELOAD (which can be set on per-library basis) to "jam" development .so files to run
  • to test this: checkout "psana" (builds some binaries) and "Detector" (this creates a .so)
• modified proposal:
  • valmar and cpo found that binaries and libraries have $ORIGIN/../lib and $ORIGIN/../lib/python in the RPATH.  feels like this should work without LD_PRELOAD and LD_LIBRARY_PATH.  We'll try that and see what failure modes emerge.
  • conclusion so far (Dec. 2022) is that this seems to be working if we also checkout the right external packages (see ExternalPackages)
• order of library path searching:  https://en.wikipedia.org/wiki/Rpath#GNU_ld.so.  we think conda uses RPATH not RUNPATH.

pkg_tree.pkl Problem

When building a test release with py3 can get this error:

(ana-4.0.48-py3) scons
scons: Reading SConscript files ...
Mkdir("build/x86_64-rhel7-gcc48-opt")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 61: invalid start byte:
  File "/sdf/home/c/cpo/ana-4.0.48/SConstruct", line 186:
    loadPkgDeps(fname)
  File "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.48-py3/lib/python3.9/site-packages/SConsTools/dependencies.py", line 262:
    env['PKG_TREE_BASE'].update( pickle.load( f ) )
(ana-4.0.48-py3) 

This is because files like /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.48-py3/data/anarelinfo/.pkg_tree.pkl were generated with py2.  There is a conversion script in in manage/bin/pickle2to3.py but it broke at some point, so currently Valerio has been copying the file from ana-4.0.46-py3