Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Accessible through url:
file:///cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

added bin/ana-rel-admin: sha256sum .../psana-conda-4.0.56.tar.gz > 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

why do not copy psana-conda-4.0.56.tar.gz  to  /reg/g/psdm/web/swdoc/tutorials/ in bin/ana-rel-admin?

Conda build

ssh pslogin (not as psreldev)

then copy it here, because psreldev does not have permission !!!!???

...

Code Block
titleedit psana1-feedstock/recipe/meta.yaml
collapsetrue
{% set version = '4.0.56' %}
 ...
 sha256: 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
 ...
extra:
  recipe-maintainers:
    - valmar
    - chrisvam
    - dubrovin

update it in github # do it in, because psreldev does not have permission !!!!??? https:/https://github.com/slac-lcls/psana1-feedstock/blob/main/recipe/meta.yaml

do it on github directly, because psreldev does not have permission !!!!???

Fixing issue in conda build with permission denied

Issue: PermissionError: [Errno 13] Permission denied: '/cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/libselinux-cos6-x86_64-2.0.94-h9b0a68f_1105'

...

Code Block
titleconda build - the tail of output
collapsetrue
    xorg-xf86vidmodeproto:     2.3.1-h7f98852_1002          conda-forge
    xorg-xproto:               7.0.31-h7f98852_1007         conda-forge
    xz:                        5.2.6-h166bdaf_0             conda-forge
    zeromq:                    4.3.5-h59595ed_0             conda-forge
    zipp:                      3.17.0-pyhd8ed1ab_0          conda-forge
    zlib:                      1.2.13-hd590300_5            conda-forge
    zstd:                      1.5.5-hfc55251_0             conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
export PREFIX=/cds/home/p/psreldev/conda-bld/psana_1700187012881/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl
export SRC_DIR=/cds/home/p/psreldev/conda-bld/psana_1700187012881/test_tmp
+ psana -h

Usage: psana [options] [dataset ...]

  Available options:
    {-h|-?|--help    }         print help message
    {-v|--verbose    } (incr)  verbose output, multiple allowed (initial: 0)
    {-q|--quiet      } (incr)  quieter output, multiple allowed (initial: 2)
    {-b|--calib-dir  } path    calibration directory name, may include {exp} and {instr}, if left empty then do not do calibrations (default: "")
    {-c|--config     } path    configuration file, by default use psana.cfg if it exists (default: "")
    {-e|--experiment } string  experiment name, format: XPP:xpp12311 or xpp12311, by default guess it from data (default: "")
    {-j|--job-name   } string  job name, default is to generate from input file names (default: "")
    {-m|--module     } name    module name, more than one possible
    {-n|--num-events } number  maximum number of events to process, 0 means all (default: 0)
    {-s|--skip-events} number  number of events to skip (default: 0)
    {-p|--num-cpu    } number  number greater than 0 enables multi-processing (default: 0)
    {-o|--option     } string  configuration options, format: module.option[=value]

  Positional parameters:
    dataset - input dataset specification (list of file names or exp=cxi12345:run=123:...)

+ exit 0

Resource usage statistics from testing psana:
   Process count: 1
   CPU time: Sys=0:00:00.1, User=-
   Memory: 1.4M
   Disk usage: 12B
   Time elapsed: 0:00:02.4


TEST END: /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO:conda_build.utils:Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO conda_build.utils:shutil_move_more_retrying(2075): Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO:conda_build.utils:shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO conda_build.utils:shutil_move_more_retrying(2082): shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes
anaconda_upload is not set.  Not uploading wheels: []
####################################################################################
Resource usage summary:

Total time: 1:22:33.6
CPU usage: sys=0:03:04.5, user=0:27:14.5
Maximum memory usage observed: 1.6G
Total disk usage observed (not including envs): 285.6M


####################################################################################
Source and build intermediates have been left in /cds/home/p/psreldev/conda-bld.
There are currently 1 accumulated.
To remove them, you can run the ```conda build purge``` command
(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$

...

Code Block
titleanaconda upload messages
collapsetrue
(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$ anaconda upload /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "chrisvam" as upload username
Processing '/cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2'
Detecting file type...
File type is "conda"
Extracting conda package attributes for upload
Creating package "psana"
Creating release "4.0.56"
Uploading file "chrisvam/psana/4.0.56/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2"
 uploaded 18698 of 18698Kb: 100.00% ETA: 0.0 minutes
Upload complete

conda package located at:
https://anaconda.org/chrisvam/psana

(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$

...

Questions

  • see interlaced in red
  • removing 7 packages for modular psana, like in config/psana-conda-svn-pkgs-test (saves ~1.5% of space in tar.gz.)
  • location for data_test can be controlled in config/psana-conda-svn-pkgs:
    • subdir=.. : manage/scratch/data_test with fixed path /cds/sw/ds/ana/conda1/manage/scratch/data_test
    • subdir=/tmp : /tmp/data_test
  • how to speed-up conda build and jamp directly to scons test-Detector?

References