Week of April 30 2018
Week of Apr 23 2018
- DM-DC2
- Set up agenda and write up notes for Monday's DM-DC2 meeting
- Rerun one processEimage job on one core on KNL as requested by Salman
- Send Salman directions about the steps to run "mini-DRP" so they can try it at ANL
- Provide NERSC update for DC2 meeting
- Attend DC2 meeting
- NERSC
- Create an area of IN2P3 to transfer their data products
- nEXO
- Install new version of SNiPER for centos7
- request new tbb system library be installed
- Send some initial SLAC usage instructions to the SNiPER developers
- Install new version of SNiPER for centos7
- SLAC
- Help Steve ssh tunnel to new IR2 jupyterhub
Week of Apr 16th 2018
- DM-DC2
- In support of SkyCorrection Task
- Generate darks using phosim v3.7.9
- Play with constructFlat.
- Rerun miniDRP task using KNL and produce some timing information
- Update LSSTDESC/obs_lsstSIm for Phosim v3.7.9
- Tagged Run1.1p obs_lsstSim fork
- Successfully updated the obs_lsstSIm files using the bias output
- Work through unit test failures
- PR
- In support of SkyCorrection Task
- NERSC
- Clear out out Stack builds as advertised on Slack
- Install parts in shared python env + weeklies + v15
- Consider setting up a common shared lib area
- Install a modern dmtcp on knl-intel and haswell-intel
- multilib install failed, went back to 64 bit only for now - which should be fine for phosim
- nEXO
- Prepare for travel to SLAC
- Help SNiPER Developers get set up with SLAC accounts
- Install new version of SNiPER at SLAC
Week of Apr 9 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- Attend Fermi Meeting
- nEXO
- HistogramSvc
- Write up SLAC SNiPER installation instructions on wiki
- Send Lisa some journal references for Gaudi
- DM-DC2
- Prepare for Task Force Meeting and write up Live Notes
- Complete initial run of first Run 1.2p visits through minidrp
- Create new dc2 Shifter image with room for a NERSC volume
- Create dc2 Shifter image using a dc2 tag to try to avoid version mismatch errors
- update workflow to use Shifter and submit PR
- Create w_2018_14 + LSSTDESC/obs_lsstSim Shifter image
- Test pipeline with w_2018_14+LSSTDESC/obs_lsstSim Shifter
- update NERSC workflow to try out processEimage on rafts
- NERSC
- Prepare updates for CI meeting
- Attend Ci meeting
- Send Katrin some information about how the set up NERSC for DESC
- Ask NERSC about anticipated need to recompile software after the scheduled Cori OS update on Apr 25-26
- Install DMstack v15 in the common software area
- Request to remove some older stack installations at NERSC
- SLAC
- Write up portal section in Overleaf
- Complete Research integrity course
- Aux Telescope
Week of Apr 2 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- Attend Fermi Meeting
- nEXO
- HistogramSvc
- Attend Simulations Meeting
- Write up SLAC SNiPER installation instructions on wiki
- DM-DC2
- Set up new Docker image which includes patched obs_lsstSIm
- Prep slides and attend SSim meeting
- NERSC
- Check on weekly builds and document their new location
- Create new docker image for new sims 2_7_0 release
- Install sims 2_7_0 from source in the common software area
- SLAC
- Complete CyberSecurity Training
- Write up portal section in Overleaf
- get antivirus going on home centos7 to allow VPN connection required to deploy portal
- Aux Telescope
Week of Mar 26 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- Attend Fermi Meeting
- nEXO
- HistogramSvc
- DM-DC2
- Prep for Task Force Meeting
- Write up Live Notes
- Contact Dominque B concerning processEimage failure
- Now fixed with new config file
- add this config file to github
- Write update for Friday's DC2 meeting
- Write update for CI meeting
- Start up new mini-DRP task in SRS pipeline
- Run ingestData for all 66 visits from Run 1.1p
- Run processEimage (obtain time and memory consumption stats)
- SLAC
- Fix LSSTTD-1257
- Deploy data portal 0.5.14
- NERSC
- move weekly builds to common software area by updating cron job
- Aux Telescope
- On Hold until we hear from Kirk
Week of Mar 19 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- Attend Fermi Meeting
- nEXO
- Fix build of ROOT on centos7, missing libEve and libRGL
- Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- Attend simulations meeting
- DM-DC2
- DM-DC2 Meeting prep
- Attend SSims meeting
- Write up Live notes
- Extend DRP Workflow to run processEimage
- Update DRP Workflow to run processEimage for multiple visits
- Meet with Nicolas to discuss DRP
- SLAC
- NERSC
- Install TreeCorr
- Install new GCRCatalogs in all conda environments
- Move weekly stack installs
- Clean up old contrib area
- request another 100 GB of space
- update cron job
- Aux Telescope
- Wait to hear from Kirk
Week of Mar 12 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- nEXO
- Fix build of ROOT on centos7, missing libEve and libRGL
- Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- DM-DC2
- Update Nicolas' ImageProcessingPipelines branch for NERSC
- Demonstrate running SRS workflow for ingest
- NERSC
- Work out production installation of lsst_sims + w_2018_09 using the shared python env
- Work out production installation of lsst_sims + w_2018_09 using the shared python env
- SLAC
- Aux Telescope
Week of Mar 5 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- nEXO
- Attend sims meeting
- Work with SLAC IT to get all necessary system libraries installed on centos7 to build nEXO's externals
- libXpm-devel
- bzip2-devel
- Motif
- Motif-devel
- mesa-libGLU mesa-libGLU-devel
- Work through build problems with Coin3d 3.1.3 (an external lib)
https://github.com/nEXO-collaboration/nexo-env/issues/1 - Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- Create some initial dev doc in their wiki
- DM-DC2
- Attend and take notes for task force meeting
- Attend SSims meeting Thursday
- DESC
- Create docker image using new lsst_sims + w_2018_09
- NERSC
- Install new lsst_sims + w_2018_09
- Add note about lsst_distrib + lsst_sims on Confluence: NERSC Software Installations
- Install CCL At NERSC in the shared DESCQA python env
- GH Issue https://github.com/LSSTDESC/CCL/issues/346
- Updated CCL NERSC Wiki: https://github.com/LSSTDESC/CCL/wiki/Using-and-Installing-CCL-on-NERSC
- Work out production installation of lsst_sims + w_2018_09 using the shared python env
- SLAC
- ATS At Harvard
- Attend test stand tutorial 9 AM - 10 AM PT Wednesday
Week of Feb 26 2018
- Fermi
- get Brian to fully clean up between GR Jenkins builds - created JIRA
- get L1 GR to build
- figure out how to clean up workspaces in Jenkins on my own.
- Tag first GlastRelease release in github GlastRelease-20-10-04-gr03 only change is to the xxxRootData packages
- Figure out the release process
- nEXO
- send an email update to Lisa
- update Slack channel about Raymond's discussions with SNiPER developers
- Work on HistogramSvc for ROOT files
- DM-DC2
- Run Monday Meeting and write minutes
- Update GitHub issues
- Contact Chris W about flats
- Talk to Tom about setting up directory for transfer to IN2P3
- test ingestDriver.py at NERSC
https://github.com/LSSTDESC/ImageProcessingPipelines/issues/6 - Update IPP repo with new mini-DRP Epic
https://app.zenhub.com/workspace/o/lsstdesc/imageprocessingpipelines/boards?repos=112563170 - Create new SRS pipeline task to run ingest on eImages
http://srs.slac.stanford.edu/Pipeline-II/exp/LSST-DESC/task.jsp?task=50591002 - Update SRS pipeline task to handle ref cat ingest
- Update SRS pipeline task to run processEimage
- attend DC2 meeting
- NERSC
- Poke NERSC about jupyter-dev outage - now fixed
- SLAC
- ATS At Harvard
Week of Feb 19 2018
- Fermi
- Finish Anaconda 3 install
- Fix up L1 builds on Jenkins
- Fermi Meeting
- nEXO
- Start new HistogramSvc for ROOT files
- Simulation Meeting Thursday
- DM-DC2
- First Task Force Meeting
- Migrate DC2 Cookbook issues to IPP (ImageProcessingPipeline) repo
- Create new issues in IPP repo
- NERSC
- sort out HPSS allocation
- SLAC
- Help David L access his account
- ATS at Harvard
- Meeting with Tony & Owen on Tuesday
- Go to Harvard on Friday
Week of Feb 12 2018
- Fermi
- attend offline meetings!
- Finish Anaconda3 installation
- Work out distribution of externals in the new regime
- Install swig 3.0.12 RH6 and RH7 on GLAST_EXT
- work out L1 build in Jenkins
- nEXO
- Meet with sims people
- NERSC Support
- re-re-install cosmosis to help out the WL folks who wanted the latest along the develop branch
- Reach out to Joe to encourage tagging of cosmosis and provide some feedback on their setup script for NERSC
- Reach out to wl pipeline group to ask about cosmosis installations.
- Start up automated weekly stack installations
- Document weekly stack installations
- Send Joe Z updates to the setup-cosmosis-nersc script
- Help Erfan get going with cosmosis at NERSC
- SLAC support
- Read over Joanne's docs
- Pow-wow with Joanne over the DP and eT and the eT DB
- set up docker for Jim on IR2
- Update DESC and Fermi documentation on getting a SLAC account
- DRP/L2 Processing
- Task Force
- Phil mentioned that "they" had to write up a charge - figure out what that means
- Modify charge after receiving suggestions
- At Nicolas' suggestion reach out to Fabio H concerning weekly stack installations
- Phil mentioned that "they" had to write up a charge - figure out what that means
- Task Force
- ATS at Harvard
- Attend CCS meeting to find out how best to support Harvard
- Contact Harvard to sort out list of devices & plans for their use for testing
- Meet with Kirk
- write up minutes from Kirk/Nick meeting and prepare to iterate with Kirk
Week of Feb 5 2018
- DRP/L2 Processing
- SLAC DESC support
- NERSC support
- set up automated installs : https://github.com/lsst-dm/shared-stack
- Help Alex set up web data access: http://www.nersc.gov/users/data-analytics/science-gateways/
https://github.com/pmelchior/des-exp-checker
- Other
- Read over Joanne's docs: http://corner.readthedocs.io/en/latest/install.html
- Write up IN2P3 account Confluence page
- Update DMstack install on Harvard AUX Telescope machine
- nexo
- Attend Sims meeting
- Schedule time to meet Lisa next week
- Try out Raymond's example using SLAC install
- finish SLAC installation
- Fermi
- Finish Anaconda 3 installation at SLAC
- work out distribution of externals for end users
Week of Jan 29nd 2018
- Data Portal
- support Joanne as she learns about the plotting
- DRP/L2 Processing
- Check on James P
- Get a copy of the DC1 WF going
- Update WF to run Jim's latest imSim Data
- Learn how to deal with pipeline issue in JMX dealing with large number of jobs stuck in submitted state
- Run Jim's imSim data through SLAC WF (first attempt in progress)
- Prep to run using Phosim 1.1p
- Document using the Slac WF and add scripts to a GH repo in LSSTDESC area
- attend Friday dc2 meeting
- SLAC support
- review, release, and install the latest version of datacat-utilities
- Read over Joanne's documentation
- -
automate weekly install of DMstack binaries for camera- - install datacat-utilities on lsst-dc10
- NERSC support
- Upload latest SQRE DMstack (w_2018_03) Docker image into Shifter
- Install w_2018_03 plus sims from source
- Create w_2018_03+sims Shifter Image (debugging issue with afw - issue identified.. fixing it is something else)
- create sims Shifter image the long way, without using lsstsqre's base image
- Install WeakLensingDeblending for Javier.
- Random LSST Camera and DESC items
- SSim meeting
- Meet with Wil on Friday
- Write up notes from meeting
- Look over DESC architects confluence page
- nEXO
- reinstall SNiPER in the new disk space
- try out examples
- sign up for the collaboration mailing list
- Fermi
- Investigate trouble with AcdDigi in L1 branch
- Find a way to upgrade Anaconda external for ST
- Install miniconda py3
- Contact Helen
Week of Jan 22nd 2018
- Data Portal
- Consider rerunning all the ITL sensors in need of updated MET ingest and MET 05 by hand
- plots - commit latest code and write up intro for Joanne
- CCB request and deploy MET updates for the DP
- Deploy DP 0.5.11
- SLAC support
- automate weekly install of DMstack binaries for camera (Jim doesn't seem to feel this is a priority)
- Read over Joanne's documentation
- NERSC support
- update DESCQA install to include tensor flow
- write up instructions for w_2018_01 shifter install
- automate installation of shifter images
- Create docker image of new sims release based on w_2018_04
- Reintall cosmosis for WL group due to updated FFTW module, trying cray-fftw this time
- Send instructions to Josh to set up his own test galsim using the extlibs we already have set up for Cori KNL gcc
- install Phosim 3.7.7 and 3.7.8 for knl-intel and haswell-intel
- meet with Mustafa about HPSS
- Fix Cosmosis installation
- Workflows/L2 DC2 Pipeline
- send plea to Tony
- Meeting with Tom about using the SLAC pipeline + pilots
- work out setting up test Pipeline2 job reusing DC1 L2 task and a handful of visits
- Prep for CI meeting
- Fermi
- Work through errors with AcdDigi on L1 branch
- Set up new Anaconda external
- Work out with Brian the requirements for setting up externals these days
- nEXO
- get Tony an disk space estimate for SLAC installs
- get some disk space from Tony at SLAC
- set up SNiPER at SLAC in a temp location (in progress)
- attend Tuesday sims meeting
Week of Jan 15th 2018
- Data Portal
- revisit plots
- Holding off MET updates until new traveler to reingest ITL sensors is available
- datacat-utilities
- Install new release 0.0.9 at SLAC
- Install new release 0.0.9 at SLAC
- SLAC support
- automate weekly installs of DMstack binaries for camera
- automate weekly installs of DMstack binaries for camera
- NERSC Support
- update DESCQA install to include tensorflow
- Write up instructions for w_2018_01 shifter install
- automate installation of shifter images
- Add WeakLensingDeblending
- Check on CCL installation
- Workflows/L2 DC2 Pipeline
- DC2 meeting Friday Jan 19th
- Slim down docker image which includes sims - now down to 3 GB on DockerHub https://hub.docker.com/r/heather999/test/tags/
- Slim down docker image which includes sims - now down to 3 GB on DockerHub https://hub.docker.com/r/heather999/test/tags/
- Fermi
- Set up draft f2f meeting page Fermi Software Week Winter 2018
- Continue working on GR L1 branch - completed checkout of L1 code successfully
- Work through errors with AcdDigi on L1 branch
- nEXO
- Read over nEXO at OSG simulation notes https://ntpc.ucllnl.org/nEXO/index.php/Simulations_on_the_OSG
- Read over nEXO at OSG simulation notes https://ntpc.ucllnl.org/nEXO/index.php/Simulations_on_the_OSG
Week of Jan 8th 2018
- DataPortal
- finish off ITL-3800C-368 ingest after sorting out http authentication errors
- revisit plots
- Hold off on MET updates until new traveler to reingest ITL sensors is available.
- datacat-utilities
- Install new release
- SLAC suport
- Install w_2018_01 binaries lsst_distrib for camera
- Work out automated weekly installs via cron (u1 is 97% full, ~990 GB left)
- Install Merlin's eotest branch
- NERSC Support
- Update DESCQA install to include tensorflow
- Try w_2018_01 shifter install (in progress)
- Testing w_2018_01 shifter install and write up instructions
- Add WeakLensingDeblending https://lsstc.slack.com/archives/G8RU8C4VD/p1515604937000489
- Workflows/L2 DC2 pipeline
- Set up lsst_sims docker image for NERSC & uploaded to Dockerhub
- Try another lsst_sims docker image using lsstsqre image as the base rather than centos7 (initially failed, have new plans to retry)
- Fermi
Attend Wednesday software meeting- Finish setting up GR L1 branch https://github.com/fermi-lat/GlastRelease
- Test L1 build in Jenkins https://fermi-lat.github.io/doc/workflow.html
- Try interactive build of GR L1 branch
- nEXO & SNiPER
- Attend Tuesday software meeting
- Contact SNiPER team about Open Source License
- Apply for OSG and LLNL accounts
- Read over nEXO at OSG simulation notes https://ntpc.ucllnl.org/nEXO/index.php/Simulations_on_the_OSG
Week of Dec 18th 2017
- DataPortal
- Workflows/L2 DC2 pipeline
- Reproduce error James P sees
- Repeat experiment with multiple visits
- Repeat experiment using Head of ImageProcessingPipeline repo
- Potentially meet with Jim & James this week
- Catch up on Parsl progress
- CI
- Set up new Docker image to use with imSIm's travis builds
- Create a smaller Docker images for use with travis
- NERSC Support
- Shared Stack
- JupyterHub documentation and simplification
- clean old project space by removing MBII-snapshots
- Request reduction in old project space quota to 40 TB
- imSim
- Fermi
- Set up GR L1 branch in GitHub
- Test build of L1 branch
- SNiPER
- Read up on SNiPER
- Meet with Lisa K on Tuesday 5 PM Eastern
- send feedback on documentation
- set up user accounts on Wiki, LLNL, OSG
- talk to Tony about nExo
Week of Dec 11th 2017
- Data Portal
- Deploy ETCCB-429
- MET updates - almost complete
- CCS ATS
- Install mvn and associated libraries and get Nick going with Lakeshore driver
- Install mvn and associated libraries and get Nick going with Lakeshore driver
- Workflows/L2 DC2 pipeline
- Send James P a link to the recordings from last week
- Repeat test using Jim's parsl wf at NERSC
- CI
- Docker image for use in travis builds
- Docker image for use in travis builds
- imSIM
- NERSC Support
- Fresh Shared Stack installation for KNL - stuck working out disk space in the common software area – started
- dmtcp installation
- Update jupiter-dev documentation
- quota checker
- Fresh Shared Stack installation for KNL - stuck working out disk space in the common software area – started
- Fermi
- Try GR with repoman
- Documentation
Week of Nov 27th 2017
- L2 pipeline DC1 rerun
- Finish off write up of chat with Tony
- Ask James P about remote participation during Spring Week
M & W 8 - 12 central time - Try a run using Simon's cookbook
- Meet with Jim & James P
- write up notes
- Try Parsl pipeline at NERSC
- NERSC software installations
- Shared Stack installation - verify that the DESCQA and py3 install is working
- Add CCL
- Install new sims 2.4.2 using DMstack w.2017.46 Haswell-gcc6 using py3
- Ask about cosmosis needs
- Install cosmosis, TreeCorr, and MLZ in projecta area under the WL WG directory
- Install phosim 3.7.3
- dmtcp
- quota checker
- Consider requesting more common software space
obtained 50 GB more
- Shared Stack installation - verify that the DESCQA and py3 install is working
- NERSC project space
- Verify MBII-snapshots is in HPSS and remove from project area
Found the content in the desc hsi area, and viewed the contents of some snap.tar via htar -tf CS/MBII-DMO-snapshots/snapdir_074.tar Also noticed that the group_xxx.tar files are duplicated in the Archive home directory. Contacted Mustafa to sort that out and remove duplicates.
- Verify MBII-snapshots is in HPSS and remove from project area
- Data Portal
- MET updates
- Meet with Anders and Seth
- MET updates
- Harvard ATS
- Check that ccs reboot restarts the MySQL, LocalRestServices, and Trending
- Check in with Nick about the Lakeshore
- Fermi
- Try GR using repoman
- Documentation
- Discuss Gaudi for nEXO
https://gaudi.web.cern.ch/gaudi/releases/v30r0/
http://lcgsoft.web.cern.ch/lcgsoft/release/
https://gitlab.cern.ch/gaudi/Gaudi - Attend Stanford IT Unconference
https://itunconference.stanford.edu/sessions/remote
Week of Nov 20th 2017
- L2 pipeline DC1 rerun
- NERSC software installations
- py3 shared stack installation
- finish phosim 3.7.2 installation
- dmtcp intel installation
- quota checker
- cosmosis installation
- check in on GH issue https://github.com/LSSTDESC/WLPipe/issues/31
- Cover Vendor Ingests M-W
- Data Portal
- plots
- check in with Tony about his changes to load the page more quickly
- Harvard ATS
- Determine who will deal with the Lakeshore drivers
https://jira.slac.stanford.edu/projects/LSSTCCSDRIVER/issues/LSSTCCSDRIVER-227
https://github.com/lsst-camera-ccs/org-lsst-ccs-drivers-lakeshore/tree/LSSTCCSDRIVER-227
https://www.lakeshore.com/Documents/336_Manual.pdf - Grab Owens updates and install on CCS machine
- Direct Nick to make updates as needed.
- Fix mounting of data drive on ccs machine
- Try a version of the TS subsystem on the ccs machine and set up groovy & properties files
- Check monochromator plays well with the TS subsystem
- Determine who will deal with the Lakeshore drivers
- Fermi
- Try a GR build with repoman
- Documentation
- Inquire about pointlike updates in CVS on Slack
Week of Nov 13th 2017
- Harvard ATS
- Create user accounts and ccs sudo account which grants sudo access to all users
- Set up directory structure following IR2 model
- Install DMstack v14
- Install eotest compatible with v14
- document the above on the Confluence page ATS Test Stands
- Set up Trending Database
- MariaDB installation
- L2 pipeline DC1 rerun https://github.com/LSSTDESC/SSim_DC1/issues/47
- Meet with Jim on Monday
- Write up Notes
- Contact James P
- Try to run Simon's Cookbook
- Chat with Tony
- Meet with Jim on Monday
- NERSC Software Installations
- Shared Stack
- first try failed due to conflicts with libiconv installed presumably for
- Phosim 3.7.2 bug fix for haswell gcc & intel, KNL gcc & intel
- DMTCP intel build
- Help out with cosmosis installation https://github.com/LSSTDESC/WLPipe/issues/31
- Shared Stack
- Data Portal
- Chat with Tony about plots
- Plots
- Bad Channels
- read noise
- NERSC Project Space
- Test MBII-snapshot HPSS copy and remove from project area
Week of Nov 6th 2017
- QA Testing
- Investigate cause of differences between Lauren's plots from NCSA and those generated at NERSC
- look at 'conda list' on both NCSA and NERSC (no differences discovered)
- generate cal_refs as was done at NCSA
- try gcc 6 build as is being done at NCSA
- Determine the cause of differences between ref catalogs at NCSA and NERSC
- Investigate cause of differences between Lauren's plots from NCSA and those generated at NERSC
- L2 Pipeline
- Data Portal
- Finish off first round of Richard's plots
- bad channels
- read_noise
- Finish off first round of Richard's plots
- NERSC Project Space
- test MBII-snapshot HPSS copy
- ATS
- prep for Tony & Owen's visit
- Installations
- SLAC
- Py3 on SLAC RH6-64 for Joanne to test out
- NERSC
- finish testing NERSC's cfitsio installation
- install phosim 3.7.1 for Cori haswell gcc & intel, KNL gcc & intel
- ask Glenn to install module files
- install phosim 3.7.2 for Cori haswell gcc & intel, KNL gcc & intel
- Shared Stack Installation using w.2017.41
- Query DESCQA and CCL groups
- py2 w.2017.41 installation
- py3 w.2017.41 installation
- SLAC
Week of Oct 30th 2017
- Data Portal
- Plots
- DC2 prep
- QA testing
- w.2017.41 installation plus branches of obs_lsstSim and pipe_analysis
- Run visitAnalysis, coaddAnalysis, etc
- QA testing
- NERSC Workflow work
- NERSC Project Space
- Work with Mustafa to get MBII-snapshot backed up
- NERSC Installations
- w.2017.? stack installation plus sims - waiting on new sims release using the camera geometry updates
- Phosim 3.7
- Testing NERSC cfitsio installation
- SLAC Installations
- descsoft w.2017.? stack plus sims installation
- CCS
- See if there is any prep work necessary for Tony's visit later in Nov
- Request Nick set up new box with CentOS7
- set up user accounts and groups
- Determine if Mike is still coming out ~Nov 1st
- See if there is any prep work necessary for Tony's visit later in Nov
- Attend SuperCDMS meeting concerning their potential move to NERSC.
Week of Oct 23rd 2017
- Data Portal
- NERSC Workflow work
- Try out HTCondor and Pegasus on cori21
- Take a look at Jim's imsim script
- NERSC Project Space
- Set the old project directory to read-only (in progress)
- claim desc ownership of the whole directory space
- Clear out all directories where the owners have indicated it is fine to do so
- Determine the end state of merging old project and projecta areas and handling of www
- Set up disk quota monitor
- Coordinate HPSS storage of MBII-snapshots
- Set the old project directory to read-only (in progress)
- NERSC installations
- Phosim v3.7
- Coordinate with NERSC to get cfitsio v3410 installed (in progress)
- Rebuild using NERSC supplied cfitsio
- Coordinate with Glenn to set up module files
- Set up software for DC2 testing
- v14_0 lsst_distrib + sims 2.4.0
- Phosim v3.7
- SLAC installations
- Set up same imsim at SLAC as used at NERSC for KNL testing (in progress)
- v14_0 on RH6-64 public (on hold pending resolution to obs_base testCameraMapper.py unit test failure)
- camera area
- descsoft
- test eotest branch
- v14_0 on IR-2
- Document KNL vs Haskell mkl imsim testing at NERSC
- Initial mkl tests indicate no huge improvement - putting testing on back burner for now
- CCS - still waiting to hear from Peter, I cannot access their TS3 machine to install and test a new version of the test stand subsystem
Week of Oct 16th 2017
- Richard's plots in DP
- Complete Sensor Arrival Time plot
- Create JSON
- NERSC installations
- Phosim v3.7
- Try out NERSC supplied FFTW
- Request new cfitsio v3410 installation required by Phosim
- Complete installation of all flavors: haswell-gcc, haswell-intel, knl-gcc, knl-intel
- Waiting on cfitsio v3410 before knl-intel will be available
- Phosim v3.7
- NERSC Project Space
- Notify all users of the need to migrate
- Handle all help requests - migrating some directories to the new projecta area
- Install disk quota monitor
- Check on old project status and prepare to set it to read-only
- Coordinate backup of nkhandai data to tape with Mustafa
- Install latest version of teststand subsystem at Harvard there is a machine made available - waiting on response from Peter
- Keep an eye out for v14 DMstack
Week of October 9th 2017
- Continue work on Richard's plots in DP
-
-
LSSTTD-1165Getting issue details...
STATUS
- Released Data Portal 0.5.7
- Testing imSim built on KNL
- Continue MKL tests at NERSC once the system recovers from the weekend outage
- Install the same version of the stack at SLAC to allow similar testing with and without MKL
- Attempt build of DMstack using the intel compiler at NERSC
- NERSC Project space
- Install disk quota monitor
- Coordinate backup of nkhandai data to tape with Mustafa
- Prepare to set the old project area as read-only
- Send mail to all owners of files on the old project space
- NERSC HPSS SRU overallocation
- Install latest version of datacat-utilities 0.0.8 at SLAC
- Install the latest teststand snapshot at Harvard to support use of the monochromator when Peter returns.
- Contacted Peter - I can't connect to the archon machine to install the test stand snapshot
- Investigate creating a harvard.groovy (perhaps based on TS8bench or TS7) and possibly create a harvard main routine.
- Keep an eye out for the v14 DMstack release and prepare to install on IR-2 and NERSC
- This release will include binary installations.. I'm thinking we may still want to install from source, perhaps that's most useful on IR-2, but we'll see.
- This release will include binary installations.. I'm thinking we may still want to install from source, perhaps that's most useful on IR-2, but we'll see.
Week of October 2nd 2017
- Release DataPortal 0.5.6 which will include
- Implement Richard's plots in the DP starting from some of Tony's examples for trending and Richard's python code.
- Continue testing KNL performance using galsim and imsim
- Attempt to compile galsim and possibly all of DMstack with the intel compiler
- Run longer tests using the MKL libraries on Haswell and KNL
- Rebuild imsim with MKL, using an updated meas_base
- Install an initial shared stack in the new software area at NERSC for testing as discussed at last week's "Shared Stack" meeting
- NERSC Project Space
- Set up an initial quota monitor at NERSC
- Re-connect with Mustafa to get the mkhandai 42 TB archived to tape.
- Investigate an upgrade of the monochromator driver at Harvard and confirm that to achieve what Peter plans to do - to run his ATS work in the lab alongside most of the devices from his TS3 setup.. is possible by running both the teststand and rafts subsystem simultaneously.
- Install latest SNAPSHOT of test stand at Harvard and investigate creating a harvard.groovy (perhaps based on TS8bench or TS7) and possibly creating a harvard main routine.
- Keep an eye out for the v14 release of the DMstack and prepare to install at SLAC/IR-2 and NERSC.
Though this is assigned to Jim, so perhaps I have no need to worry about SLAC:
- LSSTTD-1157Getting issue details... STATUS