You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 270
Next »
Week of Mar 19 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- nEXO
- Fix build of ROOT on centos7, missing libEve and libRGL
- Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- DM-DC2
- DM-DC2 Meeting prep
- Write up Live notes
- SLAC
- Implement and deploy new run search box
- NERSC
- Aux Telescope
Week of Mar 12 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- nEXO
- Fix build of ROOT on centos7, missing libEve and libRGL
- Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- DM-DC2
- Update Nicolas' ImageProcessingPipelines branch for NERSC
- Demonstrate running SRS workflow for ingest
- NERSC
- Work out production installation of lsst_sims + w_2018_09 using the shared python env
- SLAC
- Create some documentation to use jupyterlab at SLAC
- Update Fermi and DESC SLAC Account instructions - maybe nEXO's too
-
LSSTTD-1256
-
Getting issue details...
STATUS
- Aux Telescope
Week of Mar 5 2018
- Fermi
- Finish off creating L1 branches for all GR packages
- Get Brian to fully clean up between GR jenkins builds - see JIRA
- Figure out release process for L1
- Look at documentation: old Workbook, Github, Confluence..
- nEXO
- Attend sims meeting
- Work with SLAC IT to get all necessary system libraries installed on centos7 to build nEXO's externals
- libXpm-devel
- bzip2-devel
- Motif
- Motif-devel
- mesa-libGLU mesa-libGLU-devel
- Work through build problems with Coin3d 3.1.3 (an external lib)
https://github.com/nEXO-collaboration/nexo-env/issues/1 - Finish building nexo SNiPER on centos7
- Work on HistogramSvc
- Create some initial dev doc in their wiki
- DM-DC2
- Attend and take notes for task force meeting
- Attend SSims meeting Thursday
- DESC
- Create docker image using new lsst_sims + w_2018_09
- NERSC
- Install new lsst_sims + w_2018_09
- Add note about lsst_distrib + lsst_sims on Confluence: NERSC Software Installations
- Install CCL At NERSC in the shared DESCQA python env
- Work out production installation of lsst_sims + w_2018_09 using the shared python env
- SLAC
- Continue watching jupyterlab progress
- Create some documentation to use jupyterlab at SLAC
- Update Fermi and DESC SLAC Account instructions - maybe nEXO's too
-
LSSTTD-1256
-
Getting issue details...
STATUS
- ATS At Harvard
- Attend test stand tutorial 9 AM - 10 AM PT Wednesday
Week of Feb 26 2018
- Fermi
- get Brian to fully clean up between GR Jenkins builds - created JIRA
- get L1 GR to build
- figure out how to clean up workspaces in Jenkins on my own.
- Tag first GlastRelease release in github GlastRelease-20-10-04-gr03 only change is to the xxxRootData packages
- Figure out the release process
- nEXO
- send an email update to Lisa
- update Slack channel about Raymond's discussions with SNiPER developers
- Work on HistogramSvc for ROOT files
- DM-DC2
- NERSC
- Poke NERSC about jupyter-dev outage - now fixed
- SLAC
- Watch JupyterLab demo install and keep Steve informed
- Try out eT DB updates to prep for Joanne's departure
- Update DESC and Fermi documentation on getting a SLAC account
- Check on Tom M's reported problem with the status of some CCD-011 bits in the DP
- ATS At Harvard
Week of Feb 19 2018
- Fermi
- Finish Anaconda 3 install
- Fix up L1 builds on Jenkins
- Fermi Meeting
- nEXO
- Start new HistogramSvc for ROOT files
- Simulation Meeting Thursday
- DM-DC2
- First Task Force Meeting
- Migrate DC2 Cookbook issues to IPP (ImageProcessingPipeline) repo
- Create new issues in IPP repo
- NERSC
- SLAC
- Help David L access his account
- ATS at Harvard
- Meeting with Tony & Owen on Tuesday
- Go to Harvard on Friday
Week of Feb 12 2018
- Fermi
- attend offline meetings!
- Finish Anaconda3 installation
- Work out distribution of externals in the new regime
- Install swig 3.0.12 RH6 and RH7 on GLAST_EXT
- work out L1 build in Jenkins
- nEXO
- NERSC Support
- re-re-install cosmosis to help out the WL folks who wanted the latest along the develop branch
- Reach out to Joe to encourage tagging of cosmosis and provide some feedback on their setup script for NERSC
- Reach out to wl pipeline group to ask about cosmosis installations.
- Start up automated weekly stack installations
- Document weekly stack installations
- Send Joe Z updates to the setup-cosmosis-nersc script
- Help Erfan get going with cosmosis at NERSC
- SLAC support
- Read over Joanne's docs
- Pow-wow with Joanne over the DP and eT and the eT DB
- set up docker for Jim on IR2
- Update DESC and Fermi documentation on getting a SLAC account
- DRP/L2 Processing
- Task Force
- Phil mentioned that "they" had to write up a charge - figure out what that means
- Modify charge after receiving suggestions
- At Nicolas' suggestion reach out to Fabio H concerning weekly stack installations
- ATS at Harvard
- Attend CCS meeting to find out how best to support Harvard
- Contact Harvard to sort out list of devices & plans for their use for testing
- Meet with Kirk
- write up minutes from Kirk/Nick meeting and prepare to iterate with Kirk
Week of Feb 5 2018
- DRP/L2 Processing
- SLAC DESC support
- NERSC support
- Other
- nexo
- Attend Sims meeting
- Schedule time to meet Lisa next week
- Try out Raymond's example using SLAC install
- finish SLAC installation
- Fermi
- Finish Anaconda 3 installation at SLAC
- work out distribution of externals for end users
Week of Jan 29nd 2018
- Data Portal
- support Joanne as she learns about the plotting
- DRP/L2 Processing
- Check on James P
- Get a copy of the DC1 WF going
- Update WF to run Jim's latest imSim Data
- Learn how to deal with pipeline issue in JMX dealing with large number of jobs stuck in submitted state
- Run Jim's imSim data through SLAC WF (first attempt in progress)
- Prep to run using Phosim 1.1p
- Document using the Slac WF and add scripts to a GH repo in LSSTDESC area
- attend Friday dc2 meeting
- SLAC support
- review, release, and install the latest version of datacat-utilities
- Read over Joanne's documentation
- -
automate weekly install of DMstack binaries for camera- - install datacat-utilities on lsst-dc10
- NERSC support
- Upload latest SQRE DMstack (w_2018_03) Docker image into Shifter
- Install w_2018_03 plus sims from source
- Create w_2018_03+sims Shifter Image (debugging issue with afw - issue identified.. fixing it is something else)
- create sims Shifter image the long way, without using lsstsqre's base image
- Install WeakLensingDeblending for Javier.
- Random LSST Camera and DESC items
- SSim meeting
- Meet with Wil on Friday
- Write up notes from meeting
- Look over DESC architects confluence page
- nEXO
- reinstall SNiPER in the new disk space
- try out examples
- sign up for the collaboration mailing list
- Fermi
- Investigate trouble with AcdDigi in L1 branch
- Find a way to upgrade Anaconda external for ST
- Install miniconda py3
- Contact Helen
Week of Jan 22nd 2018
- Data Portal
- Consider rerunning all the ITL sensors in need of updated MET ingest and MET 05 by hand
- plots - commit latest code and write up intro for Joanne
- CCB request and deploy MET updates for the DP
- Deploy DP 0.5.11
- SLAC support
- automate weekly install of DMstack binaries for camera (Jim doesn't seem to feel this is a priority)
- Read over Joanne's documentation
- NERSC support
- update DESCQA install to include tensor flow
- write up instructions for w_2018_01 shifter install
- automate installation of shifter images
- Create docker image of new sims release based on w_2018_04
- Reintall cosmosis for WL group due to updated FFTW module, trying cray-fftw this time
- Send instructions to Josh to set up his own test galsim using the extlibs we already have set up for Cori KNL gcc
- install Phosim 3.7.7 and 3.7.8 for knl-intel and haswell-intel
- meet with Mustafa about HPSS
- Fix Cosmosis installation
- Workflows/L2 DC2 Pipeline
- send plea to Tony
- Meeting with Tom about using the SLAC pipeline + pilots
- work out setting up test Pipeline2 job reusing DC1 L2 task and a handful of visits
- Prep for CI meeting
- Fermi
- Work through errors with AcdDigi on L1 branch
- Set up new Anaconda external
- Work out with Brian the requirements for setting up externals these days
- nEXO
- get Tony an disk space estimate for SLAC installs
- get some disk space from Tony at SLAC
- set up SNiPER at SLAC in a temp location (in progress)
- attend Tuesday sims meeting
Week of Jan 15th 2018
- Data Portal
- revisit plots
- Holding off MET updates until new traveler to reingest ITL sensors is available
- datacat-utilities
- Install new release 0.0.9 at SLAC
- SLAC support
- automate weekly installs of DMstack binaries for camera
- NERSC Support
- update DESCQA install to include tensorflow
- Write up instructions for w_2018_01 shifter install
- automate installation of shifter images
- Add WeakLensingDeblending
- Check on CCL installation
- Workflows/L2 DC2 Pipeline
- Fermi
- Set up draft f2f meeting page Fermi Software Week Winter 2018
- Continue working on GR L1 branch - completed checkout of L1 code successfully
- Work through errors with AcdDigi on L1 branch
- nEXO
Week of Jan 8th 2018
- DataPortal
- finish off ITL-3800C-368 ingest after sorting out http authentication errors
- revisit plots
- Hold off on MET updates until new traveler to reingest ITL sensors is available.
- datacat-utilities
- SLAC suport
- Install w_2018_01 binaries lsst_distrib for camera
- Work out automated weekly installs via cron (u1 is 97% full, ~990 GB left)
- Install Merlin's eotest branch
- NERSC Support
- Workflows/L2 DC2 pipeline
- Set up lsst_sims docker image for NERSC & uploaded to Dockerhub
- Try another lsst_sims docker image using lsstsqre image as the base rather than centos7 (initially failed, have new plans to retry)
- Fermi
- nEXO & SNiPER
Week of Dec 18th 2017
- DataPortal
- Workflows/L2 DC2 pipeline
- Reproduce error James P sees
- Repeat experiment with multiple visits
- Repeat experiment using Head of ImageProcessingPipeline repo
- Potentially meet with Jim & James this week
- Catch up on Parsl progress
- CI
- Set up new Docker image to use with imSIm's travis builds
- Create a smaller Docker images for use with travis
- NERSC Support
- Shared Stack
- JupyterHub documentation and simplification
- clean old project space by removing MBII-snapshots
- Request reduction in old project space quota to 40 TB
- imSim
- Fermi
- Set up GR L1 branch in GitHub
- Test build of L1 branch
- SNiPER
- Read up on SNiPER
- Meet with Lisa K on Tuesday 5 PM Eastern
- send feedback on documentation
- set up user accounts on Wiki, LLNL, OSG
- talk to Tony about nExo
Week of Dec 11th 2017
- Data Portal
- Deploy ETCCB-429
- MET updates - almost complete
- CCS ATS
- Install mvn and associated libraries and get Nick going with Lakeshore driver
- Workflows/L2 DC2 pipeline
- Send James P a link to the recordings from last week
- Repeat test using Jim's parsl wf at NERSC
- CI
- Docker image for use in travis builds
- imSIM
- NERSC Support
- Fresh Shared Stack installation for KNL - stuck working out disk space in the common software area – started
- dmtcp installation
- Update jupiter-dev documentation
- quota checker
- Fermi
- Try GR with repoman
- Documentation
Week of Nov 27th 2017
- L2 pipeline DC1 rerun
- Finish off write up of chat with Tony
- Ask James P about remote participation during Spring Week
M & W 8 - 12 central time - Try a run using Simon's cookbook
- Meet with Jim & James P
- Try Parsl pipeline at NERSC
- NERSC software installations
- Shared Stack installation - verify that the DESCQA and py3 install is working
- Install new sims 2.4.2 using DMstack w.2017.46 Haswell-gcc6 using py3
- Ask about cosmosis needs
- Install cosmosis, TreeCorr, and MLZ in projecta area under the WL WG directory
- Install phosim 3.7.3
- dmtcp
- quota checker
- Consider requesting more common software space
obtained 50 GB more
- NERSC project space
- Verify MBII-snapshots is in HPSS and remove from project area
Found the content in the desc hsi area, and viewed the contents of some snap.tar via htar -tf CS/MBII-DMO-snapshots/snapdir_074.tar Also noticed that the group_xxx.tar files are duplicated in the Archive home directory. Contacted Mustafa to sort that out and remove duplicates.
- Data Portal
- MET updates
- Meet with Anders and Seth
- Harvard ATS
- Check that ccs reboot restarts the MySQL, LocalRestServices, and Trending
- Check in with Nick about the Lakeshore
- Fermi
- Try GR using repoman
- Documentation
- Discuss Gaudi for nEXO
https://gaudi.web.cern.ch/gaudi/releases/v30r0/
http://lcgsoft.web.cern.ch/lcgsoft/release/
https://gitlab.cern.ch/gaudi/Gaudi - Attend Stanford IT Unconference
https://itunconference.stanford.edu/sessions/remote
Week of Nov 20th 2017
- L2 pipeline DC1 rerun
- NERSC software installations
- py3 shared stack installation
- finish phosim 3.7.2 installation
- dmtcp intel installation
- quota checker
- cosmosis installation
- Cover Vendor Ingests M-W
- Data Portal
- plots
- check in with Tony about his changes to load the page more quickly
- Harvard ATS
- Fermi
- Try a GR build with repoman
- Documentation
- Inquire about pointlike updates in CVS on Slack
Week of Nov 13th 2017
- Harvard ATS
- Create user accounts and ccs sudo account which grants sudo access to all users
- Set up directory structure following IR2 model
- Install DMstack v14
- Install eotest compatible with v14
- document the above on the Confluence page ATS Test Stands
- Set up Trending Database
- L2 pipeline DC1 rerun https://github.com/LSSTDESC/SSim_DC1/issues/47
- Meet with Jim on Monday
- Contact James P
- Try to run Simon's Cookbook
- Chat with Tony
- NERSC Software Installations
- Shared Stack
- first try failed due to conflicts with libiconv installed presumably for
- Phosim 3.7.2 bug fix for haswell gcc & intel, KNL gcc & intel
- DMTCP intel build
- Help out with cosmosis installation https://github.com/LSSTDESC/WLPipe/issues/31
- Data Portal
- Chat with Tony about plots
- Plots
- NERSC Project Space
- Test MBII-snapshot HPSS copy and remove from project area
Week of Nov 6th 2017
- QA Testing
- Investigate cause of differences between Lauren's plots from NCSA and those generated at NERSC
- look at 'conda list' on both NCSA and NERSC (no differences discovered)
- generate cal_refs as was done at NCSA
- try gcc 6 build as is being done at NCSA
- Determine the cause of differences between ref catalogs at NCSA and NERSC
- L2 Pipeline
- Data Portal
- Finish off first round of Richard's plots
- NERSC Project Space
- test MBII-snapshot HPSS copy
- ATS
- prep for Tony & Owen's visit
- Installations
- SLAC
- Py3 on SLAC RH6-64 for Joanne to test out
- NERSC
- finish testing NERSC's cfitsio installation
- install phosim 3.7.1 for Cori haswell gcc & intel, KNL gcc & intel
- ask Glenn to install module files
- install phosim 3.7.2 for Cori haswell gcc & intel, KNL gcc & intel
- Shared Stack Installation using w.2017.41
- Query DESCQA and CCL groups
- py2 w.2017.41 installation
- py3 w.2017.41 installation
Week of Oct 30th 2017
- Data Portal
- DC2 prep
- QA testing
- w.2017.41 installation plus branches of obs_lsstSim and pipe_analysis
- Run visitAnalysis, coaddAnalysis, etc
- NERSC Workflow work
- NERSC Project Space
- Work with Mustafa to get MBII-snapshot backed up
- NERSC Installations
- w.2017.? stack installation plus sims - waiting on new sims release using the camera geometry updates
- Phosim 3.7
- Testing NERSC cfitsio installation
- SLAC Installations
- descsoft w.2017.? stack plus sims installation
- CCS
- See if there is any prep work necessary for Tony's visit later in Nov
- Request Nick set up new box with CentOS7
- set up user accounts and groups
- Determine if Mike is still coming out ~Nov 1st
- Attend SuperCDMS meeting concerning their potential move to NERSC.
Week of Oct 23rd 2017
- Data Portal
- Richard's Plots - bad channels (in progress)
- Look at new Report Labels and determine what the DP still requires from the Reports Table
LSSTTD-1190
-
Getting issue details...
STATUS
- Make sure we request a restart of Tomcat
- NERSC Workflow work
- Try out HTCondor and Pegasus on cori21
- Take a look at Jim's imsim script
- NERSC Project Space
- Set the old project directory to read-only (in progress)
- claim desc ownership of the whole directory space
- Clear out all directories where the owners have indicated it is fine to do so
- Determine the end state of merging old project and projecta areas and handling of www
- Set up disk quota monitor
- Coordinate HPSS storage of MBII-snapshots
- NERSC installations
- Phosim v3.7
- Coordinate with NERSC to get cfitsio v3410 installed (in progress)
- Rebuild using NERSC supplied cfitsio
- Coordinate with Glenn to set up module files
- Set up software for DC2 testing
- v14_0 lsst_distrib + sims 2.4.0
- SLAC installations
- Set up same imsim at SLAC as used at NERSC for KNL testing (in progress)
- v14_0 on RH6-64 public (on hold pending resolution to obs_base testCameraMapper.py unit test failure)
- camera area
- descsoft
- test eotest branch
- v14_0 on IR-2
- Document KNL vs Haskell mkl imsim testing at NERSC
- Initial mkl tests indicate no huge improvement - putting testing on back burner for now
- CCS - still waiting to hear from Peter, I cannot access their TS3 machine to install and test a new version of the test stand subsystem
Week of Oct 16th 2017
- Richard's plots in DP
- Complete Sensor Arrival Time plot
- Create JSON
- NERSC installations
- Phosim v3.7
- Try out NERSC supplied FFTW
- Request new cfitsio v3410 installation required by Phosim
- Complete installation of all flavors: haswell-gcc, haswell-intel, knl-gcc, knl-intel
- Waiting on cfitsio v3410 before knl-intel will be available
- NERSC Project Space
- Notify all users of the need to migrate
- Handle all help requests - migrating some directories to the new projecta area
- Install disk quota monitor
- Check on old project status and prepare to set it to read-only
- Coordinate backup of nkhandai data to tape with Mustafa
- Install latest version of teststand subsystem at Harvard there is a machine made available - waiting on response from Peter
- Keep an eye out for v14 DMstack
Week of October 9th 2017
- Continue work on Richard's plots in DP
-
LSSTTD-1165
-
Getting issue details...
STATUS
- Released Data Portal 0.5.7
- Testing imSim built on KNL
- Continue MKL tests at NERSC once the system recovers from the weekend outage
- Install the same version of the stack at SLAC to allow similar testing with and without MKL
- Attempt build of DMstack using the intel compiler at NERSC
- NERSC Project space
- Install disk quota monitor
- Coordinate backup of nkhandai data to tape with Mustafa
- Prepare to set the old project area as read-only
- Send mail to all owners of files on the old project space
- NERSC HPSS SRU overallocation
- Install latest version of datacat-utilities 0.0.8 at SLAC
- Install the latest teststand snapshot at Harvard to support use of the monochromator when Peter returns.
- Contacted Peter - I can't connect to the archon machine to install the test stand snapshot
- Investigate creating a harvard.groovy (perhaps based on TS8bench or TS7) and possibly create a harvard main routine.
- Keep an eye out for the v14 DMstack release and prepare to install on IR-2 and NERSC
- This release will include binary installations.. I'm thinking we may still want to install from source, perhaps that's most useful on IR-2, but we'll see.
Week of October 2nd 2017
- Release DataPortal 0.5.6 which will include
- Addition of Grade and Procurement labels for the Sensor Acceptance pages
- RTM assignment
LSSTTD-1121
-
Getting issue details...
STATUS
- NCR "Ready to close"
LSSTTD-1144
-
Getting issue details...
STATUS
- Implement Richard's plots in the DP starting from some of Tony's examples for trending and Richard's python code.
- Continue testing KNL performance using galsim and imsim
- Attempt to compile galsim and possibly all of DMstack with the intel compiler
- Run longer tests using the MKL libraries on Haswell and KNL
- Rebuild imsim with MKL, using an updated meas_base
- Install an initial shared stack in the new software area at NERSC for testing as discussed at last week's "Shared Stack" meeting
- NERSC Project Space
- Set up an initial quota monitor at NERSC
- Re-connect with Mustafa to get the mkhandai 42 TB archived to tape.
- Investigate an upgrade of the monochromator driver at Harvard and confirm that to achieve what Peter plans to do - to run his ATS work in the lab alongside most of the devices from his TS3 setup.. is possible by running both the teststand and rafts subsystem simultaneously.
- Install latest SNAPSHOT of test stand at Harvard and investigate creating a harvard.groovy (perhaps based on TS8bench or TS7) and possibly creating a harvard main routine.
- Keep an eye out for the v14 release of the DMstack and prepare to install at SLAC/IR-2 and NERSC.
Though this is assigned to Jim, so perhaps I have no need to worry about SLAC:
LSSTTD-1157
-
Getting issue details...
STATUS
