Table of Contents

Introduction

The analysis release build system, SConsTools, provides a mechanism for integrating unit tests. Each package in the release, or a package that a user is developing, can have its own tests. All tests can be run by a simple command. Users may find this useful for testing their packages. For packages that psana developers add to the analysis release, these tests are automatically run during the nightly build. This page is primarily for psana developers, to go over how to add unit tests to test packages in the release during nightly builds. There are special considerations to make for tests that are part of the nightly build discussed below.

Creating a Package test directory

As an example, lets make a package with both a Python and a C++ unit test. For the example below, I am starting from the directory rel in my home directory. I make a new release, a new package in the release, and the crucial step is that I make a subdirectory called test in my package:

...

Other files in the test directory are ignored.

Python Unit Test

Add the file MyPkg/test/myfirst

...

Change the sys.exit(1) to sys.exit(0) and the test will succeed. You can also take out the sys.exit line, be default Python will return 0 after the script runs.

C++ Unit Test

A simple C++ test would like this, create the file MyPkg/test/mysecond.cpp

...

This test will also fail. Note, scons test stops after the first test fails. If you have not changed myfirst to return 0, only one of myfirst and mysecond will be run before failure is reported.

Using Frameworks for Unit Tests

It is worthwhile to learn how to use a testing framework. Psana developers are encouraged to use unittest for Python, and boost::unit_test for C++ in order to be consistent with existing tests in the release. However this is not necessary. You can use whatever framework you like.

Python unittest Framework

Below is an example of using unittest with Python. Add the file MyPkg/test/using_python_framework with the following content:

...

Refer to the documentation https://docs.python.org/2/library/unittest.html for more information on unittest.

Boost unit_test Framework

For an example of using the boost C++ unit test framework, create the file MyPkg/test/using_boost_framework.cpp with the contents (this is mostly copied from the boost website:

...

For more examples, one can look in the test subdirectories of packages like AppUtils, ConfigSvc, XtcInput, psana, psana_test, Translator

Nightly Build Considerations - External Test Data

There are several things psana developers need to consider when writing tests for packages that are part of the analysis release that will be run as part of the nightly build. This mostly involves how to work with external test data files.

...

• Check it in as part of your package
• Make a copy of it in a place accessible to psrel running on the psdev machines

Test Data Checked into the Package

We do not want to keep large amounts of data under version control. I think 10 kilobytes or so is Ok, but when it gets larger one should use the external location discussed below. Files that you do check in could go right in the test directory alongisde the test, or you can create new directories for test data. Another standard directory in the SConsTools system is data, which is fine, but it is intended for package data as opposed to testing data. One could also create a subdirectory to the test directory, such as

...

    def test_mytest(self): 
        text = file('MyPkg/test/fixtures/myfile.txt','r').read()
        self.assertTrue(text.startswith('file text'))

External Test Data Location

For xtc files, we have a directory,

...

Once you have prepared some test data, you can either add it to the Translator subdirectory, or the multifile subdirectory, or create a new sub directory, maybe with your package name (like I did when I made the Translator subdirectory). If you want to add it to Translator or multifile, please contact me (davidsch) as these files have specific naming conventions and there are unit tests in the psana_test package that access them. Creating a new subdirectory requires less coordination, however if you think the test data is going to be useful to others, we should work together on it. One of the benefits of using the psana_test package, is I have a mechanism for checking in the md5 checksums of the test data into svn. This allows the unit tests to verify that the test data has not changed.

...

Using the psana_test package

Here I'll go through an example of preparing a xtc file with a new data typeusing the psana_test package to prepare a new test file to test a new pdsdata type. Of general interest will preparing of the test file, and functionality in the python library code in psana_test.

Suppose we don't have unit tests to see how psana handles Epix100aConfig version1 and EpixElement verion 2. I have identified an xtc file with this type, namely

/reg/d/psdm/xcs/xcsi0314/xtc/e524-r0213-s03-c00.xtc

One of the tools in psana_test will let me see which datagrams have epix. I can string together a unix command to see:

psana1302:/reg/d/psdm/xcs/xcsi0314/xtc $ xtclinedump xtc e524-r0213-s03-c00.xtc | grep -v -i epic | grep -v -i "type_name=Xtc" | head -40 | grep -i "dg=\|epix"
dg=    1 offset=0x00000000 tp=Event sv=      Configure ex=1 ev=0 sec=54754FDF nano=1E31D0E8 tcks=0000000 fid=1FFFF ctrl=84 vec=0000 env=0000161C
xtc d=2  offset=0x00022F4C extent=00108834 dmg=00000 src=01003069,19002300 level=1 srcnm=XcsEndstation.0:Epix100a.0 typeid=84 ver=1 value=10054 compr=0 compr_ver=1 type_name=Epix100aConfig plen=1083424 payload=0x0B...
dg=    2 offset=0x0012B780 tp=Event sv=       BeginRun ex=0 ev=0 sec=54755EBD nano=35A7112E tcks=0000000 fid=1FFFF ctrl=06 vec=0000 env=000000D5
dg=    3 offset=0x0012B820 tp=Event sv=BeginCalibCycle ex=0 ev=0 sec=54755EBE nano=00D5EE07 tcks=0000000 fid=1FFFF ctrl=08 vec=0000 env=00000000
dg=    4 offset=0x0012C188 tp=Event sv=         Enable ex=0 ev=0 sec=54755EBE nano=0124366D tcks=0000000 fid=1FFFF ctrl=0A vec=0000 env=80000000
dg=    5 offset=0x0012C228 tp=Event sv=       L1Accept ex=1 ev=1 sec=54755EBE nano=05EE73D6 tcks=005094A fid=144F9 ctrl=8C vec=146F env=00000003
xtc d=3  offset=0x0012C278 extent=0010A440 dmg=00000 src=01003069,19002300 level=1 srcnm=XcsEndstation.0:Epix100a.0 typeid=75 ver=2 value=2004B compr=0 compr_ver=2 type_name=EpixElement plen=1090604 payload=0x00...
dg=    6 offset=0x00266284 tp=Event sv=       L1Accept ex=1 ev=1 sec=54755EBE nano=06EE644D tcks=0050974 fid=144FF ctrl=8C vec=1475 env=00000003

so if I copy bytes [0,0x00266284) I will get the first datagram with an EpixElement in it. I'd also like to get the last datagrams in the file, for a proper end of the calib cyle, etc. These are

dg= 1206 offset=0x5C499EA0 tp=Event sv=        Disable ex=0 ev=0 sec=54755EFA nano=172A7720 tcks=0000000 fid=1FFFF ctrl=0B vec=0000 env=00000000
dg= 1207 offset=0x5C499F40 tp=Event sv=  EndCalibCycle ex=0 ev=0 sec=54755EFA nano=17A3F59A tcks=0000000 fid=1FFFF ctrl=09 vec=0000 env=00000000
dg= 1208 offset=0x5C499FE0 tp=Event sv=         EndRun ex=0 ev=0 sec=54755EFA nano=19054BF0 tcks=0000000 fid=1FFFF ctrl=07 vec=0000 env=00000000

So I also want bytes [0x5C499EA0, end of file), and I can see the file length is 1548329088.

Part of the psana_test package is a library function to do this. From a Python shell, I do

In [9]: import psana_test.psanaTestLib as ptl
In [10]: ptl.copyBytes('/reg/d/psdm/xcs/xcsi0314/xtc/e524-r0213-s03-c00.xtc', [[0,0x00266284],[0x5C499EA0, 1548329088]], '/reg/g/psdm/data_test/Translator/test_089_xcs_xcsi0314_e524-r0213-s03-c00.xtc')
copying 2 sets of bytes:  [0,2515588)  [1548328608,1548329088)  from src=/reg/d/psdm/xcs/xcsi0314/xtc/e524-r0213-s03-c00.xtc to dest=/reg/g/psdm/data_test/Translator/test_089_xcs_xcsi0314_e524-r0213-s03-c00.xtc

To create the test file.