Page History

...

However for data files checked in with the package, the general problem is where to put them, and how to find them at run time. Probably the best practice is to use the data subdirectory. This is intended for application data with the package, so you may want to make a subdirectory under it for testing files. The advantage of the data directory is that it is wired into the release. If you know your package name, you can use an environment variable to find the data directory. For example, if we hadSo if we add the directories

• MyPkg/data/testdata

And put a file in there called mytestfile.txt, then at run time, we could use the environment variable SIT_DATA to find the data subdirectory for the release. The data subdirectory for the release will have soft links to all the data directories for the packages. That is we will have the structure

• unitTestTutorial/MyPkg/data    # our data dir
• unitTestTutorial/data               # release data dir
• unitTestTutorial/data/MyPkg    # soft link to unitTestTutorial/MyPkg/data

SconsTools will create two directories:

• unitTestTutorial/data               # release data dir
• unitTestTutorial/data/MyPkg    # soft link to unitTestTutorial/MyPkg/data

Moreover psana provides a utility to find files in the data directory of the release. Namely the Python class AppDataPath in the AppUtils package (there is a C++ version there as well). So if we add the file

• MyPkg/data/testdata/mytestfile.txt

That starts with the string "my", then we could write the following unit test:And the environment variable SIT_DATA will be a : separated list of directories for data, starting with an absolute path to unitTestTutorial/data. Then one could then write a Python unit test like

    def testMyFile(self):
        import os
       assert 'SIT_DATA' in os.environ, "SIT_DATA not defined. Was sit_setup run?"
       dataPath = os.environ['SIT_DATA'].split(':')[0]from AppUtils.AppDataPath import AppDataPath
       testDataFilePath testFileDataRelPath = os.path.join(dataPath, 'MyPkg', 'testdata', 'mytestfile.txt')
       assert os.path.exists(testDataFilePath) testFilePath = AppDataPath(testFileDataRelPath).path()
        assert len(testFilePath)>0 , "test file (relative to release data dir): %s not found." % testDataFilePath
 testFileDataRelPath
        fileText = file(testFilePath, 'r').read()
        self.assertTrue(file(testDataFilePath).read().fileText.startswith("my"),
                       msg="Test file=%s doesn't start with my" % testDataFilePath) msg="Test file=%s doesn't start with my" % testFilePath)

The mechanism by which AppDataPath works is to use the environment variable SIT_DATA. This is set by SConsTools when scons is run. It is a : separated list of paths, the first being the absolute path to unitTestTutorial/data, the second being the absolute path to the data directory of the base release. AppDataPath goes through these paths in order, returning the first match. One thing to note, scons test only runs tests for packages that are part of the working release. It does not run tests for all the packages that are part of the base release. Given this, I think the simplest thing when writing unit tests is to only use the first path in SIT_DATA which will be for the working release. When working with application data, a package will want to try at least the first two paths in SIT_DATA to check for a MyPkg subdirectory. This allows it to see if it is checked out into the working test release, or just part of the base releasethere is no reason to search the base release, but there should be no harm as well. Harm could conceivably befall a developer who was modifying a test that is checked into an existing package in the base release. Were the developer to change the name of the test file in the working/test release, but not modify the unit test code to use the new name, then AppDataPath would find the old test file in the base release data directory.

External Test Data Location

...