• Created by Unknown User (davidsch), last modified on Jan 05, 2015

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Introduction

The analysis release build system, SConsTools, provides a mechanism for integrating unit tests. Each package in the release, or a package that a user is developing, can have its own tests. All tests can be run by a simple command. Users may find this useful for testing their packages. For packages that psana developers add to the analysis release, these tests are automatically run during the nightly build. This page is primarily for psana developers, to go over how to add unit tests to test packages in the release during nightly builds. There are special considerations to make for tests that are part of the nightly build discussed below.

Creating a Package test directory

As an example, lets make a package with both a Python and a C++ unit test. For the example below, I am starting from the directory rel in my home directory. I make a new release, a new package in the release, and the crucial step is that I make a subdirectory called test in my package:

psanacs051:~/rel $ newrel ana-current unitTestTutorial
psanacs051:~/rel $ cd unitTestTutorial/
psanacs051:~/rel/unitTestTutorial $ sit_setup
psanacs051:~/rel/unitTestTutorial $ newpkg MyPkg
psanacs051:~/rel/unitTestTutorial $ mkdir MyPkg/test

Now when one does

scons test

You are building and running the test target. SConsTools will look in the test subdirectory for all packages. It looks for unit tests in these test subdirectories. It looks for:

  • Any file without an extension is treated as a test script. This script will be installed and run.
  • Any file that looks like C or C++ code (.c, .cpp extension, etc) is treated as a test program. It will be compiled, installed, and run.
  • If a test script or program returns non-zero, it failed and scons will report this.

Other files in the test directory are ignored.

Python Unit Test

Add the file MyPkg/test/myfirst

#!@PYTHON@
import sys
if __name__ == '__main__':
 print "Running my test."
 sys.exit(1)

When you do scons test, you will get the output

Running UnitTest: "build/x86_64-rhel5-gcc41-opt/MyPkg/myfirst"
************************************************************************************
*** Unit test failed, check log file build/x86_64-rhel5-gcc41-opt/MyPkg/myfirst.utest ***
************************************************************************************

This is because the script returned something non-zero. If you look at the file build/x86_64-rhel5-gcc41-opt/MyPkg/myfirst.utest you see the output of the script.

The syntax @PYTHON@ is explained in the SConsTools page.

Change the sys.exit(1) to sys.exit(0) and the test will succeed. You can also take out the sys.exit line, be default Python will return 0 after the script runs.

C++ Unit Test

A simple C++ test would like this, create the file MyPkg/test/mysecond.cpp

#include <iostream>
int main() {
 std::cout << "Cpp test" << std::endl;
 return -1;
}

This test will also fail. Note, scons test stops after the first test fails. If you have not changed myfirst to return 0, only one of myfirst and mysecond will be run before failure is reported.

Using Frameworks for Unit Tests

Most people will use a framework for unit tests. You can use whatever framework you like. We have been using the standard Python unit test module, as well as the boost unit_test package. 

Python unittest Framework

An example using the Python unit test framework would be

#!@PYTHON@

import sys
import unittest

class MyTest( unittest.TestCase ) :

    def setUp(self) :
        """ 
        Method called to prepare the test fixture. This is called immediately 
        before calling the test method; any exception raised by this method 
        will be considered an error rather than a test failure.  
        """
        pass

    def tearDown(self) :
        """
        Method called immediately after the test method has been called and 
        the result recorded. This is called even if the test method raised 
        an exception, so the implementation in subclasses may need to be 
        particularly careful about checking internal state. Any exception raised 
        by this method will be considered an error rather than a test failure. 
        This method will only be called if the setUp() succeeds, regardless 
        of the outcome of the test method. 
        """
        pass

    def test_mytest(self):
        a=3
        b=4
        self.assertEqual(a,b)

if __name__ == '__main__':
  unittest.main(argv=[sys.argv[0], '-v'])

after doing scons test, you will get failure, and see looking at the utest output file will find

********************************************************************************************************
*** Unit test failed, check log file build/x86_64-rhel5-gcc41-opt/MyPkg/using_python_framework.utest ***
********************************************************************************************************
scons: *** [build/x86_64-rhel5-gcc41-opt/MyPkg/using_python_framework.utest] Error 256
scons: building terminated because of errors.
psana1302:~/rel/unitTestTutorial $ cat build/x86_64-rhel5-gcc41-opt/MyPkg/using_python_framework.utest
test_mytest (__main__.MyTest) ... FAIL
======================================================================
FAIL: test_mytest (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "build/x86_64-rhel5-gcc41-opt/MyPkg/using_python_framework", line 31, in test_mytest
    self.assertEqual(a,b)
AssertionError: 3 != 4
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)

 

Boost unit_test Framework

exa,

 

 

 

 

 

  • No labels