Wednesday 19 September 2012

Displaying database unit test package names in CI

I've been using tSQLt tests for a while now with Continuous Integration, and one of the things I've noticed is that test results from nUnit or JUnit are presented with a nice package name, but that the tSQLt tests either have no package name, or list as (root) - depending on your CI system.

TeamCity

Jenkins

I have recently had the occasion to realise quite how annoying this can be, particularly if you're using Jenkins and running tests with the same names in multiple databases (TeamCity does list the database if you dig into the tests tab, but oddly not everywhere). This presents the problem that you can't easily see the name of the database in which a failure has occurred, to aid prompt troubleshooting.

So - I've done a little digging, and found this post on why this might be the case for Jenkins. I then modified the way which tSQLt tests are output to achieve the test labeling which Jenkins needs, i.e. "classname.testname". I felt that database name was the best class to suit my needs, but you can modify to suit your requirements.

The modification is to [tSQLt].[XmlResultFormatter]. I changed the selects in the union, for example in the snippet below:



I've changed the 'Class' lines to read:


Do this for all the 'Class' lines in the procedure. There is 1 change in the second union, and 2 in each of the third and fourth unions.

Making this change gives us the following result when we re-run the tests from Continuous Integration:

TeamCity
Jenkins

As you can see this small modification helps to aid visibility to the test failure display.

Once you've modified the framework, you can either script the change as part of your build process, or if you have a source controlled database, then you can simply change the source controlled version of the [tSQLt].[XmlResultFormatter] stored procedure.

By the way - you can also output test execution times to your CI system by modifying this same procedure - see this mailing list post for details.

I think these changes really help to integrate database unit tests within the CI process, and make them easier to use in a complex development environment.

No comments:

Post a Comment