Externalizing Settings for Data-Dependent Unit Tests using Proteus DatabaseUnitTestBase

1

in a recent request for the Proteus Project, a user asked about defining test-related settings in external .config files to control connection strings settings, the location(s) of support files like the serialized dataset xml data files and the dataset xsd schema files, and possibly other things.  While it might be of value to ‘formalize’ this capability by designing an actual fixed .config file format and adding direct support for it into Proteus, it occurs to me that one of the reasons for this question may be that its not entirely clear to adopters of the framework how it was intended to allow just such extension to its behavior ‘natively’.

This post is intended to try to put some background context around the design concepts that went into the architecting of the unit-test-related parts of the Proteus Project codebase and in the process understand how we can simply extend the behavior of the library to read settings from any arbitrary location.

Digression: Note that there are effectively two halves to the Proteus Project library: the half intended to provide convenience utility methods in support of database-dependent unit testing via the NDbUnit project and the half that provides infrastructure for domain-modeling.  This post is entirely about the half related to unit testing support.

Anatomy of the DatabaseUnitTestBase Class

The core of the database-unit-testing support provided by the Proteus project is the DatabaseUnitTestBase class.

Before everyone jumps all over me for appending the ugly wart ‘Base’ suffix to a class name, this is a convention that I always use when designing abstract classes that are only ever intended to be used as a supertype from which to derive (and of course the abstract keyword helps the compiler ensure that this will be the case!).

We all still use the letter ‘I’ as a prefix for interface declarations, so get over yourselves.  Clearly we still agree as a community that some warts are acceptable!

Fundamentally, there are several underlying design ideas that characterize this class:

  • its declared abstract so that use of it is achieved by deriving your own class from it, either directly or indirectly through as deep an inheritance hierarchy as you might want to suit your test-structure needs
  • it provides reasonable default assumptions for its settings so that if you just derive-and-go (and accept the conventions!) it will ‘just work’ with minimal config tweaking required
  • all of its operations that require access to settings (e.g., things like connection strings, paths to xml/xsd files, etc.) retrieve those settings via public property accessors
  • all of these public property accessors are declared virtual in the base class so that you can override them in any derived class as you see fit

Its this last aspect (overridable virtual properties) that provides us the needed extensibility point for us to extend the behavior of the base class to read its settings from some arbitrary config store of our choice in answer to the issue posted to the Proteus issues list.

Design Strategy for our Extension

In order to support externally-configurable settings for the DatabaseUnitTestBase class, the first thing we need to do is devise a simple set of design ideas for our approach:

  1. We will store everything in the app.config file; Microsoft went to all that trouble to give us this file format and integrate support for editing it in Visual Studio, MSBuild auto-magically copying it to the build output directory and renaming it <assemblyname>.<extension>.config when compiling, etc. so let’s leverage all that work
  2. Since we’re going be storing (among other things) connection strings, we will similarly leverage the app.config file’s native support for named connection strings and store the connection strings we might need there
  3. Named connection strings will be correlated to each of the tests by using the typename of the test fixture class as the ‘name’ of the named connection string
  4. For settings other than the connection strings, settings in the app.config file will be correlated to test fixtures by the simple convention of prefixing them with the typename of the test fixture class

A Sample Config File

Just so that we have an idea of where we are headed, here’s a look at a sample app.config file that supports two different test fixtures in our test project: Fixture1 and Fixture2:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <connectionStrings>
        <add name="Fixture1" connectionString="Data Source=.;Database=testdb;Trusted_Connection=True" />
        <add name="Fixture2" connectionString="Data Source=otherserver;Database=otherdb;Trusted_Connection=True" />
    </connectionStrings>
  <appSettings>
    <add key="Fixture1_TestSchemaFile" value="c:\MyTestData\SomeSchema.xsd" />
    <add key="Fixture1_TestDataFile" value="c:\MyTestData\SomeData.xml" />
    <add key="Fixture2_TestSchemaFile" value="c:\OtherTestData\schema.xsd" />
    <add key="Fixture2_TestDataFile" value="c:\OtherTestData\data.xml" />
  </appSettings>
</configuration>

In this example, for tests in Fixture1 we want to use the following settings:

  • a local server with a database named testdb
  • the xsd schema file (SomeSchema.xsd) and the xml data file (SomeData.xml) are located in c:\MyTestData\

And for tests in Fixture2 we want to use the following different settings:

  • a remote database server named “otherserver” with a database named otherdb
  • the xsd schema file (schema.xsd) and the xml data file (data.xml) are located in c:\OtherTestData\

Implementation: Derive and Override

As mentioned earlier, in order to change the behavior of the DatabaseUnitTestBase class, we need to derive from it and override some or all of its behaviors (as needed).  As you can see from the following screenshot, if you type the override keyword and hit the space bar, Visual Studio is kind enough to show you all of the possible super-class methods you can override in your derived class…

image

As you can see from the above screenshot, nearly every setting is accessible via an overridable property ‘getter’ and nearly every behavior is accessible via an overridable method.

Don’t like the way the base class reads its connection string?  Override the DatabaseConnectionString property getter.  Don’t like what the library is doing in its DatabaseFixtureTearDown method?  Override the DatabaseFixtureTearDown() method and provide your own behavior.  And because overridden class elements in a derived class still have access to the underlying base class’ original methods, its entirely possible to extend and enhance the original behavior rather than completely replace it as in the following snippet:

protected override void DatabaseFixtureTearDown()
{
    //do something special BEFORE the database state is reset
    base.DatabaseFixtureTearDown();
    //do something special AFTER the database state is reset
}

Overriding What We Need, Ignoring What We Don’t

In our specific example, in our derived class we only need to override the property getters for DatabaseConnectionString, TestDataFilename, and TestSchemeFilename so that they read the values we want out of the app.config file rather than simply performing their default out-of-the-box behavior.

Just for the record, the default assumptions of the DatabaseUnitTestBase class that we will be overriding are:

  • that the connection string is stored in the app.config and is named testDatabase
  • that the test data is stored in a file named TestData.xml and is stored in a folder named TestData placed directly under the project root folder in Visual Studio
  • that the dataset schema is stored in a file named Database.xsd and is stored in a folder named TestData placed directly under the project root folder in Visual Studio

In our case, none of these assumptions the DatabaseUnitTestBase class makes is true so we must override its behavior to reflect retrieving the settings from the app.config file based on the different conventions we want from our ‘Design Strategy’ discussed earlier in this post.

Implementing Our Fixture1 Class

So what’s an implementation of this going to look like?  Let’s take a quick look at how simple this actually is…

Get Discriminator

To get started, we can immediately see that we’re going to need a simple convenience method in our class that can get us the name of the test fixture since its effectively the discriminator in our app.config settings.

private string GetSettingsDiscriminator()
{
    return this.GetType().Name;
}

Almost an absurdly simple method, this inspects the current type (which will be the test fixture class) and returns it’s unqualified name (e.g., in the case of Fixture1, this will return the string “Fixture1”).

We bother to write this simple method none-the-less because it helps abstract out how the discriminator is determined so that if at some later point we want to bother to determine it some other way, calling code doesn’t have to concern itself with how the discriminator is actually determined.  A very simple example of this is that it might (later) be worthwhile changing the discriminator to be the entire fully-qualified typename (by returning this.GetType().Fullname instead) so that more than one Fixture1 class could co-exist in the same test assembly (but obviously in different namespaces else they would collide and the compiler would complain!).  For our example, returning the unqualified typename is fine.

Overriding The Connection String

Overriding the retrieval of the connection string is very simple, we just need to use the .NET ConfigurationManager class to retrieve the named connection string based on our discriminator value (our fixture name).  Now that we have our GetSettingsDiscriminator() method defined, this becomes quite simple:

protected override string DatabaseConnectionString
{
    get
    {
        return ConfigurationManager.ConnectionStrings[GetSettingsDiscriminator()].ConnectionString;
    }
}

Overriding the Schema File

Following a similar pattern, all we need to do to enable our desired retrieval of the Schema File on a fixture-specific basis is override the TestSchemaFilename property getter as in…

protected override string TestSchemaFilename
{
    get
    {
        return ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestSchemaFile"];
    }
}

Following our ‘design guidelines’ of storing the fixture-specific non-connection-string settings with keys prefixed with the fixture name, we again invoke our helper method and just concatenate it onto the front of the expected appSettings key.

Overriding the Data File

Unsurprisingly, we follow the same approach to overriding the retrieval of the Data File on a fixture-specific basis, overriding the TestDataFilename property to perform the same kind of operation (read from the app.config file)…

protected override string TestDataFilename
{
    get
    {
        return ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestDataFile"];
    }
}

Where to Put All This?

That’s really about all there is to it.  Since the methods within the DatabaseUnitTestBase class are expressed in terms of these public properties, when the class goes about its work, it will use the overridden property accessors to read the values out of the app.config rather than using its ‘default’ values for these settings.

This leaves us with a bit of a conundrum however: since these base-class methods need to be overridden in each test fixture class that derives from DatabaseUnitTestBase, we would seem to need to reproduce all this code (even though its hardly much) in each of our test fixtures.

Inheritance to the Rescue!

Rather than going that route, I would suggest instead that all these overrides simply be refactored into a common class from which you can then inherit each of your test fixtures as needed.  In my example, I’m calling this new class ConfigDrivenDatabaseUnitTestBase as in…

public abstract class ConfigDrivenDatabaseUnitTestBase : DatabaseUnitTestBase
{
    protected override string DatabaseConnectionString
    {
        get
        {
            return ConfigurationManager.ConnectionStrings[GetSettingsDiscriminator()].ConnectionString;
        }
    }

    protected override string TestDataFilename
    {
        get
        {
            return ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestDataFile"];
        }
    }

    protected override string TestSchemaFilename
    {
        get
        {
            return ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestSchemaFile"];
        }
    }

    private string GetSettingsDiscriminator()
    {
        return this.GetType().Name;
    }
}

I’m leaving it also abstract so that its clear (and the compiler will enforce!) that its intended to only be used as a super type from which to derive other classes.

Using the ConfigDrivenDatabaseUnitTestBase Class

Using the new class in your own tests is now identical to using the original non-overidden DatabaseUnitTestBase class: simply derive from ConfigDrivenDatabaseUnitTestBase and get started as you normally would using the original class…

[TestFixture]
public class Fixture1 : ConfigDrivenDatabaseUnitTestBase
{
    [Test]
    public void Test()
    {
        //your test here!
    }
}

[TestFixture]
public class Fixture2 : ConfigDrivenDatabaseUnitTestBase
{
    [Test]
    public void Test()
    {
        //your test here!
    }
}

Clean-Up and Error-Prevention

This example is necessarily down-and-dirty and contains no error-checking or even fall-back behavior if settings aren’t found in the app.config file.  Some ideas here for the reader include…

  • throwing an exception if a setting isn’t found
  • falling back to the ‘default’ behavior is a setting isn’t found

The exception approach might make sense if you want to only use this new base class in cases where you know you will use app.config-based settings and want explicit notification when a setting is missing as in…

protected override string TestDataFilename
{
    get
    {
        string filename = ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestDataFile"];
        if (String.IsNullOrEmpty(filename))
            throw new ConfigurationException("Could not find setting for TestDataFilename!");

        return filename;
    }
}

Since the overridden methods of base classes are still accessible to derived classes, its also possible to ‘fall-back to default behavior’ if custom settings aren’t found.  This might make sense when you want to use the new base class everywhere and then selectively override just the settings for certain fixtures that need different settings by adding them to the .config file if/when needed as in…

protected override string TestDataFilename
{
    get
    {
        string filename = ConfigurationManager.AppSettings[GetSettingsDiscriminator() + "_TestDataFile"];
        if (String.IsNullOrEmpty(filename))
            filename = base.TestDataFilename;

        return filename;
    }
}

This second approach would permit you to use this new ConfigDrivenDatabaseUnitTestBase class as a base for all your fixtures without regard for whether you place settings for each in your app.config file.

Override, Override, Override!

This specific example gives just a flavor for the kinds of extensibility that are possible using the Proteus library for data-dependent unit testing.  I hope this helps clarify how its really quite simple to change the behavior of (nearly!) the entire base by deriving your own class from it and overriding properties and methods as you see the need for your specific situation(s).  Since all of the methods and properties are virtual, it should be possible to twist the class into doing just about whatever you find you need.

Obviously if there are things that this approach to derive-override-extend doesn’t support, let me know and I can look into extending the library in other ways too!

Mean time, Happy coding!