Integration Testing your Asp .Net Core app with an in memory database

Parts:

  1. Integration testing your asp .net core app with an in memory database (this)
  2. Integration testing your asp .net core app dealing with anti request forgery csrf formdata and cookies

Revisions:

  • 14th august 2016 – updated for .net core 1.0
  • 29th april 2016 – first version covering RC1

Recently I am working with .Net (C#). And we’re working in .Net Core, which is awesome. (new stuff, wooh yeah!). I wanted to set up integration testing and it was tough to find resources to make it all happen. Which is pretty obvious considering how new some stuff is.

I found some articles scattered around this topic. But, there is not a full guide from “start till ‘full integration testing’ + in memory database”. So because of the lack of it, here is my take.

I couldn’t have made it this far without some notable resources (see below) and the answer to my Github question (with a friendly and very constructive response, thanks Asp.Net guys!).

Overview: What this blog post covers

  1. Setting everything up – your first integration test
  2. Run your tests against an in memory database + making sure the memory database has its tables set up.
  3. Then make it as fast as possible

Do note: An in memory database is NOT the same as your SQL Server. But if that is not bothering you (or not in these test cases), no worries there.

Step 1: First make it work – setting everything up

I assume you don’t have any integration test running yet. I am using xUnit. Read this well written article[#1] how to set up your integration test base. I summarise here quickly, but if you get stuck read the article. Then get back here.

Hook up your dependencies, here are mine (taken from project.json):

...
"dependencies": {
    ... // my project dependencies
    "FluentAssertions": "4.2.1",
    "xunit": "2.1.0",
    "xunit.runner.dnx": "2.1.0-rc1-build204",
    "Microsoft.AspNetCore.TestHost": "1.0.0",
  }
...

Within your test class, define:


	public TestServer server { get; }

	public HttpClient client { get; }

Then in the constructor of your test class:

var builder = new WebHostBuilder().UseStartup<Startup>();

server = new TestServer(builder);

client = server.CreateClient();

Now also create a test case. Something along the lines of:


[Fact]
public async void TestVisitRoot() {
    var response = await client.GetAsync("/");
    response.EnsureSuccessStatusCode();
}

This is basically the example from original article, but stripped down (where applicable).

Try running the test case first. It should run the app as if you ran it normally and it would visit the homepage. It also connects to your real database, webservices and whatnot.

Congrats, you completed step one. On to the next. Where we will be…

Step 2: Replacing database with an in-memory SQLite database

I assume you use a SQL Server in your ‘real world’ scenario. For integration testing you want to have a predictable state before running the test. An empty database is pretty predictable (after you fill it up with test data ;-)).

Also an in-memory database saves you the hassle of dealing with files, permissions, removing (temp) files, etc.

In order to inject our in memory database, we need to override the Startup class. We need to create a seam in our class so we can write our test-specific (ie override) database setup code there.

We start by creating a class TestStartup which extends from Startup. Then we make sure we use TestStartup in our constructor in our test class:

var builder = new WebHostBuilder()
	.UseStartup<TestStartup>() // use our testStartup version


In your Startup class you need a method where you are setting up your database. You probably do this within a Configure or ConfigureServices method. Instead of wiring up the database within that method, extract that code in a separate method and call it SetupDatabase.

The code in that method might look a bit like this:

public virtual void SetUpDataBase(IServiceCollection services)
{
	services
		.AddEntityFramework()
		.AddSqlServer()
		.AddDbContext<YourDatabaseContext>(options =>
			options.UseSqlServer(
				Configuration["Data:DefaultConnection:ConnectionString"]
		));
}

Make sure you define this method as virtual. This allows us to override it within our TestStartup class.

Before we override the method, we need to make sure we have the appropiate SQLite dependency defined in our project.json. So add it. This should make the dependencies look like:

...  
"dependencies": {
    ...
    "FluentAssertions": "4.2.1",
    "xunit": "2.1.0",
    "xunit.runner.dnx": "2.1.0-rc1-build204",
    "Microsoft.AspNetCore.TestHost": "1.0.0",
    "Microsoft.EntityFrameworkCore.InMemory": "1.0.0",
    "Microsoft.EntityFrameworkCore.Sqlite": "1.0.0",
  }

Now, in your TestStartup override method SetupDatabase and let it set up your SQLite in memory database:

public override void SetUpDataBase(IServiceCollection services)
{
	var connectionStringBuilder = new SqliteConnectionStringBuilder { DataSource = ":memory:" };
	var connectionString = connectionStringBuilder.ToString();
	var connection = new SqliteConnection(connectionString);

	services
		.AddEntityFrameworkSqlite()
		.AddDbContext<CmsDbContext>(
			options => options.UseSqlite(connection)
		);
}

Try running your app. See how it behaves.

You might run into problems where it complains about not having a database setup, or no tables being found. No worries, there are a few things left to do.

Ensure creation of database

At some place in your webapp you most likely create your dbContext, along the lines of:

//Create Database
using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>()
	.CreateScope())
{
	var dbContext = serviceScope.ServiceProvider.GetService<YourDatabaseContext>();

	// run Migrations
	dbContext.Database.Migrate();
}

For making sure your in-memory database has a database setup (with tables, etc). In general you want to do this…:

//Create Database
using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>()
	.CreateScope())
{
	var dbContext = serviceScope.ServiceProvider.GetService<CmsDbContext>();

	dbContext.Database.OpenConnection(); // see Resource #2 link why we do this
	dbContext.Database.EnsureCreated();

	// run Migrations
	dbContext.Database.Migrate();
}

Of course you want this only for integration tests. So don’t leave it like that. Again, create a seam. So you get:

// method in Startup.cs
public virtual void EnsureDatabaseCreated(YourDatabaseContext dbContext) {
	// run Migrations
	dbContext.Database.Migrate();
}

// within your Configure method:
using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>()
	.CreateScope())
{
	var dbContext = serviceScope.ServiceProvider.GetService<YourDatabaseContext>();
	EnsureDatabaseCreated(dbContext);
}

And in your TestStartup you override it like so:

// method in TestStartup.cs
public override void EnsureDatabaseCreated(YourDatabaseContext dbContext) {
	dbContext.Database.OpenConnection(); // see Resource #2 link why we do this
	dbContext.Database.EnsureCreated();

	// now run the real thing
	base.Migrate(dbContext);
}

Same trick. Now re-run your test. It should work now. You could leave it like this. There are a few challanges up ahead, like dealing with cookies, anti-request forgery and so on. I might blog about those too.

Note: overriding like this might not be the only/best case after RC1, as there are changes that should make it way easier to add your own dependencies/setup that will be coming in RC2.

Now, the downside of integration tests is that b ooting them up is very slow compared to unit tests. So you want to do that only once (preferably) and then run all your tests. Yes, that also has downsides, your tests should be careful when sharing state throughout one webapp run. So make sure you keep your tests isolated.

Step 3: Speed up your integration tests! Use a TestFixture + xUnit collection

Inspired by another article[#3] and xUnit’s ability to use xUnit’s CollectionDefinition you can make sure your webapp is only booted once.

Sharing webapp between test cases using a TestFixture

This solves the problem: creating a web app for each test case.

To do this, in short, create a new class. For instance TestServerFixture. Move your client/server setup in this class. So it looks like this:

public class TestServerFixture : IDisposable
{
	public TestServer server { get; }

	public HttpClient client { get; }

    public TestServerFixture()
    {
		// Arrange
		var builder = new WebHostBuilder()
			.UseEnvironment("Development")
			.UseStartup<TestStartup>();
			// anything else you might need?....

		server = new TestServer(builder);

		client = server.CreateClient();
	}

	public void Dispose()
    {
		server.Dispose();
		client.Dispose();
    }
}

Note the differences, the testFixture implements an IDisposable interface. Your setup which was in your test class constructor, has moved to the constructor of the fixture.

Now, to make things easier (as you will create more and more integration test classes), create an abstract test class, which will be setup to receive the TestServerFixture. Then you can extend from this abstract class in your concrete test classes.

The abstract class would look like this:

public abstract class AbstractIntegrationTest : IClassFixture<TestServerFixture>
{
	public readonly HttpClient client;
    public readonly TestServer server;


    // here we get our testServerFixture, also see above IClassFixture.
    protected AbstractIntegrationTest(TestServerFixture testServerFixture)
	{
		client = testServerFixture.client;
		server = testServerFixture.server;
	}
}

As you can see we use an IClassFixture. Which is used for shared context between test cases..

This little boilerplate code will now allow us to get our concrete test class to look like:

public class MyAwesomeIntegrationTest : AbstractIntegrationTest
{
	public MyAwesomeIntegrationTest(TestServerFixture testServerFixture) : base(testServerFixture)
	{
	}

 [Fact]
public async void TestVisitRoot() {
    var response = await client.GetAsync("/");
    response.EnsureSuccessStatusCode();
}

// etc more tests...
}

Sharing your webbapp between test classes using xUnit’s CollectionFixture

This solves the problem: creating a web app (using a TestFixture) for each test class.

So awesome you have multiple test classes and you notice that you boot up your webapp everytime. And you want to solve this for (some) classes. Well that is possible. For obvious pro’s and con’s which I won’t dive into. (beware of state! ;-))

Setting up a CollectionFixture is also explained here. But for completeness sake, let me rephrase:

First create a class that we will use to define a collection. Like so:

[CollectionDefinition("Integration tests collection")]
public class IntegrationTestsCollection
{
	// This class has no code, and is never created. Its purpose is simply
	// to be the place to apply [CollectionDefinition] and all the
	// ICollectionFixture<> interfaces.
}

Now above all test classes you want to include in the Integration tests collection, simply add this line above the class definition:

[Collection("Integration tests collection")]

Which makes, in our above example, it like this:

[Collection("Integration tests collection")]
public class MyAwesomeIntegrationTest : AbstractIntegrationTest
{
	public MyAwesomeIntegrationTest(TestServerFixture testServerFixture) : base(testServerFixture)
	{
	}

 // your tests here...
}

Now run your tests again and note you boot up your webapps for each collection only once. Hence if you put everything in one collection, your webapp will only boot once.

Conclusion

We can run integration tests. If we want we can run them against an in memory database. Using seams we can inject our own test setup code, which probably changes in RC2 or further. If we want to speed up our tests we can put them in one “Integration collection” and use a single TestFixture.

Resources:

1. Asp.net docs – Integration testing
2. SQlite in memory database create table does not work
3. Fast testing
4. My original question/ticket at Github

6 comments

  1. Found this post from the GitHub issue and it’s really helped get me out of the hole I was in (which originally trying EF’s in memory DB with no luck).

    I am experiencing a couple issues though I thought you might be able to clear up. When I have dbContext.Database.EnsureCreated() present just above the migration call I’m getting a table already exists error (SQLite Error 1: ‘table “AspNetRoles” already exists’). Is EnsureCreated actually required and does it also run the migrations automatically if the tables don’t exist?

    Secondly, I had a whole heap of issues with migration relating to foreign keys (SQLite does not support this migration operation (‘DropForeignKeyOperation’)). I had to nuke my migration scripts and start over to get it to work. Obviously this isn’t ideal as at some point in the future I’m going to run into the same issue. Is this a short-fall of SQLite or do you know what I’m doing wrong (or have any suggestions)?

    Cheers!

    1. Hey James,

      I am quite sure you need to run the EnsureCreated function. If you get a table already exists, it basically means just that (the table exists). It can have several reasons:
      1. You do not run in memory and the SQLLite file already exists, hence the DB was filled in the first run using EnsureCreated. Make sure the file is removed for every run.
      2. You do run in memory mode (datasource setting), but you need to be sure you do dbContext.Database.OpenConnection() *before* you do EnsureCreated. Also see: http://stackoverflow.com/questions/17477246/sqlite-in-memory-database-create-table-does-not-work

      About SQLite, yeah it will fall short on many occasions. This is by no means a full fledged solution for more complex scenario’s with more hefty interactions on the DB side (in my case it is very straight-forward and easy).

    1. Hey Michal,

      Thx! Yeah it would certainly be handy to do. Perhaps you could help me with that? I can set up a repo and we could set up some simple web app up together?

  2. Hi, I followed your step 3 (Sharing your webbapp between test classes using xUnit’s CollectionFixture) and I saw it still creates each TestServer for each test class. After playing with it for a while and read the original one, I fixed it by:
    [CollectionDefinition(“Integration tests collection”)]
    public class IntegrationTestsCollection : ICollectionFixture
    Then remove “IClassFixture” from the abstract class AbstractIntegrationTest
    Also, we can decorate the “[Collection(“Integration tests collection”)]” above that abstract class so the concrete test classes don’t need to decorate it again to avoid duplication code.

  3. Hi, thank you for a great article. I am struggling with one thing; to insert some test/seed data after creation of the Sqlite database. I was thinking of adding the code to EnsureDatabaseCreated method, but I’ve tried this and other things as well without any success. If I call SaveChanges the db reports that the table does not exist when I try to get the data… If you have any small examples of that it would be greatly appreciated – thank you πŸ™‚

Leave a Reply

Your email address will not be published. Required fields are marked *