Periodically I start a new project using Rails, with RSpec, Capybara, and Selenium for acceptance testing, and a short way into it I find myself banging my head against bizarre inconsistencies with the test database. I’ll set up some records in the test DB, only to have the Selenium-driven browser-based tests act like those records never existed. Eventually, I’ll realize what I did wrong and curse my feeble brain for not remembering the last time I solved the same problem.

The problem is always the same: the tests are being wrapped in database transactions, so any code running outside the actual test process (like, say, a server process servicing a Selenium-driven browser request) does not see the database fixture I’ve so carefully assembled.

I just walked a pairing client through the same fix, so in the interests of remembering the steps, and hopefully preventing some other folks from tearing their hair out, here are the steps needed.

First of all, and this is very important, go into spec/spec_helper.rb and change this line:

config.use_transactional_fixtures = true

To:

config.use_transactional_fixtures = false

This will disable rspec-rails’ implicit wrapping of tests in a database transaction. Without disabling this, none of the following configuration will matter.

Now configure database_cleaner. I usually create a separate file called spec/support/database_cleaner.rb for this. Inside, I put something like this:

RSpec.configure do |config|

  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

Let’s take that step by step.

config.before(:suite) do
  DatabaseCleaner.clean_with(:truncation)
end

This says that before the entire test suite runs, clear the test database out completely. This gets rid of any garbage left over from interrupted or poorly-written tests—a common source of surprising test behavior.

config.before(:each) do
  DatabaseCleaner.strategy = :transaction
end

This part sets the default database cleaning strategy to be transactions. Transactions are very fast, and for all the tests where they do work—that is, any test where the entire test runs in the RSpec process—they are preferable.

config.before(:each, :js => true) do
  DatabaseCleaner.strategy = :truncation
end

This line only runs before examples which have been flagged :js => true. By default, these are the only tests for which Capybara fires up a test server process and drives an actual browser window via the Selenium backend. For these types of tests, transactions won’t work, so this code overrides the setting and chooses the “truncation” strategy instead.

config.before(:each) do
  DatabaseCleaner.start
end

config.after(:each) do
  DatabaseCleaner.clean
end

These lines hook up database_cleaner around the beginning and end of each test, telling it to execute whatever cleanup strategy we selected beforehand.

And that’s it!

Note that this is all for RSpec, and does not cover Cucumber configuration.

Hopefully this will help someone else out there avoid the frustrations I’ve run into!

EDIT: A few people have asked me why I don’t just force all threads to share the same ActiveRecord connection, as demonstrated in this Gist. A few reasons:

• Using database_cleaner implies that I want ORM neutrality. database_cleaner supports ActiveRecord, DataMapper, MongoMapper, and others. The solution linked above only works for ActiveRecord.
• It’s a monkey-patched kludge which will only work so long as AR refrains from changing its connection-sharing internals. And frankly I’m not sure I trust it to work across all Ruby VM and database combinations (UPDATE: And indeed, I’ve now seen two different people say there are race conditions with the current Postgres adapter). I’d be more inclined to use it if ActiveRecord had a published configuration option which was known to work in all contexts.
• As I stressed above, I’m careful to set things up so that only the tests that need them fall back to truncation. Since :js => true tests generally don’t form the bulk of my suite (and since they are unavoidably slow anyway, due to the overhead of driving a browser), I’m not overly concerned about the added overhead. Perhaps if all of my acceptance tests drove a live browser I’d be more worried about it.
• In cases where database truncation is taking up a significant amount of test time, you can usually speed things up with some judicious control of which subset of tables get truncated for a given test. This is something database_cleaner makes pretty easy. Maybe that would make a good topic for a followup post.
• UPDATE: Oh yeah, and as @donaldball points out, sharing a transaction between test and test server means acceptance tests don’t run quite the same as they would in production. Specifically, they’ll never trigger after_commit hooks.

Myron Marston has done a stellar job shepherding it through the Rails 3 transition, but he’s busy with other awesome projects now and has asked me to look for a new maintainer. So: If you’re interested in helping keep NullDB up to date with the latest changes to ActiveRecord, please get in touch!

Lately I’ve been working with DataMapper and I’d begun to wonder how difficult it would be to write a DataMapper version of NullDB. As it turns out, the answer is “trivially easy”. In fact, it probably doesn’t even warrant making a library out of it – all the tools needed are right in the DataMapper distribution.

The task of a DataMapper null adapter is simplified in comparison to the ActiveRecord version by dint of DataMapper’s differing architecture. Where ActiveRecord scans the DB schema to extrapolate tables and columns, DataMapper makes the model classes the sole canonical source of schema information. This means that all the mucking about with schema.rb files that the original NullDB had to do is unnecessary.

The simplest form of a null database adapter is one that simply raises an exception every time code tries to touch the database. This kind of null backend is useful when you want to ensure that the software under test is completely isolated from the database. Rails Views are one example of code which you might want to keep rigidly database-independent in this fashion.

Setting up this type of null database backend in DataMapper is trivial:

  DataMapper.setup(:default, "abstract::")

This code uses the “abstract” adapter which comes with DataMapper. The Abstract adapter is primarily intended to provide a base class on which to build other adapters. But for our purposes it makes for a poor-man’s null database backend. Any attempt to write to or query from the database while this adapter is in effect will result in a NotImplementedError being raised:

  p = Post.new(:title => "NullDB for DataMapper")
  p.save # => NotImplementedError

What if we want to incoporate this technique into a complete test suite which includes both database-dependent and database-free tests? We need a way to make the null database active only while selected tests are being run. Once again, DataMapper’s design makes it easy to achieve our ends. DataMapper incorporates the concept of a “repository stack”. instead of only having a single database adapter active for a whole session, or having different models tied to different adapters, DataMapper keeps a stack of adapter sessions in a stack. New sessions can be pushed onto the stack, and then popped off when they are no longer needed.

Here’s an example of how to set up a single Test::Unit test case to use a null database backend, while leaving other tests unaffected:

class FooTest < Test::Unit::TestCase
  def setup
    DataMapper.setup(:null, "abstract::")
    DataMapper::Repository.context.push(DataMapper.repository(:null))
  end

  def teardown
    DataMapper::Repository.context.pop
  end

  def test_bar
    # ...
  end
end

There's one kind of test that NullDB enables which the code we've looked at so far does not allow. That is, testing of after-save hooks. We can't test an after-save hook with the code above, because a call to #save will error out before it ever gets to the after-save callback.

While DataMapper does not include an actual null-object style backend which turns all database operations into no-ops, it includes something close which will permit us to test after-save hooks. DataMapper ships with an in-memory store, which implements the DataMapper API in terms of a simple in-memory collection of Ruby objects. Here's an example of triggering an after-save hook with an in-memory backend:

class Foo
  include DataMapper::Resource

  property :id, Serial

  after :save do
    puts "Saved!"
  end
end

DataMapper.setup(:ephemeral, "in_memory::")

DataMapper.repository(:ephemeral) do
  f = Foo.new
  f.save
end

Running this script produces the output Saved!. This code also demonstrates the block-form of the the DataMapper.repository method, which pushes a session onto the repository stack, yields to the passed block, and then pops the session back off the stack.

So there you have it, a couple of quick and easy ways to approximate a null-database in DataMapper. I think it’s a testament to DataMapper’s design that it’s so easy to assemble the pieces it provides in ways that the writers may not have anticipated. I hope you find these techniques useful!