Configuring database_cleaner with Rails, RSpec, Capybara, and Selenium

If you write Rails code, or any Ruby code that interacts with a database, and you also write automated tests, chances are you have heard of or used the database_cleaner gem. It’s a terrific gem that abstracts away the various ORM APIs for getting the DB into a “blank slate” state.

Periodically I start a new project using Rails, with RSpec, Capybara, and Selenium for acceptance testing, and a short way into it I find myself banging my head against bizarre inconsistencies with the test database. I’ll set up some records in the test DB, only to have the Selenium-driven browser-based tests act like those records never existed. Eventually, I’ll realize what I did wrong and curse my feeble brain for not remembering the last time I solved the same problem.

The problem is always the same: the tests are being wrapped in database transactions, so any code running outside the actual test process (like, say, a server process servicing a Selenium-driven browser request) does not see the database fixture I’ve so carefully assembled.

I just walked a pairing client through the same fix, so in the interests of remembering the steps, and hopefully preventing some other folks from tearing their hair out, here are the steps needed.

First of all, and this is very important, go into spec/spec_helper.rb and change this line:

config.use_transactional_fixtures = true

To:

config.use_transactional_fixtures = false

This will disable rspec-rails’ implicit wrapping of tests in a database transaction. Without disabling this, none of the following configuration will matter.

Now configure database_cleaner. I usually create a separate file called spec/support/database_cleaner.rb for this. Inside, I put something like this:

RSpec.configure do |config|

  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

Let’s take that step by step.

config.before(:suite) do
  DatabaseCleaner.clean_with(:truncation)
end

This says that before the entire test suite runs, clear the test database out completely. This gets rid of any garbage left over from interrupted or poorly-written tests—a common source of surprising test behavior.

config.before(:each) do
  DatabaseCleaner.strategy = :transaction
end

This part sets the default database cleaning strategy to be transactions. Transactions are very fast, and for all the tests where they do work—that is, any test where the entire test runs in the RSpec process—they are preferable.

config.before(:each, :js => true) do
  DatabaseCleaner.strategy = :truncation
end

This line only runs before examples which have been flagged :js => true. By default, these are the only tests for which Capybara fires up a test server process and drives an actual browser window via the Selenium backend. For these types of tests, transactions won’t work, so this code overrides the setting and chooses the “truncation” strategy instead.

config.before(:each) do
  DatabaseCleaner.start
end

config.after(:each) do
  DatabaseCleaner.clean
end

These lines hook up database_cleaner around the beginning and end of each test, telling it to execute whatever cleanup strategy we selected beforehand.

And that’s it!

Note that this is all for RSpec, and does not cover Cucumber configuration.

Hopefully this will help someone else out there avoid the frustrations I’ve run into!

EDIT: A few people have asked me why I don’t just force all threads to share the same ActiveRecord connection, as demonstrated in this Gist. A few reasons:

  • Using database_cleaner implies that I want ORM neutrality. database_cleaner supports ActiveRecord, DataMapper, MongoMapper, and others. The solution linked above only works for ActiveRecord.
  • It’s a monkey-patched kludge which will only work so long as AR refrains from changing its connection-sharing internals. And frankly I’m not sure I trust it to work across all Ruby VM and database combinations (UPDATE: And indeed, I’ve now seen two different people say there are race conditions with the current Postgres adapter). I’d be more inclined to use it if ActiveRecord had a published configuration option which was known to work in all contexts.
  • As I stressed above, I’m careful to set things up so that only the tests that need them fall back to truncation. Since :js => true tests generally don’t form the bulk of my suite (and since they are unavoidably slow anyway, due to the overhead of driving a browser), I’m not overly concerned about the added overhead. Perhaps if all of my acceptance tests drove a live browser I’d be more worried about it.
  • In cases where database truncation is taking up a significant amount of test time, you can usually speed things up with some judicious control of which subset of tables get truncated for a given test. This is something database_cleaner makes pretty easy. Maybe that would make a good topic for a followup post.
  • UPDATE: Oh yeah, and as @donaldball points out, sharing a transaction between test and test server means acceptance tests don’t run quite the same as they would in production. Specifically, they’ll never trigger after_commit hooks.
This entry was posted in Rails, Ruby and tagged , , , , , . Bookmark the permalink.
  • http://twitter.com/na43251 Edgars Beigarts

    Why didn’t you just share activerecord connection between threads – https://github.com/jnicklas/capybara#transactions-and-database-setup ?

    • http://avdi.org Avdi Grimm

      I’ve added a section at the end addressing this.

  • http://subelsky.com Mike Subelsky

    I’ve screwed this up a few times – thanks for writing the definitive guide!

    • http://avdi.org Avdi Grimm

      You’re welcome!

  • http://twitter.com/soulcutter soulcutter

    This is one of those things I take for granted, but get bitten by on occasion, so nice work identifying it and writing it up.

    • http://avdi.org Avdi Grimm

      :-)

  • Pingback: A Smattering of Selenium #116 « Official Selenium Blog

  • Praful

    Nice consolidation! :)

  • manivannan

    does not work with multiple database connection with activerecord

  • http://twitter.com/topherfangio Topher Fangio

    Thanks! This is really useful. I was doing something similar, but I believe your approach works better and is a little bit clearer to use!

  • http://twitter.com/andycamp andycamp

    Hi, Thanks for the post. I’ve been struggling with some database issues in my tests, and this seems to help, but my specs no take 4 x’s longer. It is a pretty large test suite, but 20 minutes seems prohibitive. Got any advice?

    • http://avdi.org Avdi Grimm

      Do you have a many short capybara tests?

  • moonfly

    Check also http://gist.github.com/moonfly/4950750.
    I really like the code in this post (actually more than the one in the gist), but it can be optimized slightly further if you need that extra bit of performance.

  • Colin Bradley

    Also worth adding that it’s important, that in the config you put the before(:each, :js => true) block above the DatabaseCleaner.start block (as it is in the example) – I just spent a while figuring out why my capybara tests were hanging, and this was the problem (clearly setting the strategy after starting the cleaner doesn’t work so well…!). Thanks a bundle for the post though, extremely helpful.

    • Rafael Souza

      How did you worked it around?

      • Colin Bradley

        I just had it the wrong way around – if you put the before(:each) { DatabaseCleaner.start} block above the before(:each, :js=>true) block, it starts the database cleaner before applying the truncation strategy. Which is bad. If you follow the configuration in the post above you shouldn’t have any problems.

  • Pingback: Google Reader分享 » AngularJS / Backbone Capybara Integration Tests

  • http://twitter.com/KingPong Philip Garrett

    Very useful! Thanks

  • Toby Ovod-Everett

    I decided I wasn’t entirely happy with relying on the :js tag to determine the strategy, so I modified the code to use Capybara.current_driver to determine if :rack_test was being used. This permits a single before(:each) block instead of the three listed above. See https://gist.github.com/tovodeverett/5817365 for a synopsis.

    One note – both Avdi’s approach and my approach can fall victim to an issue with RSpec < 2.14 where before(:each) blocks activated by an include get called before those activated from config.before. See https://github.com/rspec/rspec-core/issues/903 for one of several issue reports that ended up all being resolved by https://github.com/rspec/rspec-core/pull/845.

  • jonathanspooner

    If your using FactoryGirl sequences you’ll need to reset them when you use the truncation strategy.

    config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
    FactoryGirl.reload
    end

  • Timon Vonk

    If you’re running into stupid deadlock or other transaction errors because after hooks with databasecleaner are being run before the server finishes whatever it’s doing, try this! https://gist.github.com/timonv/7026067

    The trick is to force the driver to wait till the page finishes loading.

    • dkannan

      thanks

  • JoshuaMuheim

    You could dry up your code a bit using an around filter:

    config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
    end

    config.around(:each) do |example|
    DatabaseCleaner.strategy = example.metadata[:js] ? :truncation : :transaction
    DatabaseCleaner.start
    example.run
    DatabaseCleaner.clean
    end

  • Case Taintor

    Thanks for this great post! We started to get a lot of random deadlock errors on a few different projects we have, so I wanted to update you a slight change that I think will help people avoid having this issue.

    We ended up changing your:

    config.after(:each) do
    DatabaseCleaner.clean
    end


    to:

    config.append_after(:each) do
    DatabaseCleaner.clean
    end

    This is because, as it was, DatabaseCleaner.clean was being run before Capyara was resetting the driver, meaning that you could have situations where there’s an active connection while you’re trying to truncate the tables. By changing it to an append, it means that the Capybara.reset! that is added in capybara/rspec is called before your DatabaseCleaner.clean call.

    Capybara 2.2 fixed an issue where calling Capybara.reset! wouldn’t wait for the driver to reset before continuing, which could have been a cause for the deadlock. So, combine Capybara 2.2 with the above change and you should be golden.

    • http://avdi.org Avdi Grimm

      Thanks for the note!

    • Guest

      You just saved me. Thanks

    • Pablo Acuña R.

      thanks!!!

  • cezinha_anjos

    Great post. Thank you!

  • Dave Aronson

    I pretty much ignored this post when it came out. Now I’m splitting an extant app apart into API and UI halves, and trying to test the UI half. This is exactly what I need, so that the API server will see the items created by the UI-side tests! Thanks once again Avdi!

  • dontfidget

    As far as I can tell, this methodology causes rspec reload the fixtures before each test, even when the database cleaner is using the transaction strategy. Needless to say, this is really bad for test performance. Is this expected? Can you recommend a way to load the fixtures only once (or, I suppose, have them only get reloaded after tests with the truncation strategy)? Thanks.

  • leogcrespo

    Great post Avdii! This was a current problem I was having (had some Postgres race conditions in my continuous integration build) due to the ActiveRecord connection sharing, and using database_cleaner this way, together with the tip of appending the each hook for cleaning the db, worked like a charm.

  • http://alexvpopov.github.io/ Alex Popov

    Hi Avdi,

    Great post, I am also a great fan of Ruby Rouges, so thank you for all that. I have two questions:

    1. Instead of setting config.use_transactional_fixtures from true to false, couldn’t I just delete the line config.use_transactional_fixtures = true altogether. In other words – what is the default value? If the default value is true, I wonder why rspec creates this line by default at all.

    2. If I have a feature with several scenarios, if I want to create some records once (because I don’t manipulation the actual records, but test some things like that there are exactly 5 records per page, there is pagination, pagination works etc.), I can’t use rspec’s before(:all) hook, as config.after(:each) { DatabaseCleaner.clean } will detroy these records. Is there a way to tell database_cleaner gem to not clean the db just for a certain scenario?

    Cheers,
    Alex

    • http://avdi.org Avdi Grimm

      1. I always prefer explicit configuration, especially when I’m doing something that may go against people’s expectations (regardless of what the RSpec default is).
      2. Probably, but I don’t know offhand. Sorry.

  • Robert Fletcher

    I’ve referred back to this post a lot, but recently discovered that I was using truncation globally, even though I thought I was only doing it on my feature specs. Just tossed up this gist of how I fixed the situation:

    https://gist.github.com/mockdeep/9904695

    Rspec runs `before` hooks in the order they are defined, and `after` hooks in reverse order, so I opted to explicitly use `prepend_before` and `append_after` to avoid confusion. Also, I added a couple of methods to help debug database cleaner. Unfortunately, they don’t have a neat API that I’ve found for finding out what strategy is active.

    • Robert Fletcher

      Another side note, I use :type => :feature instead of :js => true since we don’t always need or remember to tag our specs as using javascript.

  • Abhay Agarwal

    great post. thank you

  • http://www.bennyklotz.at/ BennyKlotz

    Hey there!

    Great post and very useful for setting this up with rspec :)

    Got another problem in setting this up with “native” rails testing (minitest)

    When I use the ActiveRecord Connection Gist I get this weird error:
    ActiveRecord::StatementInvalid: Mysql2::Error: This connection is in use by: #

    And for the database_cleaner attempt, I turn off transactional fixtures


    class ActionDispatch::IntegrationTest
    self.use_transactional_fixtures = true

    end

    In my Feature file I then use:


    class Feature < ActionDispatch::IntegrationTest
    def setup
    DatabaseCleaner.strategy = :truncation
    DatabaseCleaner.start
    end

    def teardown
    DatabaseCleaner.clean
    end

    But this isn’t working, Fixtures / Factories created in a before block for the feature test aren’t available for capybara / webkit.

    You got any hint how to solve this?

    Cheers benny

  • jbnunn

    Wow Avdi, thank you so much for this–been struggling with this exact issue for 3 hours. Fixed now.

  • Luka

    Beautiful! Thanks!

  • Jeffrey Fulton

    Thank you, thank you, thank you!!!!!