Testing with MiniTest

Table of Contents

Ever since I started doing TDD I’ve used RSpec. It’s a great tool, and for a long time it was part of my standard testing stack. This stack also contains things like Cucumber and FactoryGirl.

Now, this stack works great. But that doesn’t mean it’s the best, it has its issues:

Cucumber
#

Cucumber is, in almost every project, added complexity without any benefit. The idea behind cucumber is that it allows you to write your features / user stories in plain English and prove that those features are functional. The process of writing plain English, and using a boat load of steps (regular expressions, really) to convert that to executable code is tedious. Unless those cucumber features actually find their way to a customer, I don’t see any added value in writing them. Developers know how to write code, why not omit the entire English-to-code conversion entirely.

FactoryGirl
#

Factory Girl is a great way of generating object to work with. Unfortunately, more often than not, these generated object are ActiveRecord models. When creating those, you will most likely hit the database. For simple apps that isn’t as much of a problem. When an application grows, models become dependent on each other. You can’t have an order without a customer, with a valid address and now without order lines and a valid products with sufficient stock. You could mock all these, but especially with integration and feature tests, you want the whole thing.

RSpec
#

The first thing I fell in love with with RSpec was its clean syntax. The RSpec DSL allows you to write your specs in such away the are very readable. This is how it’s suppost to look:

describe MyClass do
  context "with a valid email" do
    let(:email) { "john@example.com" }
    subject { MyClass.new(email: email) }

    it "reports number of mails sent" do
      expect(subject.email!).to eq(1)
    end
  end
end

But when an application grows, so do the number of specs. And the complexity of the boilerplate needed to set the stage for your test. I’ve seen spec files over over 2k lines where a spec on line 1982 gets setup by let statements and before blocks spread over all the preceding 1981 lines. Yes, this can be resolved to an extend by refactoring the test, splitting it up and what not. but still, it’s very difficult to read a spec and know what it’s actually doing.

What’s the alternative?
#

Curiosity drove me to investigate other testing frameworks and soon I discovered Minitest. What I love about minitest is the lack of DSL (unless you use minitest/spec, of course), it also feels much faster when running tests, but I have not run any benchmarks to support that feeling.

Here’s how I’d write that same test in Minitest:

class MyClassTest < MiniTest::Unit::TestCase
  def test_reports_number_of_mails_sent
    assert_equal 1, MyClass.new(email: "john@example.com").email!
  end
end

Note that everything needed to perform this test is contained in that single test method. You could do the same with RSpec, but it’s DSL seems to prefer another convention.

Assert, like, whatever
#

What I truely love about Minitest are the simple assert methods. In essence, that’s all you need to know about Minitest: assert.

Did you ever see this in Rspec:

expect(my_car).to be_drivable

There’s quite a lot going on here. Out of nowhere you have a my_car instance, which was probably declared in a let somewhere in this file. You could assume it’s of the type Car, but who knows. You also don’t know what else was done to this instance in some before block. Furthermore, this line assumes that the my_car object responds to a method named driveable?. Yes, with a questionmark. There is nothing explicit about what’s going on here. If you’re a novice Ruby developer, you will probably have some difficulty figuring out what’s going on here. If you’re an experience Ruby developer, you will as well.

Consider the following Minitest alternative:

my_car = Car.new(wheels: 4)
my_car.fuel!(type: :diesel, litres: 60)
assert my_car.driveable?

This is plain Ruby. If you know Ruby, you understand what’s going on here. After all, your tests are meant to drive your internal design. Well, here it is, in all it’s glory.

The only method you need to know, really, is assert.

assert(test, msg = nil)

You supply assert with a test value. When true, the test passes. When false, it doesn’t.

There are quite a few other assert* methods to help you do common assertions. You don’t have to use them, as the can all very easily be written with a common assert. They are only there for convience. The ones I use most are:

assert_empty - assert the supplied argument is empty
assert_nil - assert the supplied argument is nil
assert_equal - assert the supplied arguments are equal (using ==)
assert_match - assert the supplied regex mathes with something else
assert_includes - assert the supplied collections includes a certain object

Then, for every assert method, there’s also a refute method that fail when true (instead of passing). It is that simple. You can read more about these in the Minitest::Assertions documentation.

Fixtures or Fixture Replacements
#

Fixtures are nothing more than a pre-defined set of data to run your tests against. In the case of Rails fixtures are mostly data that go into the database. They get loaded once and each test is run inside a database transaction on that dataset.

FactoryGirl and Machinist are fixture replacements in the sense that you define how your data should look and then generate what you need for each test.

Using fixtures has two major benefits as opposed to FactoryGirl:

All data is loaded only once, before the test suite runs. With FactoryGirl it’s not uncommon that, for a specific test file, you need to create 10-20 records in the database, everytime, for 20 specs. This soon adds up an is mostly what makes test suites slow.
You test against a database full of data. This may sound strange, but with time you’ll create an extensive database of data used for testing. It’s easy to add (data-wise) edge cases and see how they behave in your application.

Capybara
#

I already mentioned Cucumber for feature testing. Under the hood, most of the time, cucumber will be using capybara to emulate a client browser. Minitest can handle this as well and it works just as you’d expect:

class TestCarCRUD
  def test_branded_car_listing
    visit "/cars/toyota"
  
    within("#cars") do
      # "Toyota Corolla Verso" from fixtures
      assert page.has_content?(cars[:toyota_corolla].name) 
    end
  end
end

The minitest-rails-capybara gem might be of help here to make your setup easy.

Getting started with Minitest
#

I don’t want to go into too much detail about using Minitest with Rails. It’s already well documented elsewhere and basically all you need is the minitest-rails gem. For feature testing you’d also need minitest-rails-capybara.

The Testing Silver Bullet
#

There probably isn’t a silver bullet when it comes to testing Rails applications. I love Minitest for its ease of use, lightweightedness, explicitness, and the fact that’s just Ruby. In combination with fixtures it’s possible to write a very fast test suite.

I think that’s RSpec tries to do too much magic and DSL’ing which makes it much harder to write clean tests. As a developer I see little benefit from using a specific DSL for writing tests. Having paired up with a few junior developers, new to Ruby and Rails, they tend to pickup Minitest tests much faster than RSpec and Cucumber.

Be sure to give Minitest a try and let me know your thoughts!