The Minitest Dilemma

If you were to start a new open-source Ruby project today, what testing library would you choose: rspec or minitest? Perhaps you have a strong opinion on this topic and the answer is obvious. For me, it presents a dilemma.

As a bit of background, Pakyow started on minitest and later moved to rspec. This has since been determined to be maybe a bad idea, and thus this post. From what I recall, the decision to use rspec came down to two things:

  1. More developers seemed to know and prefer rspec to minitest.
  2. Rspec appeared to have a more active community, better tooling, etc.

Number 1 was important because we wanted code contributions to be as friction-less as possible. Naturally we'd align ourselves with the tools most likely to be used by potential contributors.

Number 2 was really just personal frustration that built up over time while attempting to find and use basic tooling with minitest. It wasn't that the tooling didn't exist, but it was hard(er than rspec) to find and use due to lack of information (this has improved over the last couple of years).

For example, how would you use a code coverage tool with minitest? Search for minitest coverage and you'll find lots of people asking the same question but no definitive answer.

Before moving Pakyow to rspec, I'd only used the library a handful of times -- in some open-source contributions and on a client project that I inherited from another team. For the most part I preferred the simplicity of minitest.

Rspec was great at first. I liked the tooling and the active community. Sure, there was a learning curve, but I could always find an answer to my question. And this turns out to be a great thing given that I still spend a lot my time looking up obscure parts of rspec's DSL.


So, back to my dilemma. Minitest is simple, focused, and lightweight. Rspec is none of those things. Minitest::Spec along with a handful of extensions provides a near-rspec-like feel with less magic and complexity. I should prefer minitest. And yet I actually use rspec.

What gives? I have a hunch. Essentially, minitest's simplicity means there's less to talk about. Rspec is more visible because it's more complex and requires more discussion. Look for yourself; when searching for ruby testing library, rspec is the third result and minitest is nowhere to be found. There's a lack of visibility.

If there's a problem to be solved here, it's that the obvious parts of minitest need to be better documented. Organize the new documentation in the format of "I want to do xyz thing", with an example.

I want to see coverage for my test suite.

Add the following to your test helper:

require 'simplecov'

Given the simplicity of minitest, I see no reason that someone should need to browse rdoc to find a list of assertions (or expectations), or read the readme of an extension to learn how to use it in their project. Learning how to start using minitest should be just as easy and obvious as actually using it.

I'm thinking about contributing documentation like this at some point. Should I? Tweet or email your thoughts: @bryanp |