I came across a design question when working on elm-test recently:
What if you could write tests in the same file as the code they're testing?
There are a few testing systems that support this:
-
clojure.spec(to vastly oversimplify) lets you define tests inline in the same file as your business logic. - Rust allows putting tests in the same file as the code being tested, although the documentation recommends putting them in a separate file (without elaboration as to why).
- Doctests (such as in Python, Elixir, Go, and Haskell) allow writing tests in the same file—theoretically for documentation purposes, although potentially for testing purposes instead.
-
EUnitin Erlang runs all functions in a given module that end in_testas tests, including business logic modules. - Racket supports inline tests.
- Pyret supports "testing blocks" which can live in any source file.
-
Test::Inlineadds support for this to Perl.
I've never used any of these, but I'm wondering how the experience would be in practice. Is it amazing? Unexciting? An outright bad idea? What are the pros and cons compared to putting tests in a separate file?
If you've tried writing tests like this before, what did you think?
Latest comments (17)
So Richard, did you come to a conclusion?
In all my Java projects, tests live in a separate tree - src/main vs src/test. In my node projects, tests live in separate files next to the file being tested.
In the Java case, it's trivial to have extra test code that is there to support your tests. It's also easy to see what is test code and what is production code.
In the Node case, it originally started because of the require syntax - when I did it with separate trees I had hellish require paths. However, it's kinda nice having them so close together. You can see what is and isn't tested trivially, and flip between them really easily.
I like the idea of having them in the same file, but haven't tried it out properly yet.
Basically I agree on the "it depends" argument that was given already. However, that approach can only work in a 1:1 mapping. Hence, not with integration/system tests.
I think it's much more interesting, to research better tooling for production/test code. One thing I'm particularly working on is a ReSharper plugin (github.com/matkoch/TestLinker) that provides navigation between the two entities, plus a few other things, like generating test stubs and detection of affected tests through production code changes.
Kept in separate source files, but I always put them right next to where my source code lives. They should be immediately discoverable right there, not relegated to some far away distant corner of the project.
C does fine with #ifdef
Separate file for more involved tests.
D has them integrated in the language:
dlang.org/spec/unittest.html
And allows them to be put in source code. Interestingly enough they are (can be) executed when program runs. Enforcing this looks like a good idea, since the tests must pass for the program to be run (which is perhaps more important than the place where you put the test code, although OT).
Related: what about making sure the comments stay correct and up to date in the code. Not really useful to test the code, but could be extended to do so. github.com/bahmutov/comment-value
I personally love doc tests in Elixir. They're not good for things that require a lot of setup because their repl based. I think it's great because it helps you remember to write them and keep them up to date. Also helps them act as documentation. Especially if they are example based tests.
I write a lot of Rust (and Elm!) and having the option to write tests in the same file comes in handy sometimes, even if it's just temporary. I tend to move my tests into a separate file eventually, but if it's just a test or two, sometimes the extra file/boilerplate isn't justified.
One of the nice things about putting tests in the same file is that naturally you have access to all the module's non-exported symbols and can test internals if desire. You get this with Rust either way since the test module is typically a conditionally-compiled child module of the thing you're testing, but it seems that with Elm this wouldn't be possible otherwise.
Speaking of conditional compilation, how would putting tests in the same file work in Elm? It seems that test logic would be included in the compiler output unless some sort of conditional compilation mechanism was added.
Prefer keeping them separate, makes it easier to tell what is being changed at a glance at looking at a change request.