Treat doctests more as executable documentation rather than as just more unit tests. Sometimes, with all the mocks and setup needed to do a good unit test, the doctest can look really ugly and unreadable; you should prioritize readability and clarity over comprehensiveness. A new developer should be able to read the doctest and have a good idea about what the function does.
Watch out for dictionary outputs, which are not guaranteed to be printed in any particular order. There are a couple ways to avoid this; what I typically do is to test that the output dictionary is equal to a hard coded dictionary. >>> func() == {‘one’: 1, ‘two’: 2}
Not every function warrants a doctest. Don’t bend over backwards trying to achieve 100% code coverage with your doctests. If that’s your project goal, use traditional unit tests to achieve most of your code coverage.
Thanks for the tips Karl, especially for the dicts, I havent thought to put a case for that.
Regarding coverage and mocks, you are right, they are meant to be mostly for pure functions and not to replace unit tests. But again, this is up to everyone, as with all tools.
Not every function warrants a doctest. Don’t bend over backwards trying to achieve 100% code coverage with your doctests. If that’s your project goal, use traditional unit tests to achieve most of your code coverage.
This is indeed the recommended approach. Yet, I'm doing something different and quite satisfying.
When I have a function not worth a test case, I'm writing doctests for it. This is perfect for internal functions: I do not pollute a module test suite with implementation details. It can also guide me on how to write the function (like TDD for smaller units) and the resulting function tends to be way better designed.
We all know those quite local pieces of code and thus never documented or tested. I always thought Doctest would be overkill there, but after experimenting, I feel it is right the opposite: doctests are the most lightweight way to document and test these units! And it proved worth the hassle: the docstrings have been very useful to explain how to use old code, rationales behind it, catch a lot of regressions and even improve the design. I wrote a post of how it works for me some time ago.
Of course, I'm not talking about the "right" way to doctest—quite the contrary, most doctesters work the other way around—but this is a possibility that a) helped me a lot and b) I'd like to see tried more.
Well, no function needs a doctest but, in my experience, it is useful to put a doctest in every function. Even the smallest private functions. For three reasons:
We get documentation. With a doctest, we have a basic explanation of how we expect the function to be used, naturally. Even if the function is not published, it will be helpful when you're maintaining code that uses the function. And it helps a lot to reuse the code.
We get some tests. The most basic behaviors of the function are now protected against unintended changes. If we have to handle a new corner case, we can add it to the doctest as well and ensure it will be preserved.
And finally, the least obvious one: we get a better function. If we force ourselves to doctest a function, we find ourselves working to make it more documentable. Some lazy decisions we could take (relying on globals, mixing I/O and processing etc.) are not that convenient anymore. Our function got more and better seams, is easier to call and is more predictable.
Again, this is my experience after trying an unorthodox approach. Would you try it to be if it is true? :)
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
I love doctests! Here’s my extra few tips:
Watch out for dictionary outputs, which are not guaranteed to be printed in any particular order. There are a couple ways to avoid this; what I typically do is to test that the output dictionary is equal to a hard coded dictionary.
>>> func() == {‘one’: 1, ‘two’: 2}Not every function warrants a doctest. Don’t bend over backwards trying to achieve 100% code coverage with your doctests. If that’s your project goal, use traditional unit tests to achieve most of your code coverage.
Thanks for the tips Karl, especially for the dicts, I havent thought to put a case for that.
Regarding coverage and mocks, you are right, they are meant to be mostly for pure functions and not to replace unit tests. But again, this is up to everyone, as with all tools.
Hey people, nice post and nice tips :)
This is indeed the recommended approach. Yet, I'm doing something different and quite satisfying.
When I have a function not worth a test case, I'm writing doctests for it. This is perfect for internal functions: I do not pollute a module test suite with implementation details. It can also guide me on how to write the function (like TDD for smaller units) and the resulting function tends to be way better designed.
We all know those quite local pieces of code and thus never documented or tested. I always thought Doctest would be overkill there, but after experimenting, I feel it is right the opposite: doctests are the most lightweight way to document and test these units! And it proved worth the hassle: the docstrings have been very useful to explain how to use old code, rationales behind it, catch a lot of regressions and even improve the design. I wrote a post of how it works for me some time ago.
Of course, I'm not talking about the "right" way to doctest—quite the contrary, most doctesters work the other way around—but this is a possibility that a) helped me a lot and b) I'd like to see tried more.
When you say'a function not worth a test case', what makes it need a doctest case?
Well, no function needs a doctest but, in my experience, it is useful to put a doctest in every function. Even the smallest private functions. For three reasons:
We get documentation. With a doctest, we have a basic explanation of how we expect the function to be used, naturally. Even if the function is not published, it will be helpful when you're maintaining code that uses the function. And it helps a lot to reuse the code.
We get some tests. The most basic behaviors of the function are now protected against unintended changes. If we have to handle a new corner case, we can add it to the doctest as well and ensure it will be preserved.
And finally, the least obvious one: we get a better function. If we force ourselves to doctest a function, we find ourselves working to make it more documentable. Some lazy decisions we could take (relying on globals, mixing I/O and processing etc.) are not that convenient anymore. Our function got more and better seams, is easier to call and is more predictable.
Again, this is my experience after trying an unorthodox approach. Would you try it to be if it is true? :)