Hacktoberfest Begins!

Hello fellow spooky hackers! 🎃 This is my first post to Dev.to, so I'll introduce myself. I'm Grant Shangreaux, a developer with Unabridged Software, and long time Free Software enthusiast.

In the spirit of contributing back to the software community this season, I'm spending time working on some of the tools I use on a daily basis here at Unabridged. I spent many years as a user of free software before I made any code contributions. It can be hard to know where to begin, and it also feels a bit scary 👻 to put yourself out there and try to work on someone else's code.

I hope these articles give you ideas for getting involved with #hacktoberfest. Even a small free software contribution goes a long way toward improving the public good.

Getting Started

It took many years to feel confident enough to even think about modifying someone else's code. I didn't know where to start. Eventually, I realized that I'd been "hacking" on a free software project indirectly for years.

Look At Your Tools

There's dozens of free software programs I use every day right under my finger tips. My primary tool is Emacs, which is extended by many packages written in Emacs Lisp, all of which are free to read and modify. I have found this ecosystem to be the easiest place to make contributions, both in code and by offering help to people in the community.

Contributing to tools you use is a good idea. You have the benefit of being familiar with how they work, and more importantly, where they don't work. Find an issue on the GitHub repo of a tool you use, and see if you can reproduce it! Even this can be a valuable contribution to a project maintainer, and its where you will start if you hope to fix something.

For my first PR this #hacktoberfest, I ended up choosing minitest-emacs.

Why minitest-emacs?

Many of our projects at Unabridged use Ruby's MiniTest for Rails development. I've been annoyed by a bug where verifying a single test from a file was randomly not working. There was no error message, but no test would actually run. On the GitHub repo, I discovered someone else was having the same issue, and figured I would give it a shot.

The minitest-emacs project has several things that made me want to choose it for contribution:

1. It's a tool I use. I already have in mind features I'd want to implement, and bugs I'd like to fix.
2. Small code base just a few files, including tests. Tests are a big bonus, not every Emacs package has tests.
3. Ruby/Rails related, which ties into my day-to-day work at Unabridged.
4. Built with Emacs Cask package management tool.
5. It looks like it could use help. There have not been any updates recently.

The number one thing is finding some place to start. The familiarity I already had with minitest-emacs is the main reason I chose it. The first three items made the project very familiar and accessible. Number four was a sign that building the package myself should be easy. Having tooling to build the package yourself is immensely helpful. The last element of the list could also be a reason not to open a PR. However, I see it as a place where my community of Emacs/Ruby/MiniTest users would benefit from some help. At the very least, I can begin to find out the status of the project.

Fixing a Bug 🐛 Broken Verify Single Test Command 😭

The Cask file made it easy to install development copies of the package dependencies, and execute the test runner. Verifying that all the tests are currently passing, I can now begin to hunt down the 🐛.

Reproducing the Bad Behavior

Inconsistency

First things first, we have to try to reproduce the undesired behavior. I opened up a Ruby test file and tried to run the verify-single command with minitest-emacs:

bundle exec ruby -Ilib\:test\:spec .../vacations_controller_test.rb -n/it_can_get_the_new_vacation_form/
Run options: -n/it_can_get_the_new_vacation_form/ --seed 62232

# Running:

*

Fabulous run in 0.897869s, 1.1137 runs/s, 2.2275 assertions/s.
1 runs, 2 assertions, 0 failures, 0 errors, 0 skips

Ohhh kay… it worked. That is not helpful.

Let's try a different test

I'd like to say that I did something strategic here to reproduce the bug, but I just moved to a different test and tried again:

bundle exec ruby -Ilib\:test\:spec ../vacations_controller_test.rb -n/_create_with_valid_params_makes_a_new_record/
Run options: -n/_create_with_valid_params_makes_a_new_record/ --seed 140

# Running:


Fabulous run in 0.002342s, 0.0000 runs/s, 0.0000 assertions/s.
0 runs, 0 assertions, 0 failures, 0 errors, 0 skips

Ah ha! This is the bugged behavior I was looking for. Now… what clues can I get from these two different test runs? Is there some hidden state that is breaking?

In the original issue, it was reported that the name generated by minitest-emacs for the -n option is incorrect. I noticed in the second test I ran, the test name generated begins with an underscore, which is odd. I got lucky 🍀!

In the Ruby test file, the name string was "#create with valid params makes a new record". The # character is being replaced with _ by minitest-emacs when generating the -n command line option. Deleting the # and rerunning the single test seems to fix the issue.

Now... onward to the source!

Making My First Test Fail

Happily, this is a project that includes unit tests 😃 . Because of this it was easy to trace down a function that was replacing both the "#" and ":" characters with "_".

(ert-deftest test-minitest--post-command ()
  (defvar test-description "#method_name behavior of Module::Class")
  (defvar method-name "_method_name_behavior_of_Module__Class")
  (should (equal method-name (minitest--post-command "test" test-description)))
  (should (equal method-name (minitest--post-command "it" test-description))))

This is exactly the observed behavior that is causing the tests not to run. It seems likely that MiniTest itself has changed, because at some point this test was written to describe the expected behavior of replacing those characters with underscores.

I modified the test to expect that the "#" and ":" characters will not be replaced with "_", i.e. the method-name variable is "#method_name_behavior_of_Module::Class"

cask exec ert-runner -p test-minitest--post-command

F

Ran 1 test in 0.168 seconds
1 unexpected results:
   FAILED  test-minitest--post-command

Making My First Test Pass

Simply removing some characters from the regex makes it pass.

Before

(defun minitest--post-command (cmd str)
  (format "%s" (replace-regexp-in-string "[\s#:]" "_" str)))

After

(defun minitest--post-command (cmd str)
  (format "%s" (replace-regexp-in-string "[\s]" "_" str)))

Result

cask exec ert-runner -p test-minitest--post-command

.

Ran 1 test in 0.000 seconds

Now some QA testing

Before opening a PR, I did some research I hoped would help the maintainer. For one, I updated my local minitest-emacs package and saw the buggy behavior go away with manual testing. But I had a few questions remaining.

1. Why is this happening?

   There must have been a reason that the method (#) and class (:) syntax markers were being replaced with underscores when the code was written. It would be good to find out what may have changed with MiniTest itself to identify this.

   I looked through the MiniTest changelog (another great thing to have in a public project!), but could not find anything directly related. However, it did seem like there had been changes to how the -n command line option was being parsed. At the very least, I can mention this in the PR.

2. Why is the behavior different than the issue report?

   The issue report is for the same broken behavior, but the cause is not what I fixed. My assumption again here is that changes to the MiniTest gem itself is the reason for this. It might be necessary to identify which versions Ruby and MiniTest are supported in order to determine what the proper behavior of minitest-emacs should be. I made a reply on the original issue, to see if the author is still around to provide more context.

Making a Pull Request

Since the maintainer may have more context for my questions above, I thought I would go ahead and make the pull request as-is. I made sure to include the versions of Ruby and MiniTest I was testing with. Since the repository has not been updated in some time, I would like to see if the activity brings any action. If it does not, this may be an opportunity to see what it takes to help maintain the project.

Meanwhile, I can do some more testing and investigation into the behavior with other supported versions of MiniTest and Rails, and look into other issues that could use some help!