Lessons Learned in Chasing Unicorns

I'm old.

I've been doing open source for a long time. I started in 1996, during my first year of high school. I was always eager find the perfect solution, the most elegant code, and I would even go out of my way to try and anticipate how folks would use the things I wrote. There were some results I found, most of them weren't great.

• My open source contributions took so long to do, sometimes people got there before me even though my idea was "better"
• Sometimes I ran out of time chasing that perfect unicorn that I lost interest and abandoned the project
• Because I had invested so much emotional energy into the work, it was very upsetting when I received criticism or notes, even if they were offered with sincere appreciation and good intent
• Because I spent so much time thinking about perfection, I was overly critical of others

With time, I came to learn some important truths.

• There is no such thing as perfect, and that's for the best
• You don't know how users will use things until you ask, and you should ask
• Peer review is good, and it's important to receive it well so that people will keep giving it to you
• Future compatibility and "future-proofing" is almost completely a fool's errand

Thoughts and Feelings

So now we come to the question of why my mind is in this place. I just finished a major release of an open source component that I work on which helps you publish data to an API. The major improvement is that I added support for a second version of the JSON schema that the API uses, and it was a combination of large and small changes. The part that had me waxing philosophical, however, was the thoughts and feelings I was experiencing while performing this massive re-factor.

I tend to be highly critical of myself—a trait which I know many of us share. In writing this post, I didn't start off wanting to talk about feelings; instead, I wanted to go through the technical challenges associated with adding version support. I still may write that article, but as I was going about writing it occurred to me that we very seldom discuss our feelings about our work, and I'd like to change that.

Feelings are important, they're part of the human experience, and we need to talk about them if we want to improve.

Talking to Myself

Below is a series of thoughts I had, and my own answers to them. I hope this is helpful for you, too.

"Why didn't you add version support from the start?"

The number one frustration I had with myself is why I hadn't added version support proactively. I mean, we all know that as time passes, we will see new versions of schemas emerge. Why didn't I build versioning support in from the get-go?

Well, there are so many things I could have messed up if I had done this:

• I could have done a poor job anticipating which aspects of the schema would change
• I could have added so much bloat to the project in the anticipation of new versions that project was less useful
• I could have taken so long in the initial version that my intended audience moved on without me

The right time to add support for something in your program is when you have a clear idea of what you need. Guessing will, more often than not, add bloat, jeopardize quality, or risk an even larger re-factor when you're wrong.

I was right to wait, even though I didn't like it.

"This should have been X so this task would be easier!"

I second-guess myself a lot. When I was younger this would often lead to decision-paralysis, or a never-ending cycle of re-factoring. There are things that we can do to make re-factoring easier:

• Always have decent test coverage, that way you know what breaks when you make changes
• Really learn and internalize semantic versioning so that you're more cognizant of breaking changes in refactoring
• Separate your code by the job it does, making each piece only as large as it needs to be to perform its own function
• Write good documentation; this will help you really think things through

However, past me didn't louse things up by not having done what present me thinks I should have done now. Time machines do not yet exist (probably never will, but that's another post), so there's no way that past me could possibly know what present me thinks I should have done.

Beating myself up about this imagined discrepancy will not help me, and it most certainly will not help my project.

"Nobody wants to do this, they want to do that."

With the project I'm working on, I am enjoying a massive and joyous privilege: I'm getting help and feedback from people who are involved. This is an open source project with significant corporate moral and logistical support, but it didn't start that way. When I started, I had only an idea. People like ideas, people will talk about ideas, but when it's time to make business decisions about open source projects, folks want a working project and not just an idea.

Before I had my first prototype, nobody knew whether this was a good idea. I even wrote a section in my project's documentation explaining why I was doing the project, because I expect folks to ask that very question. How was I supposed to seek out information on how people were going would use the project if I was already anticipating the question of whether it was worthwhile to begin with?!

Instead, I did my best in thinking of how I would use it. Sure, I asked questions about what others thought, and I incorporated their feedback as best I could, but at the end of the day we need to trust ourselves to understand the problem well enough to solve it. That's hard to do sometimes, though.

"Maybe we don't need that test, it'll take too long to write."

Expediency is frequently the enemy of working software. Wanting to "finish" things is important, and it does feel great to say something is "done." However, if we don't really hold ourselves to our own standards for quality, is it done? Really done?

My module enjoys roughly 90% test coverage, and I like it that way. I remember having this same thought, and this same feeling of impatience, when I first wrote the prototype and the initial release. I remember the discomfort of insiting that I write the tests anyway, and it really paid off. The tests I wrote in the initial version made the second version so much easier to write.

You almost always want to write that test.

That's Enough of That

Talking about thoughts and feelings is hard, and in my years it has frequently been discussed as a weakness. Everybody intrinsically understands that reason and emotion are tied, and that they can work together to make life better. There is no living person who is contemporaneously awake, self-aware, and operating outside of emotion.

I hope you will take my thoughts on this and use them to improve your own awareness of the thoughts and feelings you experience, how they can affect your work, and how they affect the world around you.