The story starts when I began using Larder recently, and while I loved the mobile app and browser integrations, I really just wanted to use it from the command line too. I checked the roadmap and found that a CLI was suggested back in 2016, but there had been no other motion on that effort. A quick glance revealed they had a nice and simple API, so I popped open a terminal and got to work.

I've been writing more and more Ruby lately, and I've been loving it! I thought this would be a fun time to become a publisher of ruby gems and not just a consumer, so I set out to write the project in Ruby. (Also, I heard great things about a few key Ruby libraries that would make this effortless, but we'll get to that.) It wasn't long before I had a first commit that printed out info on a user based on a provided token from a yaml file. Not half an hour later, I had added lard folders and pretty-printed them too. I'll post pictures of the output later, but for now, you can see this on the project's readme:

It started simply:

# lib/lard.rb
require 'httparty'
class Lard
  include HTTParty
  base_uri 'https://larder.io'
  def initialize(token)
    @options = {
      headers: {
        'Authorization' => "Token #{token}"
      }
    }
  end
  def user
    res = self.class.get '/api/1/@me/user', @options
    JSON.parse res.body, symbolize_names: true
  end
end

# ...
# bin/lard
#!/usr/bin/ruby
require 'yaml'
require './lib/lard'

config = YAML.load_file 'lard.yml'
lard = Lard.new config['token']
puts lard.user.inspect

But hold up a second - am I really calling it "Lard?" What, am I 5 years old? Well... sort of. I actually have some decent reasons though, so bear with me:

• I'm not associated with Larder, so this is third party and can't use their name as my own
• It's a command line app, so the less typing, the better - short names rule!
• It's a Larder client first and foremost, though, so the name should be memorable
• And, again, I'm five years old, so I think it's just silly enough that its other merits stand strong to make it a good name

I could tell this was going to be fun, and not too stressful, so I was sold on keeping the project and making it feature-complete with regards to what their API provided as quickly as possible. So, to make sure I wouldn't step on anyone behind Larder's toes, I shot them a tweet asking if it would be alright to open-source my work (my repo was private, at the time).
They very kindly agreed... so I was off to the races!

I wrote a little bit more code to fetch bookmarks from a folder, broke my ruby install by mucking about with rvm and rbenv and finding they had collided on my mac, and finally went to bed.

The next morning I woke up early to work on the project and fix my ruby install (exciting, right!?) and finish fetching bookmarks. Eventually I ran off to work, but when I came home, I spent all evening fixing the ruby install to compile with openssl as needed, and got work late that night since I was too hyped to sleep. I only coded an hour that night, but I got a lot done as I was able to pretty-print out an entire folder of bookmarks, paginating with the Larder API and all.

The next day, I decided it was time to quit writing my own CLI kit and adopt such a wildly popular one, Thor. It was super nice, and this was where I really started learning in this project. I had already learned how to make basic HTTP requests with net/http and HTTParty, but now I was learning a speedy and nice way to write command line interfaces in Ruby without breaking a sweat.

I also almost admitted I was doing all this on a mac by committing the .DS_Store file. 🤫

Then, I published Lard 0.0.1, the first version of the gem! Woohoo!! SPOILER ALERT: I didn't know it at the time, but gem install lard didn't bring along Lard's dependencies, so don't bother installing this particular version...

Finally I made a hackjob at posting a bookmark (which turns out you can post and edit at the same endpoint, it matches based on the URL you provide). But it didn't work yet - I was seeing an HTTP error 301: Redirection, and my POST request was getting an error back:

{"detail":"Method \"GET\" not allowed."}

But I was sending a POST request! I spent a long time debugging this. After a while I cracked open wireshark and saw that I was encountering a 301: Redirection and sending a GET request to the redirected URL. Now that was odd. I didn't know why that would happen. Normally HTTP requests follow through 300's like they weren't even there, right?

Well, not this request. It was late, I was tired, and I just wanted this to work so I could start using the CLI myself and not just developing it. I decided to get some air, went on a walk, picked up a milkshake to ruin whatever few calories I'd managed to burn on the walk. I chatted with my friend and compiler partner Cameron about the issues, and found the solution on my walk home. I was so burned out I didn't want to write the code for it though, so I opened an issue for it and went to bed.

Remove use of HTTParty #4

hawkins posted on Oct 26, 2018

Honestly, this lib has been something of a headache with its less-than-stellar docs and presumptuous api to say the least, so I've boiled the HTTP calls down to something pretty simple such that we can hopefully remove httparty pretty easily.

On top of all that, I'd like to get back support for ruby < 2, I think I can justify replacing httparty with net/http or another lib that's a bit lower level and with lower dependencies.

Concerns:

• 3xx codes
  • If I remember right, POST https://larder.io/api/1/@me/* hit 301's to somewhere, so I need whatever interface I scrap together to follow redirects and maintain HTTP verbs throughout redirections.
• Easily infer errors
  • res[:error] is nice - I don't want to lose that semantic response if possible

View on GitHub

The next morning, I was cooled off and wrote the patch to fix httparty (yes, it was one stupid function call to change an option otherwise not exposed and one that seems contrary to HTTP 300 semantics). Now I don't want to trash talk this library - other than this gripe, it was super nice. But this hobby project should be fun, and I don't want headaches - I get enough of those living with chronic migraines! So just know that if you use HTTParty, that's perfectly fine, but this very minor gripe was enough to give it a bad taste for me. I have the luxury of making very simple requests in this project, so going back to net/http where I know exactly what I'm getting isn't a big deal at all.

I kept using HTTParty for now, but I knew it wouldn't be here to stay much longer. I wanted to do fun things, features, not refactoring. So later that night, wrote lard tags, lard search, and lard login.

I tweeted back to Larder now that I had something to show, and they were excited!

That got me way too hyped, and I really needed to go to bed, so I kept tagging and publishing new versions between features hoping that would make me stop and rest, but alas:

Bump 0.0.4  … committed 4 days ago

This should've been 2 at this point but I kept thinking I'd stop and go
to bed, but here we are. A few bugs and features later, and this is it.

then only 8 minutes later:

Correct nil check  … committed 4 days ago 

I said I was done for the night, but I have no self control when it
comes to side projects

Finally after this, I published version 0.0.5 (still not installing gem dependencies with it, mind you) and went to sleep.

The gang at Larder told me the API limits folder requests to 200 items per request, so I applied that in my code as soon as I got a chance to the next evening. Then I did lots of refactoring type things to make working in the project feel less-icky, starting with error messages. Then I learned about the magic that is rubocop,and I desperately questioned how I ever wrote large scale ruby projects without it. Wow! Nicest linter I've seen in a long time. My pet language, Druid, has a lot to learn from this for it's "friendly compiler" effort...

Most excitingly, I set up Travis CI! If you're not familiar with Travis, Travis CI is a super awesome and free continuous integration platform that integrates insanely well with GitHub and practically every popular language and software packaging platform like RubyGems. Perfect for running tests, linting, and deploying updates to your package automatically when you open a PR or tag a release! Check out Lard's status page on Travis where you can see the latest builds it has ran, what triggered them, and whether tests are passing/failing!

The next day, October 26th (6 days straight working on Lard!), Josh Sharp from Larder opened an issue which caught me by surprise:

Missing dependencies? #7

joshsharp posted on Oct 27, 2018

Heya, I had a go at installing and running Lard yesterday, and I ran into some issues. I'm not a Ruby person, but I think the issue here was that its dependencies were not being installed when I installed the gem.

The first time I ran it, I got an issue about httparty missing, which I figured out how to install as a gem myself 😇 but since then I'm still getting an error:

hades:~$ lard
Traceback (most recent call last):
        6: from /home/hades/.rvm/gems/ruby-2.5.3/bin/ruby_executable_hooks:24:in `<main>'
        5: from /home/hades/.rvm/gems/ruby-2.5.3/bin/ruby_executable_hooks:24:in `eval'
        4: from /home/hades/.rvm/gems/ruby-2.5.3/bin/lard:23:in `<main>'
        3: from /home/hades/.rvm/gems/ruby-2.5.3/bin/lard:23:in `load'
        2: from /home/hades/.rvm/gems/ruby-2.5.3/gems/lard-0.0.5/bin/lard:3:in `<top (required)>'
        1: from /home/hades/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/home/hades/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- paint (LoadError)

If I'm doing something wrong, please let me know! Otherwise hope this helps you fix an issue.

View on GitHub

Woah, one of the two devs from Larder tried my project? Awesome!! And it failed catastrophically, OOOOOPS!! I was so embarrassed, but I was excited because he'd clearly found an issue that I could learn from! That's why I love open sourcing everything I do, because someone is bound to have a different experience, and I LOVE learning from these opportunities!

Right away, I had a theory:

Comment for #7

hawkins commented on Oct 27, 2018

Hey thanks so much for trying it out and reporting an issue! 🎉

This is actually my first published ruby gem, so I'm guessing I didn't mark the Gemfile which lists our dependencies (httparty, paint, thor) to be included in the published gem or something to that affect. Oops! Will look into it tonight, thanks a bunch!

In the mean time, if you'd like to keep toying with the app, you might be able to get away with this:

gem install httparty
gem install paint
gem install thor

This will make sure you've got the dependencies installed system-wide, but really, gem install lard should have covered that, so I'll figure out how to cross the T's here. Thanks again!

View on GitHub

... and I learned two things at once:

1. I should do some very basic googling before responding to issues. Even if I really just want to fix the user's issue as quick as possible (with the gem install * commands I gave him), I might be able to give a more insightful and easier solution after a quick search
2. Sure enough, I was right about how Gem dependencies don't come from Gemfile:

   Comment for #7

   

   hawkins commented on Oct 27, 2018

   I should shut my mouth more - very quick google search revealed this is a rookie gem publisher mistake! 😅 Fixed it with lard@0.0.6, so feel free to run gem uninstall lard and gem install lard to grab the latest version and its dependencies and we can pretend like this never happened. 😉

   View on GitHub

So I patched it and deployed it as 0.0.6 so Josh could try again later.

Then, more refactoring. I decided to split the CLI from the lib so that the CLI is easier to work with and other packages can leverage lard programatically in the future. Pretty straight forward stuff, just turning LardHTTP from a module to a class, maintaining some state, and leveraging that library from the CLI.

What I didn't know about this was how it could work simultaneously in production and development, though. Since this is my first foray into publishing a Ruby Gem, I've never had to link libraries from the file system, I could always just say "Hey Gem, find that thing for me" and be on my way. Not this time!

I first kept require 'lard' in the code that would be deployed to production. This works, because when someone runs gem install lard, Gem will know where lard is, so require 'lard' will resolve successfully. In development, though, lard is not installed, we just have its source files. So at the time, I would manually change this line to require '../lib/lard' (or use require_relative) and change it back to require 'lard' before committing, so production code would not be looking relatively, but development code would. Gross.

After some digging, I eventually found that leaving require 'lard' in the code is the right move, but that I can run the development binary bin/lard with a command line argument to link the lard library from the source code by using ruby bin/lard -Ilib. This looks familiar to you if you know GCC, but here's how it works:

• We run the ruby binary
• We pass it bin/lard, a ruby file from my source code that contains the CLI code (and the infamous require 'lard' call at the top of its file)
• We specify the -Ilib argument to map the lib folder (where lib/lard.rb exists) as an include folder for this runtime
• Ruby processes require 'lard'
• Gem is unable to resolve 'lard' from its normal folder, and sees that it is not relative
• Gem finds that the lib folder was linked, though, and that it contains a lard.rb file - so this require is resolved to this file!

This is great. Now, I can just run that command during development and be perfectly comfortable to not commit broken require statements. Woohoo!

Now technically day 7, even tho it's 12 AM on a Friday night/ Saturday morning, I cleaned up the lard user command to pretty-print user details finally.

At this point, the library was technically complete in everything I wanted it to do at a minimum to be a viable command line interface for Larder!

So I celebrated by going back to sleep finally and didn't get back at it until the next evening (same day, next night). I'm a little bit nocturnal on the weekends, if you haven't noticed...

This is where I really made Travis shine - by configuring it to automatically deploy new gem versions when I tag a release in git. Yaha! (I also took some time to go back and do this to Prettier Webpack Plugin, like I should have done years ago) Hey, why not, shameless plug:

var PrettierPlugin = require("prettier-webpack-plugin");

module.exports = {
  // ... config settings here ...
  plugins: [
    new PrettierPlugin()
  ],
};

Then, after some linter warnings, I finally got around to removing httparty. Whew! It felt great to have a little bit more control over what's happening, and to know I don't have to read the library author's mind to know how to use it anymore. Alright alright, sorry, let's move on.

Continuing the trend of "I'll sleep when it's done!", I kept on with Day #8 at midnight. The next issue I tackled was another interesting learning point for publishing Ruby Gems. Check it out:

How can we ensure the User-Agent gets updated with every new version? #11

todo[bot] posted on Oct 28, 2018

https://github.com/hawkins/lard/blob/295e5d9cf7b86de00f6376f9b2627bd11b44f9ac/lib/lard.rb#L100-L112

---

This issue was generated by todo based on a TODO comment in 295e5d9cf7b86de00f6376f9b2627bd11b44f9ac when #10 was merged. cc @hawkins.

View on GitHub

I wanted to add a User-Agent to all network traffic coming from Lard to indicate it was coming from Lard. Trouble is, like the whole require 'lard' headache up above, I didn't want to have to make code changes here to be in sync with the gem version itself manually. That would be silly! Computers are smarter than me, surely they can do this themselves. My first thought was a rake task that would complain if I didn't change it manually. So when I tag a release, Travis would find that rake screams about the release not updating the User-Agent, so I at least get forced to change it without publishing an accidentally-wrong User-Agent.

Okay, but that doesn't fix the root issue. I thought back to how gemspec's are actually written in Ruby and leverage the Gem library. So I thought "maybe Gem can tell me what version of 'lard' I'm using?"

Sure enough, it can, with Gem.loaded_specs['lard'].version. Bingo! But... wait. No, no dice. That only works in production, when gems are loaded from the local gem repository. Not in development, when they're linked virtually. Oof... well, okay. I thought I could add a constant to the library and pull that into lard.gemspec then, since gemspec's are just Ruby code after all. So I tried it... and after realizing I neglected to mark lib/lard.rb as requirable and wasting 0.0.7 as a test for this, I found that this version style would otherwise work!

In the process, I learned that RubyGems offers a pattern of pre-release versions, which are meant for shipping possibly-broken code like what I just wrote. Awesome! I just fixed two problems with one PR!

Finally, I decided it was time for proper tests. Now I love testing, especially when automated via Travis, but I've been struggling with how to test a program like this properly for a few reasons:

• How do we test effectively when our library relies so heavily on external network connections?
• What about our CLI, what do we test? The final binary, or the modules of it?
• How do we avoid testing the CLI kit's underlying capabilities (Thor)?

After brainstorming that at 1am (always a good idea!), I opened an issue with some thoughts and got to work:

Add tests #15

hawkins posted on Oct 28, 2018

I waited a while on this because I wasn't sure how best to test it, but here's a few ideas I'd like to see about implementing:

• test Lard lib functions but mock the http responses
• test LardCLI functions like print_*
• test bin/lard executable if I can come up with a way to put an API token on Travis.

View on GitHub

So to end out the night, I installed RSpec and wrote some basic tests to get the ball rolling.

# spec/lard_spec.rb
require 'lard'

RSpec.describe Lard, '#authorized' do
  it 'returns false to indicate user is unauthorized' do
    l = Lard.new
    expect(l.authorized).to be false
  end

  it 'returns true to indicate user has a token' do
    # Though we can't actually guarantee the token is authorized
    # without an http request
    l = Lard.new 'token'
    expect(l.authorized).to be true
  end
end

RSpec.describe Lard, '#api_url_prefix' do
  it 'returns the Larder API\'s Base URL' do
    l = Lard.new
    url = l.api_url_prefix
    expect(url).to be_an_instance_of String
    expect(url).to be_an_start_with 'https://'
  end
end

RSpec.describe Lard, '#get' do
  it 'will fail if not authorized' do
    l = Lard.new
    expect(l.authorized).to be false
    expect { l.get 'user' }.to raise_error(RuntimeError)
  end

  it 'can fetch user' do
    l = Lard.new 'token'
    expect(l.get('user')).to be_an_instance_of Hash
    expect(l.get('user')[:links]).to be_an_instance_of Integer
  end
end

# spec/spec_helper.rb
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)

RSpec.configure do |config|
  # Handle web request mocking
  config.before(:each) do
    user_body = { 'id' => 1,
                  'username' => 'username',
                  'first_name' => 'greatest',
                  'last_name' => 'ever',
                  'timezone' => 'America/New_York',
                  'avatar' => nil,
                  'date_joined' => '2016-01-21T03:03:33Z',
                  'links' => 500,
                  'is_trial' => true }
    stub_request(:get, 'https://larder.io/api/1/@me/user/')
      .with(
        headers: {
          'Accept' => '*/*',
          'Authorization' => 'Token token',
          'Host' => 'larder.io',
          'User-Agent' => ['Lard/0.0.8', 'Ruby']
        }
      )
      .to_return(status: 200, body: user_body.to_json)
  end
# ... the rest of `rspec --init` output
end

Recap: What I learned

I learned a ton in just 8 short days of writing an MVP command line interface for bookmarks management while working full time on other projects, so it's hard to sum it all up, but I've sat here for an hour now writing a silly story no one will ever read, so I might as well sum it up so I can regale my grand-children with the escapades of my youth:

• Indie devs rock! The two behind Larder, Josh Sharp and Belle Cooper made an awesome product and were super supportive through the whole process. I can't wait to make Lard as awesome as possible, even if I'm the only user, just as a tribute to Larder itself.
• Language version management is hard. My mac came with Ruby installed, but it's sort of known that you shouldn't use it - so I installed rbenv some time ago. Well, I didn't remember that I had Ruby installed on my mac when I started this, so I grabbed rvm this time. Whoops! That put me in quite a pickle, but I learned how rvm links Ruby and its dependent libraries (having a non-system dependent OpenSSL being an issue for me) and managed to get that solved.
• Gemfile != gemspec. Almost every Ruby project I've used had a Gemfile that installed everything you need. This was my first Ruby gem I wrote myself, so I had to learn how to specify runtime dependencies on the Gem itself, as well as development dependencies that can belong in the Gemfile. Still a little unclear as to how to reduce duplication between them, but that's a story for another day.
• Rubocop rules. I've always loved linters, especially auto-formatters like Prettier, but Rubocop is the best of both worlds in my opinion.
• Don't reach for the libraries until you know you need them. I generally don't, but in hobby projects like this, sometimes I reach for the big guns just to move quickly so I can have fun and not do drudge-work. This time, that gave me a big headache, so I've got more ammo to support rolling my own infrastructure on the next project. (Like we're doing over at Druid!)
• Ruby Gems are super smart. From linking development dependencies, making your gemfile just plain Ruby so you can avoid repeating yourself by importing some constants from your code to your spec, I was super pleased with working with Ruby Gems all around. Writing them and publishing them is a breeze! Especially with Travis.
• Oh, Travis CI freaking rules. I already knew this, but setting up auto deployment was the real hot stuff on it this time.
• Don't over-think your testing struggles. I could have set up testing way sooner, mocking HTTP requests is very much a solved problem when it comes to ruby testing, as my brief time with RSpec and Webmock showed last night. I wish I had set up testing sooner, but I had a lot to learn and iterate on so I put it off too long.

What's next for Lard?

Well, first of all, TESTS! I'm absolutely getting proper testing in place as soon as I sit down to code again on Lard. But then, exciting new features:

• Improving lard bookmark to make it more dynamic and flexible to how you want to describe or change your bookmarks
• Aliasing commands with Thor (so lard s means lard search, etc)
• Working with your bookmarks offline, so you can make changes and upload them to Larder servers next chance you're online. Or, work entirely offline, if that's your workflow. (But Larder rules, so Lard will always be a Larder client first and foremost)

Conclusion

Writing Lard was a blast, and I'm stoked on how it will help my workflow and how much I learned about Ruby and its ecosystem. I'm excited to see how far I can take the project, and I hope I convinced at least one reader out there somewhere to either Try out Lard and break my code or to make their own project writeup someday!

If you want to try Lard, feel free to run gem install lard and lard help to get started. Or pop over to my GitHub to check it out and say hello!

Alright, well I've spent entirely too much time in front of screens this week, so it's time I get out and enjoy the last part of this weekend. Happy weekend, everybody, and thanks for reading!