Scarf Sessions is a new stream where we have conversations with people shaping the landscape in open source and open source sustainability. This post will give a recap of the conversation I had with our guest Alex Biehl
Alex is a software engineer at Scarf who recently open sourced a tool to generate Haskell server stubs called Tie.
For the full interview please see the video posted on our YouTube channel.
Stream Recap
Can you talk more about what programming languages the engineering team at Scarf uses?
Alex: “We use a different mix of languages. Though there are two things that stand out for the main products, which are Haskell for the backend like data processing metrics, pipeline and all. And the other one is of course, JavaScript for the front-end, like the UI of the product.
“Also one of the defining aspects of our infrastructure is that we use something called Nix to assemble all the infrastructure bits. We use it to build out our binaries, build the containers from those binaries, help manage and distribute our artifacts for deployment.”
Can you describe the difference between Haskell and other programming languages?
Alex: “Haskell is known as a functional programming language. One of the legends of Haskell is that if it compiles, it works okay. Which is, of course, not always true, but it's a good approximation for a lot of things.
“And coming from that background our founder, Avi, decided that it would be a good foundation for a product.”
Did you work in open source before joining Scarf?
Alex: “I've always been an active open source contributor. Most of the time in the Haskell ecosystem. I’ve been contributing to a lot of Haskell infrastructure as well.
“I was a maintainer for Haddock. It's the de facto standard documentation tooling for Haskell. And I also contributed to the core build system called Cabal and also contributed a few small patches to GHC itself, as well as, all the ecosystem around it."
Christina: “I'm always interested in how people get started in open source.
"Was it something that you were just using these things, so you decided to contribute to the project?
“Or were you specifically like, “Hey, I want to do something with open source."
“How did you get started?"
Alex: “I think I'm kind of a pragmatist in that regard. I noticed something isn't working and now you have the chance to either, you know, build it yourself or piggyback off what's already there and improve it.
“And the pragmatic choices of course, to not reinvent the wheel and improve what's there and iterate on that. And that's what brought me to open source and what brings me to contributing to open source even to date.“
Where did the idea for Tie come from?
Scarf Redesign
Alex: “At scarf we are in the process of revamping our UI for the product.
And we thought it would be a good time for us to also not only revamp the user interface, but also revamp the underlying parts that serve those UIs. Right now the API is written in a very ad hoc way.
“We had a chance to revamp the API, we looked at what we had and analyzed all the shortcomings and disadvantages that we saw. And we found that defining your API in Haskell itself is prone to the usual developer habits.
“Something that developers do is build out patterns and mechanisms to type out lines of codes. They reduce duplication and they make things. They consolidate things to make it shorter, to make it more correct, to avoid repeating themselves.
“It happens that the API itself gets less and less explicit because it's hidden behind all those structures that we build in order to easily write the API itself.
“That made it very hard for us to actually find out what the API itself is. Like what is the actual route for this particular endpoint? You have to look it up and dig deep in order to find out. And another issue we found is we often ended up with something I call “scope creep”. Sometimes types that are meant for the public API contract of the API make it into the domain and business logic.
“Which means that in the long term you are coupling your API contract, which you want to support for your customers, to the internal logic of your product. And every time you want to make a seemingly internal change to your logic you might accidentally change the public contract as well."
Developing Tie
“This brought me to two realizations. First, we want an explicit human readable API contract, which we can share with our customers.
Second, I wanted to settle it on co-generation. That means I don't want to type out these very boring interface types or contract types myself. I really want a tool to have it generated for me so that I never ever accidentally make a change in the Haskell code that would break the contract. With those goals in mind the most common choice today for these kinds of things is Open API. Open API denotes domain specific language to model web APIs.
“Where Tie comes into play is that it understands Open API. And Tie takes the Open API format and generates Haskell code from it. It does this in such a way that we only have to implement the actual list for the specific routes and don't have to deal with all those serialization logics and peculiarities.“
What did you find hard or unexpected about open sourcing Tie?
Alex: “One of the things that I fell really short on at first was writing good documentation.
I hadn't even created a good example in the repository when I first open sourced the project. It didn’t have a “How to use” section although we were already using it ourselves in production.
“It was born as a weekend project after two or three days of work and we were putting it into production already after that weekend. So I quickly had other support issues I had to work on before I had time for documentation.
Christina: “This is why bigger teams have dedicated people for documentation, I totally understand. And to Alex's credit he very quickly messaged me because I have been an open source community manager in the past and he asked me how do I make a good readme?
“I think that's a really good point though. Often maintainers or someone who really wants to open source a tool they were using and it was helpful for them don't have time for everything else, like the documentation.
Do you have any advice for anyone putting anything out?
Alex: “Don’t let the perfect be the enemy of good."
"You should add documentation and all the parts that got lost on the initial phase of the project. But you know, what's more important I think is, getting initial feedback on the project itself.”
What’s in the future for Tie?
Alex: “Tie is not finished. It doesn't support all of Open API’s setup. The standard is vast so this is the call for contribution right here.
“Help us improve it, file issues. We can definitely help and guide any changes and improvements.”
What else are you working on?
Alex: My pet peeve is proper co-generation for contracts for external systems, for example databases. People like to use ORMs and generators.”
Christina: “Can you explain what ORMs are?”
Alex: “Object relational mappings method. Like an active record in Ruby, Django, etc. And I want a tool that takes your migration folder and generates for you the types very similar to what Tie does. I want the tool to generate types that reflect the records, the structure of the tables in the database then also allows you to basically ingest SQL.
And generate small records around that SQL which are typed. So that you can use your SQL to write theories, to define tables, to alter tables and so on, but still have the good old type system to help so that you don't mix up your types”
For more information on Tie please visit our repository and consider contributing to the project to help improve it.
Are you helping to shape the landscape in open source? Interested in joining us on a Scarf Sessions Stream? Email Christina at christina@scarf.sh.
Are you helping to shape the landscape in open source? Interested in joining us on a Scarf Sessions Stream? Email Christina at christina@scarf.sh.
To learn more about Scarf:
Website
To follow Alex:
Twitter
GitHub
To follow Christina:
Twitter
Top comments (0)