Interviewing Engineers at Semaphore

A company is only as good as its people. For a company like Semaphore that consists of small and agile teams where every individual counts, finding the right engineers is crucial to success.

In this post, we want to talk about our selection process, which we have spent many years refining. If you are an interviewer at a tech company, you should find valuable insights for selecting candidates. If you're an engineer who is thinking of joining us, this post will help you prepare.

The selection process

Since Semaphore was founded, we have strived to find an interviewing method that is fair and convenient for both the company and the candidates.

In its current form, the selection process consists of five or six interviews over the course of two weeks. Semaphore is a 100% remote company, so all conversations take place over video calls.

The process can be broken down into five stages:

The first stage is selecting people that we think will fit nicely into our work culture. We look for people who are open-minded and seek mutual understanding through empathy, collaboration, and communication. Each of us is always ready to examine our convictions with humility, seeking continual improvement and growth. We believe that the ego frequently gets in the way of people finding the best solutions.

The ideal engineer for us:

• Has been working for the same company for several years.
• Has proven experience in small or startup environments.
• Is an energetic and proactive individual who takes ownership of problems.
• Has a great thirst for learning.
• Has the capacity to deliver high-quality products and services.

In order to avoid biased decisions and make a fair selection, we have the following mechanisms in place:

• Candidates are interviewed by different people from various departments within the company.
• We ask the same questions and use predefined guidelines for all candidates.
• Interviewers do not share notes until everyone has had a chance to fully form their own opinions.

Getting to know each other

Once you are selected, we'll get in touch for a quick Zoom call.

This is perhaps the most traditional part of the selection process. We'll want to get to know you as a person and as an engineer. Be prepared to talk about the projects you have been a part of, the systems you have built, and the challenges encountered along the way (and how you overcame them).

Don't be surprised if we get curious about your experiences in past roles or ask for references. This initial meeting is also an excellent chance to discuss your expectations and needs. We think of interviews as a two-way system. You’ll have a chance to ask about the company, its people, and how we work. This means that you will be able to find out if Semaphore is a good fit for you.

The live coding interview

In the first technical assignment, we'll ask you to build something for us during a one-hour live coding session. This test is about assessing your computer science fundamentals, how you code, and how you control your tools. We believe tests should reflect what any engineer at Semaphore would face on a typical day. Don’t worry, there won’t be any puzzles or brain-teasers — they only act as artificial filters.

Because we know that live coding is stressful, we'll do our best to help you be at your best:

• You can use your favorite IDE or text editor. We don't want the tools to get in the way of the test.
• Feel free to use any language and framework you like.
• No one expects you to code from memory. You can browse the Internet as much as you need.
• We will send you a mock test similar to the actual test a few days ahead of the interview, so you know what to expect.

The second round of interviews

Once you pass the live coding test, we'll schedule another call for the next round of interviews where we'll give you the task of designing or debugging a system.

Without giving away too much about the test, we can say there are no trick questions and no right answers. This test instance evaluates your technical and collaboration skills, as well as your ability to think through a problem on your feet.

After clearing the second batch of interviews, everyone involved in the selection process will get together, share notes, and reach a decision. Suitable candidates may be invited to a final call to finish off the details.

Why we test the way we test

The selection process at Semaphore wasn't always as it is now. Years ago, in addition to the interview, we gave candidates an assignment to complete over the weekend. This homework was a great way for us to gauge their technical skills.

But giving out homework meant we lost good candidates who didn’t have enough free time. Most of us already spend too much time working during the week. Family and friends are important too.

So, we decided to do only live interviews as they are less time-consuming. But this created a new challenge: how do we gauge the technical skills of a candidate from a conversation alone? First, we tried a panel-like interview where candidates had to test their mettle by facing four Semaphore engineers simultaneously.

The extended interview method didn't last very long. For starters, it was very inefficient for us. With so many people on the call, we found that the conversations went off track rather quickly. Also, it puts candidates under the wrong type of pressure.

Next, we would ask candidates to build a very specific Rails project (we were a Ruby shop back then). Live-coding struck a good balance; it was time-efficient and gave insight into the skills of the candidates and how they worked.

Unfortunately, this method caused us to miss out on sound engineers simply because they were not experts in the framework. Realizing that, we ditched the Ruby requirement, letting candidates use any language. And this is how we interview engineers today.

Framework engineers vs. software engineers

Since much of the Semaphore stack is based on Elixir and Go, requiring mastery of any of those languages would be tempting. The problem with this is that if you test for a specific language or framework, you get framework engineers. When the test is about the stack, you discriminate against great engineers simply because their experience differs from your requirements.

Yes, an engineer with mastery over a language can start working almost immediately upon joining the company. But we found out the hard way that framework engineers usually have a low ceiling. Without strong computer fundamentals, they struggle when confronted with unconventional problems that require them to think outside of the box.

We need more. We need software engineers. They know their computer science, they think in terms of algorithms instead of frameworks, they have experience with solving real-world problems with code, and have a learning attitude that helps them pick any language in a very short time.

Antipatterns in technical interviews

In interviews, companies often put engineers through unnecessary hurdles or resort to gimmicks that don't have anything to do with the job description.

It may sound obvious, but the selection process selects the engineers who will join the company, i.e. the type of candidate you get depends on how you conduct the interview. So, ask yourself: do I want a framework engineer or a software engineer? Do I look for competence or passion? Am I testing for skills that engineers require on the day-to-day?

When you interview:

• Do you make candidates solve puzzles, face brain-teasers, hacker challenges, answer fanciful or trick questions?
• Do you test for framework and tools instead of fundamentals?
• Do you reject engineers for not knowing a specific toolset?
• Do you fail candidates for not completing the task even if they did everything right?
• Do you believe people should be able to code offline?

If you answered "yes" to any of these questions, then you might have fallen into the trap of creating unrealistic tests, which have the unfortunate tendency to select engineers that are not fit for the job. Cleverness is not the same as excellence and good memory, and knowing a framework inside-out alone won’t help an engineer who is confronted with an entirely new problem. If this fully or partially describes your interview process, it may be time to re-evaluate how you interview your next engineer.

This goes for candidates as well. The interview process provides some insight into the character of the organization. If you find yourself on the receiving end of these sorts of questions and bizarre tasks, you might end up in a job you aren’t well suited to if selected. Not because of anything you did wrong (or right), rather because the interview didn’t reflect the reality of the job.

Semaphore is hiring

Semaphore has been growing faster than ever, so getting the right people has never been so critical. We are grateful to all our excellent engineers, who have helped make Semaphore into the best CI/CD platform out there.

Did you know we're hiring engineers of all levels! If our way of thinking resonates with you, consider joining us. For more details, check out our jobs page.