I am the cofounder and accidental CEO of honeycomb.io, where we are thinking hard about how to help you debug and understand the complex systems you have now && the even crazier systems you're going to have soon. I think a lot about observability and how to help software engineers own the code they write without losing their quality of life. Before this I was an engineering manager at Facebook, built systems at Parse and Second Life, and spent most of my time worrying about databases. I miss being on call.
For further actions, you may consider blocking this person and/or reporting abuse
Latest comments (58)
No question. Just wanted to say i loved reading your responses and you sound like a hella of a fun character :)
Note quite the same as accidentally becoming CEO, but thought you might like this article about a woman who joined a COBOL bootcamp, became a programmer, and ultimately found herself promoted out of programming.
How is kubernetes evolving? Is it becoming easier to use?
Yeeeeeeeessss... slowly and painfully. Much more slowly than I would have thought. I actually recommend most folks use a hosted solution, sigh.
What's your favorite thing to do when you're not working, that's non-tech related?
For the last two years I've been walking 5-10 miles a day. I've always hated gyms and fitness in general, so it's nice tp find something that really works for me. It's practical, I can do it when I'm roaming strange cities, I can write emails and blog posts and do twitter and read ... basically I look down at my phone, and when I look up an hour has passed and I'm there. It's amazing.
I also love, love, love fancy whisky, and bonding with a good friend while sipping late into the night. I <3 my scotch malt whisky association pass. The tasting notes read like a markov's chain. _^
Do you think technologists make better CEOs than someone coming up through say marketing or finance ?
No, not necessarily. I've more often seen them be worse, because engineers tend to underestimate how important and/or challenging business stuff can be ("you just make the best product, and that will always win, right??"). Engineers have often underinvested in developing their powers of persuasion and other social skills as well.
But being a good CEO is all about surrounding yourself with people who are better than you, and knowing when and how to interfere.
What are your biggest goals for honeycomb's next 1-2 years?
First: nail user experience. We've built a service that's so fucking powerful that a few people will crawl over rusty nails and shards of glass to use it. Now it's time to circle back and make it more broadly available.
Instead of painstakingly instrumenting everything, we are building onramps to particular communities, so you can e.g. do "npm install honeycomb" and get all the basic stuff for free. I think this will help close the gap.
Second, I really deeply want to make honeycomb a mecca for distributed systems art and visualization. We need more visual metaphors for teasing out signal from noise, and more cutting edge art drawing on recent academic advances in data vis.
People are getting hit by this freight train of infra complexity, and I don't think A.I. is gonna save us in the next decade or so. What will save your ass is good design paired with intelligently plumbing your network. Turn it into a social graph question: when you get paged, what do you want to see? Obviously you want to see whatever questions were asked by the last person who got paged about this problem.
We have to look for ways to bring everyone up to the level of your best debugger, in every subject area. We have to get better at empowering teams, not individuals. Hell, I learned Linux by reading other people's .bash_history files. I want Honeycomb to feel like it's that embedded in your team and your social circle.
Third, I want to help software engineers demystify production. I want to take the CI/CD revolution another big step forward, and make it table stakes for every engineer to explore the consequences of their code in production, every time they deploy.
I think production should not just not be on the other side of a wall, but it should feel like the "fourth trimester" -- when you ship your code to prod, it's a blind, wailing, helpless, probably broken piece of shit when it first encounters real users, real data, real services, real network. You have to nurse it into growing up.
Software needs owners, not operators. Everything I'm consumed with is just details towards that goal.
Damn, thanks for such a great response.
Do you have any articles you've written or talks where you've discussed something similar to the "trimesters" of development? I love that insight, and the analogy of needing to nurse your systems/tech to maturity.
And thank you for doing an AMA!
I have a draft. I hope to finish it this week. :)
Thanks for having me!
Always count on a startup founder to have lots to say about these sorts of things 🙂
... i can keep going ... 😇
What can an organization do to be more inclusive of their DBA’s into their devops initiatives?
Use the same tools! The edges of tool usage is what creates silos.
Also, be willing to learn things like query optimization, etc so you don't have a DBA SPOF. People seem remarkably eager to wave a hand at it as tho it were black magic. IT's not. It's easy.
Can you suggest a talk or two to check out? By you or by someone else or both.
Oooohh boy. Do you like articles? Cindy Sridharan has been on fire lately, with her pieces on the subject. Honestly I had kind of stuck my neck out there, insisting this definition makes sense, and all the monitoring boys were coming down really hard on me saying it was bullshit. Cindy researched the topic, read a bunch of the stuff on both sides, and wrote some brilliant long pieces with even more detail than me where she came to basically the same conclusions. It was super validating. I think observability is an idea whose time has finally come.
As for talks ... I'm not aware of anyone else talking about this just yet, though I hope that changes soon. You might check out my Strangeloop 2017 talk. I was super specifically pitching it to software engineers, trying to meet them where they're at. Let me know if you like it or not, please!
Thanks for the tips
Which is more important - observability of systems, or bourbon?
Trick question: it's a false dichotomy. Everybody knows ops teams run on malt whiskey.
Hey Charity! What's the hardest part about dealing with the engineering stuff and the business stuff?
I don't really deal with the engineering stuff, right now. I would only annoy people if I stuck my nose in with opinions. I don't really deal with business stuff so much either though ... hmm.
I guess I would describe my role as always plugging the weakest important part of the ship, and paying lots of attention to product and strategy.
Year one was about, "do we have a product? is it different enough?"; year two was about .. "okay, now do we have a business, and will people pay for it?" and year three is indubitably going to be about user experience. So I'm spending most of my time learning things about that.
Not sure if this answers your question or not, sorry.
What was it like being homeschooled? I'm thinking about homeschooling my future kids. Also, sometimes I wonder how I would've turned out if I was homeschooled.
Terrible. I was deeply isolated. I didn't really have any friends or learn to interact with other humans for years after leaving home.
There was also the religious fundamentalism, though.. it may be hard for me to disentangle the two.
But seriously: kids need strangers and peers and friends. Family isn't enough. Beyond that, I do think it's a crime how many hours/day kids spend sitting in classrooms reviewing stupid shit, and all having to go at the same dogged pace. It's not a good use of anyone's time. It's like a warehouse more than an institution designed to inspire love of learning.
Oh dear, sorry for all that...
100% agree. Thanks for the answer.
I studied classical piano in college too! I always have Arabesque No. 1 by Debussy stuck in my muscle memory -- first thing that comes out when I get to the bench! Do you have a go to piece?
Niiiiice. :)
Rachmaninoff has always been my guilty pleasure. I spent two years learning the Revolutionary Etude when I was ~12, and I find myself compulsively fingering those long descending sequences when stressed. It's like a nervous tic.
YES!!! Everything c minor please. That one's so much fun but my fingers definitely aren't in shape for it anymore!
it's actually really depressing to play ... what comes out of my fingers doesn't sound anything like i remember it sounding 🤔
What is an area that new software engineers often overlook when writing code?
What happens to it after you deploy it.
It's not enough to compile and pass tests and not get paged about it. You should have muscle memories for going to check up on it -- did what you expected to happen, actually happen? Did anything else change?
How'd honeycomb.io get it's name? I think it's pretty sweet.
...😉
The first name was bloodhound.sh.. we thought a detective dog would be cute, but it sounded vicious. So we shortened it to hound.sh. But then we got a legal challenge from hound-ci. So the four of us (myself, Christine, Ben, and Toshok) sat down with a lot of beers and tossed out all the ideas we could come up with one night.
The next morning we sat down with coffee and sifted through them for possible contenders. One was trufflepig.io, the other was honeycomb.io. But it was owned by Slack... :(
However, our lovely friends at Slack agreed to sell us the domain (and tossed in hny.co) for just the cost of the lawyer's fees! It was amazingly generous of them, and we are incredibly grateful.
I like honeycomb because of the associations with structured data, sweetness, hard work, and oh yes queen bees. <3
One of the early name ideas for Amazon was relentless.com. Check out what happens when you click that link 🙂
They too found it kind of vicious in the end.
OMG. TIL!
Yes! All great reasons! That makes a whole lot of sense. That's great that it worked out.
How do you price a software product/service?
Start by bleeding out your eyes and tearing out your hair for six months or so.
It's incredibly hard.
More seriously, ok, I guess I would say:
Thank you for such a thorough answer! Boy this is tough 😥.
so tough <3