DEV Community

From 15,000 database connections to under 100: DigitalOcean's tale of tech debt

Sun-Li Beatteay on January 13, 2020

A new hire recently asked me over lunch, “What does DigitalOcean’s tech debt look like?” I could not help but smile when I heard the question. Sof...
Collapse
 
franksierra profile image
Frank Sierra

I love this kind of posts, that aren't based on hypothetical scenarios but real ones 👌🏻

Collapse
 
saveshodhan profile image
Shodhan Save • Edited

Nice read!
I think we should have a series of such articles with people explaining how tech debt was brought down by organisations..
It gives us good insights on software restructuring and architecture.

Collapse
 
stephanieboldt profile image
StephanieBoldt • Edited

Thanks for writing this awesome article. These kind of articles are very important as they teaches you a very important message of never give up. Entrepreneur can get the most out of this kind of articles.
We all know that digitalOcean is an awesome hosting provider. Perhaps, among the top 3 hosting providers available in the cloud hosting industry. I have been using the host from quite a time and it is very interesting to see the host grow.

Collapse
 
john_horner1 profile image
john_horner

What language(s) are the new elements, Event Router and Harpoon built with?

Collapse
 
sunnyb profile image
Sun-Li Beatteay

Golang. The majority of our code base is written in Go and all new services use Go. However, there are still parts of the stack that use Ruby/Rails and Perl.

Collapse
 
pevers profile image
Peter Evers

Awesome article!

Collapse
 
omrisama profile image
Omri Gabay

Cool article but I hate how it reads as an advert for DO.

Collapse
 
sunnyb profile image
Sun-Li Beatteay • Edited

If you mean that this reads like an advertisement to work for DO, then I would agree with that. I think DO is a great place to work, which is I why I still work here and partially what motivated me to write this.

As detailed in the intro:

I hope that telling [this story] will be helpful for future DigitalOcean developers – or any developers who find themselves in a tricky tech-debt conundrum.

If it reads like an advertisement for DO's products and/or services, that's definitely not my intention. I don't include any Call to Action or referral link. Heck, I don't even include a link to DO's own website haha.

Granted, I do talk about one of our products. Though it's a bit tough to write about tech debt without including some of the products that were impacted by it.

In any case, I do hope you're able to get other insights from this post aside from "the author sounds like a DigitalOcean fanboy" haha.

Collapse
 
sheldonhull profile image
Sheldon

I took this as a contribution back to communities knowledge and sharing your experience. Personally I did not get any impression of selling. But it did give me an appreciation for digital ocean an engineering team's approach to start decoupling the more monolithic data tier. Really appreciate this!

Collapse
 
omrisama profile image
Omri Gabay

I was indeed able to do that. Thank you sir!

Collapse
 
zanehannanau profile image
ZaneHannanAU

It is, though.

And an article from our sponsor DigitalOcean:

  1. From 15,000 database connections to under 100: DigitalOcean's tale of tech debt: how DigitalOcean scaled infrastructure with their user base.
Collapse
 
omrisama profile image
Omri Gabay

🤷‍♂️

Collapse
 
joshuawilluhn profile image
Joshua Willuhn

Cool read, thanks for sharing!

Collapse
 
rhymes profile image
rhymes

Nice story! Thanks @sunnyb

Collapse
 
mx profile image
Maxime Moreau

Hi, thanks for sharing it's really interesting. With the startup that I was working for, we had similar issues with the need to migrate our "messaging queue" but we never did something like Harpoon :)

Collapse
 
mshirlaw profile image
Matt Shirlaw

Always interested to hear how others solved real world problems. Thanks for sharing

Collapse
 
hendry profile image
Hendry

Great post! Hope to see more architecture related content

Collapse
 
jcicharski profile image
jcicharski

Fantastic explanation, thanks!

Collapse
 
ichavezf profile image
Eduardo Chavez

Its amazing this history, legacy Code its true, exists and only good architects can make better.