On-prem vs cloud: Why you have more options than you think

The debate between on-premise vs cloud is nothing new. Every company has different needs, and developers have spent endless cycles debating the pros and cons of on-prem and cloud software to solve for their particular needs.Rather than try to end the debate, we wanted to provide a deeper context and frameworks for evaluating your options. That includes looking into the rise of self-hosted software, which has the potential to, if not end the debate, at least push it in a new direction.

On-prem vs. cloud: The debate in a nutshell

A couple of decades ago, the debate between on-prem and cloud didn’t exist—simply because cloud computing wasn’t a thing yet. As cloud computing emerged and gained popularity in the mid to late 2000s, so did the debate about moving workloads from on-prem servers to cloud servers.

Equipment ownership

The primary difference between on-prem infrastructure and cloud infrastructure is that you own on-prem while you rent the cloud.On-prem infrastructure exists—you guessed it—on-premises in server rooms that a company builds and maintains. Cloud computing allows companies to host workloads on third-party servers. These servers aren’t really “in the cloud”—the biggest AWS data center in America, for instance, is in Virginia.

Essentially, when working with the cloud, you pay for infrastructure as though it were a bundle of different services—storage and compute among them. One of the advantages of this model is that you can pay for additional services (AWS offers a whole bunch, such as analytics, containers, databases, and more. The public cloud is always at the ready, waiting for you to pay up and get more.

Maintenance

The second major difference between on-prem and cloud is maintenance—particularly, the costs of that maintenance. On-prem infrastructure, by definition, is local and physical, meaning you need IT staff to buy, rack, and connect the servers that you need. And that’s just the setup. Then, you need them to manage the servers, which includes the management of hardware, software, security, and performance. If you want to expand, that’s another order and another busy IT person.

On-prem software requires licensing and installation, and when that software needs updating (which it always does), then you again need IT to download and deploy updates. According to ZDnet, it costs $731.94 per machine per year just to power a server. That base price plus all the setup and maintenance costs can add up––and then you need to cool it all, which, according to DataSpan, will be almost half your investment.

Cloud infrastructure outsources all that setup and maintenance to a data center that handles setup and maintenance (and they can do so at scale if you pay up). By running workloads in the public cloud, companies can pay for storage and compute on an as-needed basis, meaning they can scale up or down depending on usage and company growth.

With public cloud providers offering servers all over the globe, that scale also means global coverage at the click of a button. Plus, due to that global coverage, if a data center in one region goes down, a data center in another region can pick up the slack.

And whereas on-prem infrastructure requires ordering, racking, and installing a server, public cloud servers provide near-instant provisioning, meaning scale-up happens about as fast as you can request it (i.e., pay for it). To sweeten the deal, public cloud providers also offer an array of cloud services that you either can’t get on-prem or would have difficulties getting on-prem, such as machine learning or quantum computing.

Control

Control is the most complex, gray-zone difference between on-prem and cloud. Theoretically, since you (the company) own and operate your on-prem servers, you have more control than you would if you handed over your servers to a third party.
The costs of cloud computing make that clear. Entire businesses, such as The Duckbill Group (led by the inimitable, sarcastic Corey Quinn), have sprung up to help businesses control cloud spend.

AWS is notorious for runaway cloud spend. A recurring controversy, for instance, involves students spinning up AWS instances and unknowingly accumulating a bill worth thousands of dollars.

Hey friends,

I have a panicked student in my DMs who’s accidentally racked up an $8k AWS bill.

My suggestion of “talk to Support” is no good—apparently they won’t issue a billing adjustment. Anybody got better ideas, or know someone at AWS who can help them out?

RT for reach?

— Alex Chan (@alexwlchan) May 30, 2021

Cloud costs are just the beginning. Every other part of your cloud infrastructure falls prey to the same pattern—a pattern that’s in part inevitable due to the infrastructure not being yours.

Ultimately, AWS (or Google Cloud or Microsoft Azure) can do whatever it wants with its servers and services, and you’re beholden to their whims. That lack of control makes many businesses uncomfortable.

For some businesses, not owning their own infrastructure isn’t just uncomfortable—it’s illegal. Organizations that handle highly sensitive data (like medical or financial transactions) may be required by regulations such as HIPAA and FERPA to keep their business off the public cloud.

Startups and enterprises face different debates

Startups and enterprises are coming into the on-prem vs. cloud debate with different priorities.

Startups are starting from scratch and must consider agility, speed, and upfront cost; enterprises are starting with technical debt and must consider prior infrastructure investments. You can’t understand the debate, nor your place in it, until you understand the business context.

The startup perspective: Start small and scale as necessary

Before public clouds (and AWS especially), startups had to raise entire funding rounds just to afford the server infrastructure that could support a minimum viable product (MVP). But the rise of the public cloud meant that even the scrappiest startup could test out a half-baked idea without too much expenditure.

With public clouds, startups can start small and scale their cloud expenditure as they grow, meaning they only pay for what they use. This cost model makes the public cloud appealing to startups. As startups grow, however, they may experience vendor lock-in. If you started on AWS, it’s always going to be easier, logistically, to stay on AWS.

Even if the costs rise—even if the costs rise a lot—you’d still have to compare those costs to the costs of switching, which includes any potential downtime as well as the opportunity costs of reallocating engineers to migration.

Counter-example: The startup perspective described above isn’t universal. Dropbox, for instance, a 14-year-old company, eventually moved off AWS and built its own data centers. Dropbox ended up saving almost $75 million over two years.

The enterprise perspective: “If it ain’t broke, don’t fix it”

Enterprises (large companies that may have been around since before the Internet, much less cloud computing) invested in on-prem infrastructure before the public cloud was an option. As such, by necessity, most enterprises will have some measure of on-prem infrastructure to deal with.

Migration to the public cloud is notoriously costly and difficult. The vendor lock-in that startups experience, mentioned above, is worse for enterprises attached to on-prem infrastructure they can’t easily replace. And, of course, this on-prem infrastructure isn’t sitting there. It’s operating and maintaining essential workloads. Replacing it all risks downtime and necessitates time.

According to Cloud Security Alliance research, 90% of CIOs have gone through failed or disrupted migration projects. Plus, only 25% met their deadlines for migrations. To complicate things further, a Fortinet study found that 74% of companies have actually moved applications back to on-prem from the cloud.

Counter-example: Despite the reasons above, many enterprises are still shifting to the public cloud. HSBC, for instance, a bank founded in 1865 that employs over 200,000 people, migrated to AWS.

CAPEX vs. OPEX

So far, we’ve covered the debate in primarily technological terms, but there’s also a financial lens through which to view these arguments.

You can slice through the startup vs. enterprise framing and translate the debate into purely financial terms: capital expenditures (CAPEX) vs. operational expenditures (OPEX).

On-prem requires an up-front investment in server hardware (a CAPEX cost), whereas the public cloud scales up and down as you use it (an OPEX cost). Depending on your funding and how you want to account for infrastructure costs, either on-prem or cloud might be more appealing.

The rise of self-hosting (and an end to the debate?)

For years, the on-prem vs. public cloud debate has assumed that the two approaches are contradictory and opposite. In recent years, however, a new infrastructure option has emerged: the private cloud. With the private cloud, you may be able to end the debate—and have your cake and eat it too.

What is self-hosted software?

Self-hosted software is software that you run on your own cloud infrastructure. In the case of a company that purchased a new email tool, you’d run the email software in your private cloud instead of using the SaaS version that is hosted on the public cloud.

In a sense, you can get the best of both worlds. With a private cloud, you get the scalability of the public cloud but the control of on-prem. That means you can scale like a startup but remain in control like an enterprise.

The multi-cloud future

The great on-prem vs cloud debate may end in a whimper instead of a bang. Nowadays, many companies use both private and public cloud infrastructure (a setup referred to as “multi-cloud”).

Flexera research found, for instance, that 92% of enterprises and 61% of SMBs (which includes startups) have a multi-cloud strategy.

The future might not be any single option but an amalgamation of many options that evolve as companies grow and their needs change. Or once the next AWS-type revolution comes and upends everything again.

Companies are increasingly offering self-hosted options. We here at Retool, for instance, now offer the ability to self-host a Retool instance. The Retool self-hosted option is especially helpful if you have a lot of data that you can only access via a private network, a local network, or a virtual private network (VPN). Plus, you can use Docker, Kubernetes, Heroku, or Render for on-prem deployment.

Re-evaluate your infrastructure needs from first-principles

The cloud, especially for startups, has become something resembling a common-sense choice.

Of course you deploy on AWS, the assumption goes, what else would you do? Though the revolution AWS started can’t be understated, you also shouldn’t overstate its permanence. The changes the public cloud created are massive, and from its inception to the foreseeable future, deploying software in the public cloud (SaaS) will be a viable strategy for many companies.

For an increasing number of companies, however, the private cloud is becoming an option worth thinking about. Much of technology, as in business and society, is cyclical—don’t assume the death of on-prem infrastructure until you evaluate your options from a first-principles perspective.

The private cloud, especially, points toward the resurgence of on-prem. The future might be more local than you think.