So if tomorrow you were to start up a project, full stack work, how would you ensure it's scalable so that it can handle a mass influx of users the next day (extreme example much)?
Hi Aizaz, it's a really good question but without a very simple answer. Scaling doesn't mean the same thing for everyone, for every project or for every company. Scaling is also dependent on time and money, time to market and developer productivity so there's no silver bullet for it.
I'm going to use a recent tweet to help me here:
Sam Phippen
@samphippen
“Does it scale?” Is a boring question. “Does it scale enough for our users right now in a way that lets us continue to work on what’s most important” is a better one.
18:43 PM - 10 Feb 2019
115424
Sam is saying a thing that I've learned through the years. Scaling is not an abstract concept, meaning that it can't be detached from the problem at hand (as there is no single way to achieve "scale"). An extreme version of his example could be: "I'm choosing technologies A, B and C because I've heard they scale, but I have no expertise in them, so I spend all the time working against them, neglecting the product and possibly missing the time to market window". Maybe in this example the person could have used X, had had a slower product, made some money out of it and bought some server breathing room in the meantime :)
So, what does scale mean? Technically it means the ability of a system to handle a growing amount of work. But how that is going to be achieved wildly varies, depending on the problem at hand. There are general techniques: to scale algorithms, to scale IO bound operations, to scale CPU bound operations, to scale databases, to scale networking, to scale application servers and so on :D
Another big factor to keep in mind is that scaling and measuring usually go in pair. You have a baseline, you figure out roughly how many user your app can handle with the current architecture and then you start from there.
It also depends on the traffic patterns and which type of app you are creating. If you have to setup a government website for a new initiative that benefits 10% of your population and you know that the opening day millions and millions of users are going to register... well, you need to be prepared for that. How? Measuring, load testing, simulating real traffic, deploying all the tricks in the books and so on.
What does it look like on the front-end vs back-end?
What do you mean here? The frontend runs on the user's computer so there's not much to scale (optimize yes, but scaling I'm not sure). On the backend we go back to the infinite combinations of scaling possibilities :D It might mean tuning the DB, it might mean putting as much content as possible on a CDN, it might mean having a cluster of cache servers, it might mean upgrading a dependency that leaks memory and much more.
When I think of scale, I'm thinking DevOps and ensuring your servers can handle 'scale' and setting up distributed databases, using caching (Redis).
I think "devops scaling" as a part of the whole scaling landscape. You might not even have servers in your care and run on a serverless platform. Distributed databases come with considerations about complexity, data integrity, consistency in reading and so on. So does caching, the running joke is that cache invalidation is hard.
So if tomorrow you were to start up a project, full stack work, how would you ensure it's scalable so that it can handle a mass influx of users the next day (extreme example much)?
It's not a perfect science but it's not a guessing game, you can't build up a project knowing it's going to be hammered the day it's public without having prepared :D How do you get there? By building the MVP, measuring it, testing it thoroughly with different traffic patterns, overloading it, understanding what's the cost of acquisition of a new user, understanding what happens if you go from a 1000 to 10 thousand to 300 thousand users.
A few general tips:
test (the code, the system and so on), bugs can be a limiting factor in scaling
memory leaks can be a big issue, but they might not be for a while (going back to the concept highlighted by the tweet: if you leak 10 MB of memory per day and you can cope with restarting the servers once in a while, the money you save can be put to buy more RAM while you focus on a key feature, the debug can wait)
cache expensive computations if you can, cache HTML as well if you can
do as much as possible out of band (the user shouldn't be stuck for seconds waiting for an operation to finish, which in turns means that your system will crawl if all users are doing the same operation at roughly the same moment)
memory is faster than disk (so if you can put stuff in memory instead of putting it on the disk, it's better)
use load balancers (but most PaaS already have them)
start with a queue to distribute the work load, simple architectures in the beginning are better than complex ones
use HTTP in all of its capacity (conditional caching, proper codes, and so on)
prepare for the worst (what happens if something goes down?)
understand the cost of adding one more instance: every new server brings with it new connections to your data storage for example
You'll notice I didn't talk about specific technologies or languages or frameworks because I think they ultimatey don't matter that much (unless you have specific requirements that can be fullfilled by this or that tool).
Also, remember to build your product in the meantime, delegate as much as you can to the right tools. Don't reinvent the wheel until you know you need to.
It's pronounced Diane. I do data architecture, operations, and backend development. In my spare time I maintain Massive.js, a data mapper for Node.js and PostgreSQL.
Hi Aizaz, it's a really good question but without a very simple answer. Scaling doesn't mean the same thing for everyone, for every project or for every company. Scaling is also dependent on time and money, time to market and developer productivity so there's no silver bullet for it.
I'm going to use a recent tweet to help me here:
Sam is saying a thing that I've learned through the years. Scaling is not an abstract concept, meaning that it can't be detached from the problem at hand (as there is no single way to achieve "scale"). An extreme version of his example could be: "I'm choosing technologies A, B and C because I've heard they scale, but I have no expertise in them, so I spend all the time working against them, neglecting the product and possibly missing the time to market window". Maybe in this example the person could have used X, had had a slower product, made some money out of it and bought some server breathing room in the meantime :)
So, what does scale mean? Technically it means the ability of a system to handle a growing amount of work. But how that is going to be achieved wildly varies, depending on the problem at hand. There are general techniques: to scale algorithms, to scale IO bound operations, to scale CPU bound operations, to scale databases, to scale networking, to scale application servers and so on :D
Another big factor to keep in mind is that scaling and measuring usually go in pair. You have a baseline, you figure out roughly how many user your app can handle with the current architecture and then you start from there.
It also depends on the traffic patterns and which type of app you are creating. If you have to setup a government website for a new initiative that benefits 10% of your population and you know that the opening day millions and millions of users are going to register... well, you need to be prepared for that. How? Measuring, load testing, simulating real traffic, deploying all the tricks in the books and so on.
What do you mean here? The frontend runs on the user's computer so there's not much to scale (optimize yes, but scaling I'm not sure). On the backend we go back to the infinite combinations of scaling possibilities :D It might mean tuning the DB, it might mean putting as much content as possible on a CDN, it might mean having a cluster of cache servers, it might mean upgrading a dependency that leaks memory and much more.
I think "devops scaling" as a part of the whole scaling landscape. You might not even have servers in your care and run on a serverless platform. Distributed databases come with considerations about complexity, data integrity, consistency in reading and so on. So does caching, the running joke is that cache invalidation is hard.
It's not a perfect science but it's not a guessing game, you can't build up a project knowing it's going to be hammered the day it's public without having prepared :D How do you get there? By building the MVP, measuring it, testing it thoroughly with different traffic patterns, overloading it, understanding what's the cost of acquisition of a new user, understanding what happens if you go from a 1000 to 10 thousand to 300 thousand users.
A few general tips:
You'll notice I didn't talk about specific technologies or languages or frameworks because I think they ultimatey don't matter that much (unless you have specific requirements that can be fullfilled by this or that tool).
Also, remember to build your product in the meantime, delegate as much as you can to the right tools. Don't reinvent the wheel until you know you need to.
So much of building high-capacity and high-workload systems is about the code you don't write ;)