DEV Community

Cover image for Cloud Computing for Data and Software Engineers
Michellebuchiokonicha
Michellebuchiokonicha

Posted on • Edited on

Cloud Computing for Data and Software Engineers

Cloud computing has become a major buzzword in the last decade. The application of technology is enabled and made possible by cloud computing.
A simple instance is the alarm on your phone. This alarm, after being created was made available to users through cloud computing. The applications we use daily and the recently hyped innovations of ChatGPT and Gemini.AI are made available and accessible to users because of cloud computing.
It enables apps to synchronize data and manage services.

Commuting to work with maps relies on cloud computing. Your Microsoft 365 and Google Workspace are made possible because of cloud computing.

Here, we will understand cloud computing and how it helps distribute data and processes across multiple machines.
Here, we will describe the concept of cloud computing, explain the cloud delivery and service models, describe cloud computing for your purpose, and more.

We will also understand the fundamentals of cloud computing, cloud service models, infrastructure as a service, serverless computing, risks and benefits of cloud computing.

Cloud Delivery Models

Cloud Delivery models are also called cloud sourcing models and there are varying computing models.

Local: Traditionally, we would have our applications on a local machine like your laptop, containing all the data and software necessary for a particular task on that same machine with or without an internet connection.
The advantage is that the solution is cheap, fast, and readily available. And when there is no internet connection, it is pretty safe from outside attackers. The downside is that it is not easy to work collaboratively and the resources of a local machine might not be sufficient for extensive datasets. This is why many organizations store their data and install software not on local machines but on servers to which organizations can connect.

On-premise: On-premise means an organization owns hardware like servers located in the organization's rooms or geographically close by to the organization. However, if the organization decides to move to the cloud, the organization will no longer own the hardware but render it from a cloud service provider. These servers were physically located within the rooms owned by the organization hence the name, on-premise or on-prem computing. The advantages are that the organizations have complete control of the software and infrastructure. On the downside, this setup has limited scalability and high investment costs. All resources must be purchased by the organization and installed on the servers entering the cloud.

Private Cloud: This means infrastructure is reserved for a particular organization and we can connect to the service via a network as opposed to the public which requires the internet.
We can also use virtual private networks to establish a connection.

This is dependent on whether the hardware is reserved for organizations in the data centers run by the cloud service provider reserved for the organization only.

Public Cloud: In contrast to private cloud, if the organization decides to use the public cloud, the organization would share the resources with other cloud users. Another difference is that the public uses the internet while the private cloud uses a non-public network.

Hybrid cloud: This model uses a mixture of on-premise and cloud services/machines. This way, we can choose which data should be stored, processed, and analyzed in the cloud and what should remain in the organization's machines.

Multi-Cloud: This means that the organization uses cloud services provided by other cloud providers.
Using different services offered by more than one cloud provider. Eg, we can use storage services provided by both providers A and B but only processing services by cloud A and analytical services by provider B. This way we might avoid vendor-locking and select from a wide range of services the ones that are useful for our particular use case. One drawback is that these setups are more complex to manage.

Poly-Cloud: This means that we use one type of service by one cloud provider only eg, we would use a database by provider A and analytical services by provider B. Here, we use only the services by one cloud provider for one specific domain of the project. Eg, we would store all the data in Cloud A, process the data in Cloud B, and analyze the data in Cloud C.
Cloud-native is not a delivery model but implies that an application is developed with the cloud in mind from the very beginning. Eg, it is designed to be easily scalable, reliable, and available by deploying it to a cloud environment eg as microservices and containers, or as a serverless function.

Fundamental Roles in Cloud Computing

Cloud Solution Architect: This is also called cloud enterprise architect is responsible for designing a high-level concept aligned with a business strategy. Coming up with a cloud architecture to scale the organization. It deals with business understanding and technologies.

Cloud Architect: This is similar to the cloud solution Architect but is more focused on the implementation unlike the Cloud solution which is focused on the high-level concept of the architecture, the cloud architect is more focused on the hands-on implementation of that concept into concrete services that work together. They are specialized in the offerings of one or a few cloud providers.

Cloud engineers: they are the administrators and operators of the cloud systems. They maintain the cloud system and its services. They manage servers and clusters. They are not only responsible for the administration of the entire system but also individual services. This includes access control. They understand operating systems and server maintenance.

Cloud developers: They are programmers using cloud services in the applications they develop. This includes application developers because they do not come up with new cloud services but use the provided services and configure them in their applications. They use and orchestrate cloud services in their applications. They need knowledge of how the cloud systems are set up. However, their core is programming, API setup, and networking.

Cloud consultant: this is similar to the cloud solution architect. They offer advice, and guidance and provide an objective overview of cloud providers. They understand the organization's business and cloud needs and come up with solutions and architecture. They are more focused on added value for business processes.
Sometimes, these roles overlap and they can function to aid the other.

Cloud Service Models

Cloud service models are different from cloud delivery models.
This tells how responsibilities are shared between the cloud provider and us as the users/developers.

On-prem: We have complete control of the system but it also brings in a lot of responsibility of taking care of each component.

Bare-metal-as-a-service(BAAS): we might choose to not own physical hardware but get it from a provider responsible for maintenance. But why not give the provider responsibility for hardware virtualization? This is the next one called IAAS.

Infrastructure-as-a-service(IAAS): the cloud provider is responsible for the physical hardware and virtualization while users are responsible for everything that runs on top of that virtualized hardware. This means flexibility but also maintenance responsibility. Eg for the application code that we develop for these virtualized machines. example AWS EC2, Azure virtual machines, and GCP compute engines.

Container-as-a-service(CAAS): when cloud providers offer services to run containers and are responsible for maintaining all underlined components, it is called CAAS. eg Azure container instance, Azure Kubernetes serves, AWS elastic container service, AWS elastic Kubernetes service, Google Cloud run, and Google kubernetes engine.

Platform as a service (PAAS): Here, in addition to hardware and virtualization, the cloud provider is also responsible for the operating system and runtime executing the code we develop. Here the provider is responsible for the maintenance responsibility and the runtime to the provider. Examples are storage services like AWS S3, Azure block storage, and GCP block storage. Managed database storage like AWS dynamoDB and Azure Cosmos DB and IOT services like AWS IOT core, Azure IOT hub, and GCP cloud IOT core. As the name suggests, PAAS is all about providing a platform.

Serverless also called Function and a service (FAAS): It is similar to the platform as a service, however, it also implies that we are not busy maintaining and configuring any hardware but focus entirely on code development. We simply push our code to a serverless service to be executed in the cloud and we do not think about the underlined infrastructure at all.
Here, it is not so much about the platform being provided but the framework on which we can execute our application code and not on setting up and maintaining the underlined component.
Examples are AWS Lambda, Azure functions, and GCP Cloud functions.

Software as a service(SAAS): In addition to the hardware virtualization, operating system, and runtime, the provider is also responsible for the application code in this service model. An example is Office 365 or any other similar cloud-based software. All we can change in this service are the application settings but not its code. Here we use functional software and can not change anything in it except the application settings.

Risk and Advantages of Cloud Computing

Advantages

Cost reduction: It is cheaper than local or on-premise solutions. The cost of purchasing hardware and hiring specialized staff better scale to the many cloud customers than having all this cost for one organization only. Also, many services are offered as pay-as-you-go billing models. This means that we can quickly adapt the cost of our system to the load it experiences.

Also, we can start with cheap hardware and quickly scale to bigger or more machines in the cloud. We can also save costs by running processes with lower priority when machines in data centers are idol because other customers do not use them at that time.

Scalability: Because cloud providers dispose of many machines, we can scale instantaneously and unlimitedly on the cloud by adding more or bigger machines to a running cluster.

Flexibility: As there are no upfront costs, we can try out new services and change components of our system easily.

Easy access: You only need a few clicks and buttons to spin up instances of the available technology in the cloud.

Reliability: Being distributed systems with high data replication and parallelization makes cloud computing highly reliable. Also, all cloud providers clearly define service level agreements and we are reimbursed if these agreements are not fulfilled.

Physical safety: it is an advantage as there is much stricter surveillance of physical access to the data centers than to the servers of a single organization.
Certificates: All cloud providers hold standardized certificates supporting us in evaluating the applied data security and protection standards. This also helps us choose appropriate providers for given use cases.

Risk

Expenses: there can be additional expenses eg when resources are not adapted to need or we over-provision our services.
Redundancy in the system ensuring its reliability means additional network transfer. When not set up correctly, this can lead to unnecessary expenses. The good thing is that all cloud providers offer monitoring tools and automated alerts to mitigate these risks.

Security: When not set up correctly, it can be insecure. Of course, there can be more points of failure in a complex cloud system but depending on the service model, we might also give responsibilities to specialized units of the cloud provider with more workforce and higher security budget than single organizations.

Share responsibility: although this is usually considered a good thing, it can pose a risk. Independent of the cloud service model we choose, we will always be responsible for the stored data, the endpoint to our system, and the access management.
Also, legally, we always need clarity on how things are done by the cloud provider when handling personally identifiable information.

Certifications and transparency: Certifications are usually a good thing but the number of different certificates sometimes makes it difficult to understand what they mean and if they are worth something and applicable to a particular use case.

Data Sovereignty: This is important when handling personal data in the cloud. here, we must keep control of the data all the time. Eg, when the data is about a certain group of people, we must ensure that it never leaves the group of people.

Note: This is a 4-fold series on cloud computing, virtualization, containerization and data processing.
Watch out for the remaining 3 articles.

Note: This is the first of a 4-fold series.
The second can be found here.

https://dev.to/michellebuchiokonicha/virtualization-containerization-with-docker-storage-and-network-services-2bjf/edit
it focuses on docker, containerization, virtualization, storage technologies and network services.

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

Top comments (0)