Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Cloud SQL, Spanner and Datastore, Study notes of Operating Systems

An overview of various services offered by Google Cloud Platform (GCP). It covers services like Cloud SQL, Spanner, Datastore, Containers, Kubernetes, Google Compute Engine, Google App Engine, API Management Tools, and Development, Deployment, and Monitoring. It explains the benefits of using Cloud SQL and how it can protect relational databases. It also discusses the drawbacks of Google Compute Engine and Google App Engine and how Containers can be used to overcome them. The document also covers the App Engine Standard and Flexible environments, API Management Tools like Cloud Endpoints and Apigee Edge, and Cloud Functions for event-driven actions. It provides a brief on Development, Deployment, and Monitoring phases in GCP.

Typology: Study notes

2017/2018

Available from 01/15/2022

vyom-bhardwaj
vyom-bhardwaj 🇮🇳

3 documents

1 / 7

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Cloud SQL, Spanner and Datastore
More About Cloud SQL
The following example will give you an idea on the benefits of using Cloud SQL.
Consider you have two accounts in a bank, and you want to transfer Rs.10,000 from
one account to the other.
Due to a glitch, the amount gets deducted from one account but is not credited to the
other. Relational databases require a lot of wo rk to set up, maintain, manage, and
administer, but still if you want to protect your relational databases like in the above
case, then Cloud SQL is your choice.
Cloud SQL offers MySQL and PostgreSQL databases as a managed
service, that are capable of handling terabytes of storage.
It also provides several replica services like Read, failover and external.
Cloud SQL instances include firewalls, customer data encrypted on
Google's internal networks, and schedule backups, and it can also be
accessed by other GCP services.
It can be scaled vertically by changing the machine type and horizontally
through Read replicas.
If you need horizontal scalability more than what is provided by Cloud SQL,
then you can go for Cloud Spanner.
Containers, Kubernetes and Kubernetes
Engine
Google Compute Engine and Google App Engine - Drawb ac ks
In Google Compute Engine (GCE), you are allowed to share resources by
virtualizing the hardware using VMs. The developer can deploy the OS,
access the hardware, and build their applications in a self-
contained environment along with access to Networking Interfaces, File
Systems, RAM, and so on.
When the demand for your application increases, then you need to copy the
entire VM and boot the guest OS for each instance of the application which
can be slow and expensive.
By using the Google App Engine (GAE), you will gain access to the
programming services. The only job that you need to do is to write the code,
and the self-contained workloads will use these services and include any
dependent libraries.
As the demand for your application increases, the platform scales your
applications seamlessly and independently, and you need to give up the
underlying server architecture control.
pf3
pf4
pf5

Partial preview of the text

Download Cloud SQL, Spanner and Datastore and more Study notes Operating Systems in PDF only on Docsity!

Cloud SQL, Spanner and Datastore

More About Cloud SQL

The following example will give you an idea on the benefits of using Cloud SQL. Consider you have two accounts in a bank, and you want to transfer Rs.10,000 from one account to the other. Due to a glitch, the amount gets deducted from one account but is not credited to the other. Relational databases require a lot of wo rk to set up, maintain, manage, and administer, but still if you want to protect your relational databases like in the above case, then Cloud SQL is your choice.  Cloud SQL offers MySQL and PostgreSQL databases as a managed service, that are capable of handling terabytes of storage.  It also provides several replica services like Read, failover and external.  Cloud SQL instances include firewalls, customer data encrypted on Google's internal networks, and schedule backups, and it can also be accessed by other GCP services.  It can be scaled vertically by changing the machine type and horizontally through Read replicas.  If you need horizontal scalability more than what is provided by Cloud SQL, then you can go for Cloud Spanner.

Containers, Kubernetes and Kubernetes

Engine

Google Compute Engine and Google App Engine - Drawbacks

 In Google Compute Engine (GCE), you are allowed to share resources by virtualizing the hardware using VMs. The developer can deploy the OS, access the hardware, and build their applications in a self- contained environment along with access to Networking Interfaces, File Systems, RAM, and so on.  When the demand for your application increases, then you need to copy the entire VM and boot the guest OS for each instance of the application which can be slow and expensive.  By using the Google App Engine (GAE), you will gain access to the programming services. The only job that you need to do is to write the code, and the self-contained workloads will use these services and include any dependent libraries.  As the demand for your application increases, the platform scales your applications seamlessly and independently, and you need to give up the underlying server architecture control.

Containers

 Containers are preferred due to the drawbacks in the Compute Engine and App Engine.  Containers are responsible for the independent scalability of workloads. They also act as an abstraction layer between the OS and Hardware.  All you need when you get a host with an OS is it should support Containers and Container Runtime.  Your code will be portable and you can treat the OS and hardware as Black Box. Using this functionality, you can move between stages like Development, Staging, and Production from on-premises to the cloud.  For example, if you want to scale a web server a hundred times, then you can do it in seconds on a single host depending on the size of the workload.

Kubernetes

 If you want to build your application using containers acting like microservices, which are connected through network connections, then you can make them modular, scale independently, and easily deployable across a group of hosts.  Containers can be scaled up or down, started, or stopped by the hosts on- demand as your application changes or else, the host fails.  The above process can be done using a tool called Kubernetes.  The function of Kubernetes is to orchestrate several containers on hosts, scale them as microservices, and perform rollouts, rollbacks.  An open-source tool called Docker, helps you to define a format for bundling your application with machine-specific settings and dependencies into a container.  GCP also has a separate tool called Google Container Builder, a managed service that is used for building containers.

Playing with Docker

 Consider a simple Python app whose output is Hello World. When you hit its endpoint, it will give you the version.  Here, the question is, how will you get the app into Kubernetes.  The versions of Python and Flask are managed through Requirements.txt file along with the other dependencies.  You can use Docker Build command to build the container and store it on the local system as a runnable image.  Docker run command is used to run the image.  This image can also be saved and uploaded to Container Registry Service (a place to manage all docker images), from where it can be downloaded.

 For example, there are 3 replicas of an Nginx pod, and you want to update it to 5 replicas. Then, you can edit the deployment configuration file from 3 to 5.  Run kubectl apply command to update the changes to the configuration file and run kubectl get replica sets command to see the replicas and their updated states.

App Engine Standard and Flexible

Introduction

By now you are familiar with two crucial GCP products, Compute Engine and Kubernetes Engine.  One common feature in these two products is you will choose the infrastructure in which your application will run, that is, Virtual Machines for Compute Engine and Containers for Kubernetes Engine.  If you want to focus on the application code, then your choice should be App Engine.  App Engine comes under Platform as a Service (Paas). App Engine manages both Hardware and Network Infrastructure.  App Engine has many built-in services like No SQL databases, in-memory caching, load balancing, health checks, logging, and authenticating users, which can be used by your application.  App Engine will automatically scale applications like web applications and mobile backend.

App Engine Standard

App Engine offers two main environments namely App Engine Standard and App Engine Flexible.  App Engine Standard environment offers a more straightforward deployment than App Engine Flexible.  It offers free usage quota for certain services, which means low utilization applications will be able to run at a small charge.  Google App Engine Software Development Kits (SDK) are available in several languages. You can test your application using these kits before deploying your application in the real App Engine Service.  Google provides runtime for App Engine Standard Environment in Java, Python, Go, Ruby, Node.js and PHP.

App Engine Standard

 App Engine can apply restrictions to your code by running it in a software construct known as Sandbox.  Sandbox is independent of the hardware, Operating System, and Physical location of the server. Due to this, the App Engine can manage and scale your applications.  App Engine Standard composes certain constraints like one cannot write to the local file system.  If you are unable to install third-party software which is not suitable for you, then you should choose App Engine Flexible.

App Engine Flexible

 In case the App Engine Standard sandbox model constraints are not suitable for you, but you want to use App Engine, then you can choose App Engine Flexible environment.  App Engine Flexible will let you decide the container in which your application should run, unlike Sandbox.  It enables you to run Docker Containers with Compute Engine Virtual Machines. These VMs are health checked and healed, as necessary.  You can also choose the geographical location in which they have to run.  App Engine Flexible uses standard runtimes and also has access to services such as Data Store, Memcached, Task Queues, and so on.

App Engine Standard vs. App Engine

Flexible

Kubernetes Engine App Engine Standard App Engine Flexible Language Support Any Java, Python, Go, PHP Any Service Model Hybrid PaaS PaaS Primary Use- Case Container-based workloads Web and Mobile applications Both web and mobile applications and container-based workloads

 In addition to the above options, you can keep your code private and add IAM permissions to protect it by using the Cloud Source Repository (CSR).

Cloud Source Repository

 Cloud Source Repository provides the Git version control for the applications that also run on Compute Engine, App Engine and Kubernetes Engine.  You can have any number of Private Git Repositories with CSR.  Source Viewer in CSR helps you to browse and view the repository files through the GCP Console.  Event-driven actions can also be present in the applications. Let us consider an example. Suppose you have an app in which the users need to upload images.  For uploading the images, you need to process the image in various ways such as converting it to the standard image format, resizing it, and storing it in the repository.  You can integrate the above function into the application, but the major drawback is providing resources to it.

Cloud Functions

 Cloud Functions is a single purpose function that will perform image manipulations and run automatically when a new image is uploaded.  It can trigger events in Cloud Pub/Sub, Cloud Storage and HTTP call.  For each event type, you will specify which event you are interested in, and this type of declaration is called Triggers.  You will attach JavaScript functions to the triggers, and the functions respond whenever the event occurs.  Cloud Functions can be used to enhance the application without worrying about scaling.

Deployment

Setting up an environment in GCP will take a lot of steps like

  1. You can set up compute network, storage network, and their configurations.
  2. If you want to change the environment, you can do it with some commands.
  3. You can also clone the environment by executing some commands.  The above steps will take a lot of time and you can reduce it by using a Template. ( Requires specification for the environment that you want it to be).  GCP provides , Deployment Manager to automate the creation and management of the above steps by using the template.  You can create the Template file either by using YAML Markup language or Python. Then the template will be consumed by the Deployment Manager and performs the actions that need to be done.