Storage Services offered by Google Cloud Platform

GCP Storage Services

Published On: July 23, 2022
GCP Storage Services

Storage Types

Object Storage

Object or blob are the collection of data stored as a single entity. Objects are stored in a flat environment with no sub-directories. In GCP, objects are grouped in buckets and each object can be individually addressed by a URL like https://storage.cloud.google.com/BUCKET_NAME/OBJECT_NAME. The size of an object is not limited by the attached storage. Object has their metadata and object can have versions. Google Cloud Storage is a cloud storage service that provides object storage.

File Storage

In file storage, all the data is stored in a single file. File storage uses hierarchical directory structure. File storage is suitable for storing operating system files. File storage can also be stored on a network-attached storage like Cloud Filestore.

Block Storage

In block storage, data is divided into fixed sized blocks of data. Each block is stored in a separate file with unique identifiers. In Linux, the common size if a block is 4KB and in relational databases, the size of a block is greater than or equal to 8KB. Generally in GCP, block storage is used in ephemeral storage (short-lived) and persistent storage (long-lived) attached to a VM. Persistent Disk and Local SSD are a cloud storage services that provides block storage.


gcp-storage-products
gcp-storage-products

Storage Products

Cloud Storage

  • Cloud Storage is a GCP's storage service. It uses object storage to store data in containers called buckets.
  • The buckets are associated with project. The buckets can be accessed by a URL like https://storage.cloud.google.com/BUCKET_NAME.
  • The details of a object is stored in a metadata.
  • The objects are immutable, that means when the object is updated, it gets replaced by the new version.
  • When object versioning is enabled, old or deleted object can be retained as a noncurrent version.
  • The objects in a buckets can have different permissions for different users and services such as update, create or delete.
  • The Cloud Storage has multiple storage classes. Scroll down to read about storage classes.

Use cases

  • Media content storage & delivery: image, video, audio, documents, etc.
  • Integrated storage for analytics and ML.
  • Data backup and archival.

Cloud Filestore

  • Cloud Filestore uses file storage to store data in files. It is a network-attached storage.
  • Cloud Filestore offers low-latency and high-availability storage.
  • Cloud Filestore offers multiple service tiers.
  • Cloud FIletore supports NFSv3 protocol which supports asynchronous file operations.
  • This product is mainly used with Compute Engine VMs and Google Kubernetes Engine (GKE) clusters.

Service Tiers

  1. Basic HDD: The storage is stored on a hard disk. This is suitable for file sharing, software development, and use with GKE (Google Kubernetes Engine) workloads.
  2. Basic SSD: This ties similar to Basic HDD, but the storage is stored on a solid state drive. Basic SSD is more durable and has a higher performance at a higher cost.
  3. High Scale SSD: High Scale SSD is suitable for high performance computing (HPC) workloads. Genome sequencing, and financial analysis are examples of HPC workloads.
  4. Enterprise: Enterprise is a tier that provides a higher level of availability and data replication over multiple zones. This tier is designed for enterprise-grade Network File System (NFS) storage. Batch computing, media rendering and transcoding, advanced AI are examples of enterprise workloads.

Data Encryption

  1. Encryption at rest: Data is encrypted at rest. This means that the data is encrypted when it is stored in the cloud. This is the default encryption. The encryption key is managed by Google.
  2. Encryption in transit: The data is encrypted when it is transferred from the user to the storage in the cloud.

Use cases

  • Media rendering.
  • Web content management.
  • Enterprise application migration.

Local SSD

  • Local SSD uses block storage to store data in files. It is a local storage.
  • Local SSDs are ephemeral storage (short-lived) that is attached to a VM for higher performance.
  • Local SSD offers very high IOPS and low latency.
  • Local SSDs are used to store temporary data such as caches or scratch processing space.
  • The data stored in Local SSD is automatically encrypted at rest.
  • Stored data gets deleted when the VM is terminated.

Persistent Disk

  • Like Local SSD Persistent Disk uses block storage to store data in files.
  • Persistent Disks are dedicated HDDs and SSDs that are used for enterprise and database applications.
  • Persistent Disks are durable are high performance and can be used for long-lived storage.
  • Persistent Disks can be attached to VM instances or Google Compute Engine (GCE) clusters.
  • Persistent Disks are located independently of the VM instance.
  • Multiple VM instances can use the same Persistent Disk.

Types of Persistent Disks (Block Storage)

  1. Standard Persistent Disks (HDD)
  2. Balanced Persistent Disks (SSD)
  3. Performance Persistent Disks (SSD)
  4. Extreme Persistent Disks (SSD)
  5. Local SSDs

Comparing Storage Products

FieldCloud StorageCloud FilestoreLocal SSDPersistent Disk
ScalabilityYesYesNoYes
SharingRead/write from anywhereMountable on VMs, remote clients, and GKE clustersNot shareableLimited sharing
Encryption keysGoogle-managed, customer-managed, and customer suppliedGoogle-managed and customer-managedGoogle-managedGoogle-managed, customer-managed, and customer supplied
PersistentLifetime of the bucketLifetime of the instanceEphemeralLifetime of the disk
ManagementFully managedFully ManagedManually format, strip, and mountManually format and mount

Storage Classes

There are four types of storage classes in Cloud Storage. Each classes servers different purpose. The class-type is stored in the metadata of a object. Unless specified explicitly, the newly created object inherit the storage class from the bucket. The default storage class of a bucket is Standard storage.

Storage ClassAvailable SLAMin. Storage DurationUse Cases
Standard>99.99% (MR)
99.99% (R)
NoneContent Storage
Nearline99.95% (MR)
99.9% (R)
30 daysBackups
Coldline99.95% (MR)
99.9% (R)
90 daysDisaster Recovery
Archive99.95% (MR)
99.9% (R)
365 daysArchiving

*MR -> in Multi-regional, *R -> in Regional

Additional Classes

  • Multi-Regional storage
    • Objects can be stored in multi-regions or dual regions.
  • Regional storage
    • Objects are only stored in a chosen region.
  • Durable Reduced Availability (DRA) storage
    • In terms of availability, DRA has low availability SLA (99%).
    • DRA has higher pricing for operation.
    • The durability of DRA is higher than other classes.

References