gcsfuse

可直接將cloud storage bucket mount起來當作file system使用

安裝gcsfuse

# 1. Add the Cloud Storage FUSE distribution URL as a package source:
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb [signed-by=/usr/share/keyrings/cloud.google.asc] https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list

# 2. Import the Google Cloud public key.
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

# 3. install
sudo apt-get update
sudo apt-get install -y gcsfuse

Mount bucket

ref

登入gcloud
確定service account具有以下permission
1. roles/storage.objectViewer (read-only workload)
2. roles/storage.objectAdmin (read-write workload)
建立service account key, 並設定好環境變數GOOGLE_APPLICATION_CREDENTIALS指向該key

# command
gcsfuse {GLOBAL_OPTIONS} {BUCKET_NAME } {MOUNT_POINT}

# example
gcsfuse --implicit-dirs my-bucket /mnt/gcs-fuse

Unmount bucket

ref

fusermount -u /path/to/mount/point

docker-compose中使用

要在docker-compose mount gcsfuse

要設定privileged屬性為true

GKE autopilot中使用gcsfuse

因為autopilot mode不支援privileged

因此需要透過workload identity來設定

設定GKE Workload Identity

ref

以下Google service account簡稱GSA

Kubernetes service account簡稱KSA

在GSA與KSA間啟用IAM綁定

gcloud iam service-accounts add-iam-policy-binding {GSA_NAME}@{GSA_PROJECT_ID}.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:{GKE_PROJECT_ID}.svc.id.goog[{GKE_NAMESPACE}/{KSA_NAME}]"


# example
gcloud iam service-accounts add-iam-policy-binding foobar@project-1.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:project-1.svc.id.goog[default/default]"

設定annotation完成GSA與KSA間的綁定

kubectl annotate serviceaccount {KSA_NAME} \
    --namespace {NAMESPACE} \
    iam.gke.io/gcp-service-account={GSA_NAME}@{GSA_PROJECT}.iam.gserviceaccount.com

# example
kubectl annotate serviceaccount default \
    --namespace default \
    iam.gke.io/gcp-service-account=foobar@project-1.iam.gserviceaccount.com

GKE resource調整

ref

metadata:
  annotations:
    gke-gcsfuse/volumes: "true"
    # optional
    gke-gcsfuse/cpu-limit: 500m
    gke-gcsfuse/memory-limit: 1Gi
    gke-gcsfuse/ephemeral-storage-limit: 50Gi

透過volume掛載gcsfuse

ref

spec:
  containers: 
    - name: foobar
      volumeMounts:
        - name: config
          mountPath: /secrets/
        - name: gcs-fuse-csi-ephemeral
          mountPath: /gcs
  serviceAccountName: default
  volumes:
    - name: gcs-fuse-csi-ephemeral
      csi:
        driver: gcsfuse.csi.storage.gke.io
        readOnly: true
        volumeAttributes:
          bucketName: ciao-gcsfuse
          mountOptions: "implicit-dirs"

deploy完成後會發現所有pod會多一個container

Cloud Storage FUSE(gcsfuse)