PhoenixAI Kubernetes Operator supports mounting persistent volumes to PhoenixAI FE and CN pods. If not specified, the operator will use emptyDir mode to store FE meta and CN cache data. When container restarts, the data will be lost.
This document describes how to mount persistent volumes to PhoenixAI FE and CN pods. There are two ways to mount persistent volumes to PhoenixAI FE and CN pods:
Note: phoenixai operator will create a new PVC for each storageVolume. You should not create PVC manually.
If you want to use external storage to store FE meta and CN cache data for persistence, you can specify storageVolumes in
the corresponding component spec.
The following is an example of mounting persistent volumes to PhoenixAI FE and CN.
apiVersion: phoenixdata.ai/v1
kind: PhoenixAICluster
metadata:
name: kube-anywhere
namespace: phoenixai
labels:
cluster: kube-anywhere
spec:
phoenixAIFeSpec:
image: "us-west1-docker.pkg.dev/phoenix-ai-images/enterprise/fe-ubuntu:<database-image-tag>"
replicas: 1
storageVolumes:
- name: fe-meta
storageClassName: standard-rwo # standard-rwo is the default storageClassName in GKE.
# fe container stop running if the disk free space which the fe meta directory residents, is less than 5Gi.
storageSize: 10Gi
mountPath: /opt/starrocks/fe/meta
- name: fe-log
storageClassName: standard-rwo
storageSize: 5Gi
mountPath: /opt/starrocks/fe/log
phoenixAICnSpec:
image: "us-west1-docker.pkg.dev/phoenix-ai-images/enterprise/cn-ubuntu:<database-image-tag>"
replicas: 3
storageVolumes:
- name: cn-data
storageClassName: standard-rwo
storageSize: 1Ti
mountPath: /opt/starrocks/cn/storage
- name: cn-log
storageClassName: standard-rwo
storageSize: 1Gi
mountPath: /opt/starrocks/cn/log
Note that the specific storageClassName should be available in kubernetes cluster before enabling this storageVolume
feature. If StorageVolume info is not specified in CRD spec, the operator will use emptydir mode to store FE meta and
CN cache data.
See Step 2 — Get the chart to learn how to add the Helm chart repo for PhoenixAI. In this
guide, we will use phoenixai/kube-anywhere chart to deploy both PhoenixAI operator and cluster.
The values.yaml file contains the default configurations for the PhoenixAI Operator and the PhoenixAI cluster.
helm show values phoenixai/kube-anywhere > values.yaml
The following is a snippet of the values.yaml file:
phoenixai:
phoenixAIFeSpec: # fe storageSpec for persistent metadata.
storageSpec:
name: "fe"
# the storageClassName represent the used storageclass name. if not set will use k8s cluster default storageclass.
# you must set name when you set storageClassName
# storageClassName: ""
# the persistent volume size， default 10Gi.
# fe container stop running if the disk free space which the fe meta directory residents, is less than 5Gi.
storageSize: 10Gi
# Setting this parameter can persist log storage
logStorageSize: 5Gi
phoenixAICnSpec: # specify storageclass name and request size.
storageSpec: # the name of volume for mount. if not will use emptyDir.
name: ""
# the storageClassName represent the used storageclass name. if not set will use k8s cluster default storageclass.
# you must set name when you set storageClassName
storageClassName: ""
storageSize: 1Ti
# Setting this parameter can persist log storage
logStorageSize: 1Gi
storageSpec.name is the switch for its whole block. While it is "" — the shipped CN default —
no PersistentVolumeClaim is created and every other field beside it is ignored: storageClassName,
storageSize, storageCount, storageMountPath and logStorageSize all have no effect, and
nothing reports that they were ignored. Set the name first, then the rest of the block takes effect.
The following is an example of a custom values.yaml with storageSpec settings:
phoenixai:
phoenixAIFeSpec:
image:
repository: us-west1-docker.pkg.dev/phoenix-ai-images/enterprise/fe-ubuntu
tag: <database-image-tag>
storageSpec:
name: fe-data
storageClassName: standard-rwo # standard-rwo is the default storageClassName in GKE.
logStorageSize: 10Gi
storageSize: 100Gi
phoenixAICnSpec:
image:
repository: us-west1-docker.pkg.dev/phoenix-ai-images/enterprise/cn-ubuntu
tag: <database-image-tag>
replicas: 3
storageSpec:
name: cn-storage
storageClassName: standard-rwo
logStorageSize: 10Gi
storageSize: 500Gi
See Install PhoenixAI by kube-anywhere chart to learn how to deploy PhoenixAI Operator and Cluster
Normally, the storageClassName is the name of the StorageClass that you want to use for the PersistentVolumeClaim.
We have also provided some special storageClassName for you to use:
emptyDir. It is a good choice when you want to mount a volume into the container for temporary usage, e.g. /tmp. Be aware that the files and directories written to the volume will be completely lost upon container restarting.hostPath. It is a good choice when you want to the host’s storage for the container, the data will be still there as along as the container is still running on the host. The data will be unavailable upon the container rescheduling to a different host. The typical scenario is to use it as cache volume. The hostPath field is required when this type is used.
field.
e.g.:
storageVolumes:
- name: cn-cache
storageClassName: "hostPath"
hostPath:
path: /storage
mountPath: /storage
Note: In both cases, the
storageSizefield will be ignored.