Magnus K Karlsson: OpenShift 4.6 Automation and Integration: Storage

Overview

3.1. Persistent storage overview
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-overview_understanding-persistent-storage

The OpenShift storage architecture has three primary components:

Storage Classes
Persistent Volumes
Persistent Volume Claims

Persistent Volume Claims (pvc)

The project defines pvc with following

Storage Size: [G|Gi...]
Storage Class:
Access Mode: [ReadWriteMany|ReadWriteOnce|ReadOnlyMany]
Volume Mode: [Filesystem|Block|Object]

Persistent Volume (pv)

4.11. Persistent storage using NFS
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-nfs

Example Persistent Volume

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001
spec:
  capacity:
    storage: 5Gi
  storageClassName: nfs-storage
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  nfs:
    path: /tmp
    server: 172.17.0.2
  persistentVolumeReclaimPolicy: Retain

This persistent volume uses the NFS volume plug-in. The nfs section defines parameters that the NFS volume plug-in requires to mount the volume on a node. This section includes sensitive NFS configuration information.

Provisioning and Binding Persistent Volumes

Install a storage operator
Write and use Ansible Playbooks

Persistent Volume Reclaim Policy

3.2.6. Reclaim policy for persistent volumes
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#reclaiming_understanding-persistent-storage

Delete: reclaim policy deletes both the PersistentVolume object from OpenShift Container Platform and the associated storage asset in external infrastructure, such as AWS EBS or VMware vSphere. All dynamically-provisioned persistent volumes use a Delete reclaim policy.
Retain: Reclaim policy allows manual reclamation of the resource for those volume plug-ins that support it.
Recycle: Reclaim policy recycles the volume back into the pool of unbound persistent volumes once it is released from its claim.

Supported access modes for PVs

Table 3.2. Supported access modes for PVs
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#pv-access-modes_understanding-persistent-storage

Available dynamic provisioning plug-ins

7.2. Available dynamic provisioning plug-ins
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#available-plug-ins_dynamic-provisioning

Setting a Default Storage Class

7.3.2. Storage class annotations
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#storage-class-annotations_dynamic-provisioning

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"

Restricting Access to Storage Resources

5.1.1. Resources managed by quotas
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/applications/index#quotas-resources-managed_quotas-setting-per-project

requests.storage	The sum of storage requests across all persistent volume claims in any state cannot exceed this value.
persistentvolumeclaims	The total number of persistent volume claims that can exist in the project.
<storage-class-name>.storageclass.storage.k8s.io/requests.storage	The sum of storage requests across all persistent volume claims in any state that have a matching storage class, cannot exceed this value.
<storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaims	The total number of persistent volume claims with a matching storage class that can exist in the project.

Block Volume

3.5.1. Block volume examples
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#block-volume-examples_understanding-persistent-storage

apiVersion: v1
kind: PersistentVolume
metadata:
  name: block-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  volumeMode: Block 1
  persistentVolumeReclaimPolicy: Retain
  fc:
    targetWWNs: ["50060e801049cfd1"]
    lun: 0
    readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: block-pvc
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Block
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-with-block-volume
spec:
  containers:
    - name: fc-container
      image: fedora:26
      command: ["/bin/sh", "-c"]
      args: [ "tail -f /dev/null" ]
      volumeDevices: 
        - name: data
          devicePath: /dev/xvda
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: block-pvc

Persistent storage using iSCSI

4.9. Persistent storage using iSCSI
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#persistent-storage-using-iscsi

PersistentVolume object definition

apiVersion: v1
kind: PersistentVolume
metadata:
  name: iscsi-pv
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  storageClassName: iscsi-blk
  accessModes:
    - ReadWriteOnce
  iscsi:
    targetPortal: 10.0.0.1:3260
    iqn: iqn.2016-04.test.com:storage.target00
    lun: 0
    initiatorName: iqn.2016-04.test.com:custom.iqn 1
    fsType: ext4
    readOnly: false

Persistent storage using local volumes

Installing the Local Storage Operator

4.10.1. Installing the Local Storage Operator
https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html-single/storage/index#local-storage-install_persistent-storage-local

$ oc debug node/worker06 -- lsblk
...
vdb    252:16   0   20G  0 disk

$ oc adm new-project openshift-local-storage

$ OC_VERSION=$(oc version -o yaml | grep openshiftVersion | \
    grep -o '[0-9]*[.][0-9]*' | head -1)

apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
  name: local-operator-group
  namespace: openshift-local-storage
spec:
  targetNamespaces:
    - openshift-local-storage
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: local-storage-operator
  namespace: openshift-local-storage
spec:
  channel: "${OC_VERSION}"
  installPlanApproval: Automatic 1
  name: local-storage-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace

$ oc apply -f openshift-local-storage.yaml

Verify installation

$ oc -n openshift-local-storage get pods

$ oc get csv -n openshift-local-storage
NAME                                         DISPLAY         VERSION               REPLACES   PHASE
local-storage-operator.4.2.26-202003230335   Local Storage   4.2.26-202003230335              Succeeded

Provisioning local volumes by using the Local Storage Operator

$ export CSV_NAME=$(oc get csv -n openshift-local-storage -o name)

$ oc get ${CSV_NAME} -o jsonpath='{.spec.customresourcedefinitions.owned[*].kind}{"\n"}'
LocalVolume LocalVolumeSet LocalVolumeDiscovery LocalVolumeDiscoveryResult

$ oc get ${CSV_NAME} -o jsonpath='{.metadata.annotations.alm-examples}{"\n"}'
[
  {
    "apiVersion": "local.storage.openshift.io/v1",
    "kind": "LocalVolume",
    "metadata": {
      "name": "example"
    },
    "spec": {
      "storageClassDevices": [
        {
          "devicePaths": [
              "/dev/vde",
              "/dev/vdf"
          ],
          "fsType": "ext4",
          "storageClassName": "foobar",
          "volumeMode": "Filesystem"
        }
      ]
    }
  }
  ...
]

apiVersion: local.storage.openshift.io/v1
kind: LocalVolume
metadata:
  name: local-storage
spec:
  storageClassDevices:
  - devicePaths:
    - /dev/vdb
    fsType: ext4
    storageClassName: local-blk
    volumeMode: Filesystem

Magnus K Karlsson

November 2, 2022

OpenShift 4.6 Automation and Integration: Storage