MAS Deployment Series:  Creating a ReadWriteMany (RWX) NFS Dynamic Provisioner & Related Storage Class

A step-by-step guide for deploying the Kubernetes NFS CSI Driver on Red Hat OpenShift for IBM MAS environments

We are continuing our series of blog entries covering MAS deployment scenarios either left out or not appropriately addressed in the IBM Ansible scripts. This entry outlines how to establish a robust, file-level ReadWriteMany (RWX) Storage Class inside Red Hat OpenShift using the official Kubernetes NFS CSI Driver — written as a practical, step-by-step roadmap for system administrators using standard Bash environment variables, shell-evaluated manifests, and the native oc CLI tool. 

 References

The Kubernetes NFS CSI Driver upstream repository, configuration guides, and version-specific release notes are available at: https://github.com/kubernetes-csi/csi-driver-nfs

Architectural Considerations

Deploying IBM Maximo Application Suite (MAS) requires the availability of high-performing shared file system volumes configured with the ReadWriteMany (RWX) access mode. This is vital to enable concurrent pods — distributed across separate worker nodes — to simultaneously interact with shared file assets, doclinks attachments, report structures, and operational logging bundles.

While block-level storage is frequently fronted by cloud-native engines like OpenShift Data Foundation (ODF), utilizing a pre-existing enterprise NFS share is a valid, production-grade alternative. Before proceeding, review the following key design parameters:

Enterprise Production Validity

NFS shares are sometimes incorrectly labeled as sub-optimal for enterprise workloads. In practice, core cluster architectures regularly rely on abstracted shared storage filesystems — for instance, the native OpenShift internal Image Registry is routinely backed by S3 Object Storage tiers, which carry a significantly worse access latency profile than high-performance NFS arrays. High-performance NFS configurations fully meet production-grade specifications when backed by a true enterprise-tier storage appliance or an HA SAN/NAS infrastructure with hardware-accelerated NFS optimization.

Network Backplane & Interface Guardrails

Because NFS operational calls stream raw disk I/O directly over the data center network, cluster nodes must be provisioned with dedicated high-bandwidth, low-latency network interfaces. Ensuring interface separation prevents heavy storage bursts from saturating the primary node backplane, protecting inter-pod communication and control-plane heartbeat exchanges from packet drops or data starvation.

YAML Schema and Image Version Tight Coupling

The YAML manifests in this guide are strictly coupled with the specific, tested container image versions declared in Step 1. Kubernetes CSI driver specifications evolve rapidly. Implementing newer image versions or alternative tags may require structural variations or argument shifts. For upgrades, always verify your manifests against the official release templates at github.com/kubernetes-csi/csi-driver-nfs .

IBM ROKS Custom Kubelet Home Integration

Standard OpenShift deployments use /var/lib/kubelet as the root operational directory for host tracking, plugin socket structures, and CSI path mappings. However, managed cloud environments like IBM Cloud ROKS modify this location to /var/data/kubelet to decouple stateful cluster telemetry from core OS images.

Aligning the KUBELET_ROOT_DIR variable is critical. If misconfigured, the node DaemonSet will mount and register CSI paths inside a directory tree completely hidden from the active Kubelet daemon — leading to unresolvable plugin registration failures or permanent VolumeNotAttached scheduling loops.

Implications of Bidirectional Mount Propagation

Container runtimes strictly enforce namespace boundaries to isolate a container's filesystem from the host. The DaemonSet and Deployment volume mounts mapping back to the Kubelet host directory structure are explicitly designated with mountPropagation: "Bidirectional". This is vital because the NFS CSI driver executes the low-level Linux mount syscall inside its container namespace. Bidirectional propagation ensures that any mount point initialized inside the container is automatically surfaced into the host node's storage layer.

⚠ Critical — Never Omit Bidirectional Propagation

If mountPropagation is omitted or set to HostToContainer, NFS mount handles remain locked and invisible inside the transient driver container. The host worker node will be blind to the mount, causing all application workload PersistentVolumeClaims (PVCs) to fail or timeout indefinitely during volume mounting phases.

Database Tier Isolation

While high-performance NFS appliances meet production specs for general application assets, extreme sustained synchronous write workloads — such as the archive logging and active transactional paths of an IBM DB2 database engine — are best served by sub-millisecond block storage configurations. Assign relational database backplanes to dedicated high-IOPS block tiers, and safely offload application file directories, attachments, document repositories, and concurrent server tasks onto this NFS StorageClass.

Step-by-Step Configuration Pipeline

Step 1 Declare Environment Configuration Variables

Initialize your active terminal shell with the configuration criteria describing your backend storage targets, namespaces, container image tags, and host path layout. If your environment targets an IBM ROKS cluster, change KUBELET_ROOT_DIR to /var/data/kubelet.

# =========================================================================
# REQUIRED STORAGE PARAMETERS
# =========================================================================
export NFS_PROVISIONER_SERVER_HOSTNAME="10.240.0.55"            # Your NFS Server IP or FQDN
export NFS_PROVISIONER_SERVER_PATH="/var/nfs/mas-shares"        # Exposed NFS Export Path
export NFS_PROVISIONER_STORAGE_CLASS_NAME="managed-nfs-storage" # Desired SC Name

# =========================================================================
# SYSTEM INFRASTRUCTURE SETTINGS (MODIFY /var/data/kubelet FOR IBM ROKS)
# =========================================================================
export NFS_PROVISIONER_NAMESPACE="managed-nfs-storage"
export NFS_PROVISIONER_NAME="nfs.csi.k8s.io"
export NFS_PROVISIONER_NUM_REPLICAS="2"
export KUBELET_ROOT_DIR="/var/lib/kubelet"  # Use /var/data/kubelet for IBM ROKS

# =========================================================================
# CONTAINER REGISTRY IMAGES (TIED STRICTLY TO CORRESPONDING MANIFESTS)
# =========================================================================
export NFS_PROVISIONER_IMAGE_BASE="registry.k8s.io/sig-storage"
export NFS_PROVISIONER_IMAGE_PROVISIONER_VERSION="csi-provisioner:v6.1.0"
export NFS_PROVISIONER_IMAGE_RESIZER_VERSION="csi-resizer:v2.0.0"
export NFS_PROVISIONER_IMAGE_SNAPSHOTTER_VERSION="csi-snapshotter:v8.4.0"
export NFS_PROVISIONER_IMAGE_LIVENESSPROBE_VERSION="livenessprobe:v2.17.0"
export NFS_PROVISIONER_IMAGE_NFSPLUGIN_VERSION="nfsplugin:v4.0.2"
export NFS_PROVISIONER_IMAGE_NODEDRIVERREGISTRAR_VERSION="csi-node-driver-registrar:v2.13.0"

 Step 2 Allocate the Namespace and Configure Security Policy Contexts

Create the logical project namespace where the driver components will execute. Because the NFS CSI driver must interact directly with host kernels to perform network mounts, explicitly bind the driver's Service Accounts to OpenShift's hostmount-anyuid and privileged Security Context Constraints (SCC).

Step 3 Establish Role-Based Access Controls (RBAC)

The controller requires specific permissions to monitor PVC lifecycle events and register corresponding volumes. Apply the following manifests:

# Create the dedicated system project namespace
oc new-project ${NFS_PROVISIONER_NAMESPACE} \
  --description="Managed NFS Storage Provisioner Namespace"

# Apply elevated security definitions to allow low-level host volume mapping
oc adm policy add-scc-to-user hostmount-anyuid \
  system:serviceaccount:${NFS_PROVISIONER_NAMESPACE}:csi-nfs-controller-sa
oc adm policy add-scc-to-user privileged \
  system:serviceaccount:${NFS_PROVISIONER_NAMESPACE}:csi-nfs-controller-sa
oc adm policy add-scc-to-user hostmount-anyuid \
  system:serviceaccount:${NFS_PROVISIONER_NAMESPACE}:csi-nfs-node-sa
oc adm policy add-scc-to-user privileged \
  system:serviceaccount:${NFS_PROVISIONER_NAMESPACE}:csi-nfs-node-sa

cat <<EOF | oc apply -f -
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-nfs-controller-sa
  namespace: ${NFS_PROVISIONER_NAMESPACE}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-nfs-node-sa
  namespace: ${NFS_PROVISIONER_NAMESPACE}
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-external-provisioner-role
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "patch", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["snapshot.storage.k8s.io"]
    resources: ["volumesnapshotclasses", "volumesnapshots"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["snapshot.storage.k8s.io"]
    resources: ["volumesnapshotcontents"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["snapshot.storage.k8s.io"]
    resources: ["volumesnapshotcontents/status"]
    verbs: ["get", "update", "patch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-csi-provisioner-binding
subjects:
  - kind: ServiceAccount
    name: csi-nfs-controller-sa
    namespace: ${NFS_PROVISIONER_NAMESPACE}
roleRef:
  kind: ClusterRole
  name: nfs-external-provisioner-role
  apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-external-resizer-role
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims/status"]
    verbs: ["update", "patch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-csi-resizer-binding
subjects:
  - kind: ServiceAccount
    name: csi-nfs-controller-sa
    namespace: ${NFS_PROVISIONER_NAMESPACE}
roleRef:
  kind: ClusterRole
  name: nfs-external-resizer-role
  apiGroup: rbac.authorization.k8s.io
EOF

 Step 4 Register the CSI Driver and Deploy the Controller

Define the cluster-wide CSIDriver resource and launch the controller deployment. Shell execution naturally maps the exported configuration variables into the resource manifests:

cat <<EOF | oc apply -f -
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
  name: nfs.csi.k8s.io
spec:
  attachRequired: false
  volumeLifecycleModes:
    - Persistent
  fsGroupPolicy: File
EOF

cat <<EOF | oc apply -f -
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: csi-nfs-controller
  namespace: ${NFS_PROVISIONER_NAMESPACE}
spec:
  replicas: ${NFS_PROVISIONER_NUM_REPLICAS}
  selector:
    matchLabels:
      app: csi-nfs-controller
  template:
    metadata:
      labels:
        app: csi-nfs-controller
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: csi-nfs-controller-sa
      priorityClassName: system-cluster-critical
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: csi-provisioner
          image: ${NFS_PROVISIONER_IMAGE_BASE}/${NFS_PROVISIONER_IMAGE_PROVISIONER_VERSION}
          args:
            - "-v=2"
            - "--csi-address=\$(ADDRESS)"
            - "--leader-election"
            - "--leader-election-namespace=\$(POD_NAMESPACE)"
            - "--extra-create-metadata=true"
          env:
            - name: ADDRESS
              value: /csi/csi.sock
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          volumeMounts:
            - mountPath: /csi
              name: socket-dir
        - name: nfs
          image: ${NFS_PROVISIONER_IMAGE_BASE}/${NFS_PROVISIONER_IMAGE_NFSPLUGIN_VERSION}
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
              drop: ["ALL"]
            allowPrivilegeEscalation: true
          args:
            - "-v=5"
            - "--nodeid=\$(NODE_ID)"
            - "--endpoint=\$(CSI_ENDPOINT)"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix:///csi/csi.sock
          volumeMounts:
            - name: pods-mount-dir
              mountPath: ${KUBELET_ROOT_DIR}/pods
              mountPropagation: "Bidirectional"
            - mountPath: /csi
              name: socket-dir
      volumes:
        - name: pods-mount-dir
          hostPath:
            path: ${KUBELET_ROOT_DIR}/pods
            type: Directory
        - name: socket-dir
          hostPath:
            path: ${KUBELET_ROOT_DIR}/plugins/csi-nfsplugin
            type: DirectoryOrCreate
EOF

 Step 5 Instantiate the CSI Node DaemonSet Agent

The DaemonSet ensures every active cluster node runs an instance of the mount driver. Note how mountPropagation: "Bidirectional" is explicitly applied to the pods-mount-dir workspace, allowing host operating systems to access containerized file handles seamlessly.

cat <<EOF | oc apply -f -
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: csi-nfs-node
  namespace: ${NFS_PROVISIONER_NAMESPACE}
spec:
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  selector:
    matchLabels:
      app: csi-nfs-node
  template:
    metadata:
      labels:
        app: csi-nfs-node
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: csi-nfs-node-sa
      priorityClassName: system-node-critical
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      tolerations:
        - operator: "Exists"
      containers:
        - name: nfs
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
              drop: ["ALL"]
            allowPrivilegeEscalation: true
          image: ${NFS_PROVISIONER_IMAGE_BASE}/${NFS_PROVISIONER_IMAGE_NFSPLUGIN_VERSION}
          args:
            - "-v=5"
            - "--nodeid=\$(NODE_ID)"
            - "--endpoint=\$(CSI_ENDPOINT)"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix:///csi/csi.sock
          volumeMounts:
            - name: socket-dir
              mountPath: /csi
            - name: pods-mount-dir
              mountPath: ${KUBELET_ROOT_DIR}/pods
              mountPropagation: "Bidirectional"
      volumes:
        - name: socket-dir
          hostPath:
            path: ${KUBELET_ROOT_DIR}/plugins/csi-nfsplugin
            type: DirectoryOrCreate
        - name: pods-mount-dir
          hostPath:
            path: ${KUBELET_ROOT_DIR}/pods
            type: Directory
EOF

Step 6 Register and Configure the Target StorageClass

Apply the StorageClass resource to initialize dynamic subfolder routing on the backend NFS share:

cat <<EOF | oc apply -f -
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ${NFS_PROVISIONER_STORAGE_CLASS_NAME}
provisioner: ${NFS_PROVISIONER_NAME}
parameters:
  server: ${NFS_PROVISIONER_SERVER_HOSTNAME}
  share: ${NFS_PROVISIONER_SERVER_PATH}
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - nfsvers=4.1
EOF

To enforce default cluster targeting for unassigned volume declarations, apply the following annotation patch:

oc patch storageclass ${NFS_PROVISIONER_STORAGE_CLASS_NAME} \
  -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Validation — End-to-End Functional Test

Confirm that the driver control layer successfully translates storage allocations by creating a temporary PersistentVolumeClaim (PVC):

cat <<EOF | oc apply -f -
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-nfs-rwx-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ${NFS_PROVISIONER_STORAGE_CLASS_NAME}
  resources:
    requests:
      storage: 1Gi
EOF

Query the real-time allocation state of the test claim and confirm it reaches Bound status: 

oc get pvc test-nfs-rwx-pvc -n default

NAME                     STATUS    VOLUME                                                                        CAPACITY       ACCESS MODES   
test-nfs-rwx-pvc   Bound      pvc-8d76c123-4567-89ab-cdef-123456789abc      1Gi                   RWX                       

STORAGECLASS                AGE            
managed-nfs-storage       4                

Once validation is complete, remove the test claim to trigger the automated resource reclamation loop:

oc delete pvc test-nfs-rwx-pvc -n default

Because the StorageClass specifies reclaimPolicy: Delete, destroying the PVC will automatically trigger the underlying CSI controllers to cleanly remove the associated subdirectory segment from your backend NFS storage engine.