Hi All,
I am facing an issue i.e PV is not getting attached/mounted to Elasticsearch pod.
Background:
I have setup Elasticsearch cluster using eck operator. Details are as follows. Cluster is setup in OKE(Oracle kubernetes engine).
ES Version: 6.8.21Master Nodes: 3Data nodes 9
Each data node Volume is 500GB.
Everynow and then we are facing issues with storage. Hence I decided to update the storage from 500GB to 750GB.
So I updated the size of PVC of data-0 which updated its respective PV. Then I restarted the data-0 pod however the pod is not going into running state. Then I had manually updated the Block Volume from 500GB to 750 GB and then tried. Same error. Error is follows.
Error
MountVolume.MountDevice failed while expanding volume for volume "<ocid id>" : MountVolume.NodeExpandVolume failed : expandfs command failed, status: Failure, reason: Invalid command; got [/usr/libexec/kubernetes/kubelet-plugins/volume/exec/oracle~oci/oci expandfs {"kubernetes.io/fsType":"ext4","kubernetes.io/pvOrVolumeName":" <ocid id>","kubernetes.io/readwrite":"rw "} /dev/disk/by-path/<redacted> /var/lib/kubelet/plugins/kubernetes.io/flexvolume/oracle/oci/mounts/<ocid-id>]
<ocid-id> -> Oracle cloud identifier similar to ARN in Amazon.
I know this is something related OCI stuff. I just want to know if anyone run into this issue before. Any guidance or help on this?
Hi Elastic team,
Could you please provide your suggestions on this?
I think this init container "elastic-internal-init-filesystem" is trying to prepare the file system and not able to pick up the changes done manually(PVC, PV, Underlying block volume). even after replacing this init container command content with something else(i.e echo "hello") it is not working. Kindly do the needful.
opened 06:44AM - 16 Jun 21 UTC
feature
## Is this a BUG REPORT or FEATURE REQUEST?
Choose one: BUG REPORT or FEATURE… REQUEST
<!--
If this is a BUG REPORT, please:
- Fill in as much of the template below as you can. If you leave out information, we can't help you as well.
If this is a FEATURE REQUEST, please:
- Describe *in detail* the feature/behaviour/change you'd like to see.
If we can't reproduce a bug or think a feature already exists, we
might close your issue. If we're wrong, PLEASE feel free to reopen it and
explain why.
-->
## Versions
**CCM Version**:
v 0.9
**Environment**:
- **Kubernetes version** (use `kubectl version`):
```
ubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:28:32Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
```
- **OS** (e.g. from /etc/os-release):
```
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
```
- **Kernel** (e.g. `uname -a`):
```
Linux inst-tnknj-dev-kube-instgrp-master-infpools-org 5.4.0-1035-oracle #38~18.04.1-Ubuntu SMP Wed Jan 6 21:03:06 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
```
- **Others**:
## What happened?
I have installed the OCI CSI plugin ,and set the allowVolumeExpansion: true on the StorageClass, but it cannot resize the pvc
```
kubectl describe sc oci-bv
Name: oci-bv
IsDefaultClass: No
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"oci-bv"},"provisioner":"blockvolume.csi.oraclecloud.com","reclaimPolicy":"Delete","volumeBindingMode":"WaitForFirstConsumer"}
Provisioner: blockvolume.csi.oraclecloud.com
Parameters: <none>
AllowVolumeExpansion: True
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: WaitForFirstConsumer
Events: <none>
```
## What you expected to happen?
We are expecting that the pvc can be resized with "oci-bv" with AllowVolumeExpansion: True
## How to reproduce it (as minimally and precisely as possible)?
1. Set the allowVolumeExpansion: true on the storageClass
```
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"oci-bv"},"provisioner":"blockvolume.csi.oraclecloud.com","reclaimPolicy":"Delete","volumeBindingMode":"WaitForFirstConsumer"}
creationTimestamp: "2021-06-16T05:39:38Z"
name: oci-bv
resourceVersion: "4461"
selfLink: /apis/storage.k8s.io/v1/storageclasses/oci-bv
uid: 9a257ec6-c8d1-428e-9797-c2a7fb3f350a
provisioner: blockvolume.csi.oraclecloud.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
oci-bv blockvolume.csi.oraclecloud.com Delete WaitForFirstConsumer true 60m
```
2. Request a change in volume capacity by editing the PersistentVolumeClaim's spec.resources.requests.storage field. For example, change the following PVC from having a 50 gibibyte (GiB) disk to having a 104 GiB disk.
```
# pvc-demo.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-demo
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
```
After editing:
```
# pvc-demo.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-demo
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 104Gi
```
3. After that, should see the new volume in the status.capacity field get updated, but it still show the same
```spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 104Gi
storageClassName: oci-bv
volumeMode: Filesystem
volumeName: csi-ef97ef8e-1a3a-4b47-95e1-b17d42a0a183
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
phase: Bound
```
4.The storage capacity is still 50Gi, did not get changed
## Anything else we need to know?
From what I'm reading, volume expansion isn't supported within Oracle's CSI.
system
March 22, 2022, 3:13pm
4
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.