19 changed files with 897 additions and 0 deletions
@ -0,0 +1,21 @@ |
|||||
|
annotations: |
||||
|
catalog.cattle.io/certified: rancher |
||||
|
catalog.cattle.io/display-name: vSphere CSI |
||||
|
catalog.cattle.io/namespace: vmware-system-csi |
||||
|
catalog.cattle.io/os: linux |
||||
|
catalog.cattle.io/rancher-version: '>= 2.5.0-0 <= 2.5.99-0' |
||||
|
catalog.cattle.io/release-name: vsphere-csi |
||||
|
apiVersion: v1 |
||||
|
appVersion: 2.3.1 |
||||
|
description: vSphere Cloud Storage Interface (CSI) |
||||
|
icon: https://charts.rancher.io/assets/logos/vsphere-csi.svg |
||||
|
keywords: |
||||
|
- infrastructure |
||||
|
kubeVersion: 1.19 - 1.21 |
||||
|
maintainers: |
||||
|
- email: caleb@rancher.com |
||||
|
name: Rancher |
||||
|
name: rancher-vsphere-csi |
||||
|
sources: |
||||
|
- https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
version: 2.3.1 |
||||
@ -0,0 +1,79 @@ |
|||||
|
# vSphere Container Storage Interface (CSI) |
||||
|
|
||||
|
[vSphere Container Storage Interface (CSI)](https://github.com/kubernetes-sigs/vsphere-csi-driver/tree/release-2.1/manifests/v2.1.0/vsphere-7.0u1/) is a specification designed to enable persistent storage volume management on Container Orchestrators (COs) such as Kubernetes. The specification allows storage systems to integrate with containerized workloads running on Kubernetes. Using CSI, storage providers, such as VMware, can write and deploy plugins for storage systems in Kubernetes without a need to modify any core Kubernetes code. |
||||
|
|
||||
|
CSI allows volume plugins to be installed on Kubernetes clusters as extensions. Once a CSI compatible volume driver is deployed on a Kubernetes cluster, users can use the CSI to provision, attach, mount, and format the volumes exposed by the CSI driver. |
||||
|
|
||||
|
The CSI driver for vSphere is `csi.vsphere.vmware.com`. |
||||
|
|
||||
|
## Prerequisites |
||||
|
|
||||
|
- vSphere 6.7 U3+ |
||||
|
- Kubernetes v1.14+ |
||||
|
- Out-of-tree vSphere Cloud Provider Interface (CPI) |
||||
|
- A Secret on your Kubernetes cluster that contains vSphere CSI configuration and credentials |
||||
|
|
||||
|
## Installation |
||||
|
|
||||
|
This chart requires a Secret in your Kubernetes cluster that contains the CSI configuration and credentials to connect to the vCenter. You can have the chart generate it for you, or create it yourself and provide the name of the Secret during installation. |
||||
|
|
||||
|
<span style="color:orange">Warning</span>: When the option to generate the Secret is enabled, the credentials are visible in the API to authorized users. If you create the Secret yourself they will not be visible. |
||||
|
|
||||
|
You can create a Secret in one of the following ways: |
||||
|
|
||||
|
### <B>Option 1</b>: Create a Secret using the Rancher UI |
||||
|
|
||||
|
Go to your cluster's project (Same project you will be installing the chart) > Resources > Secrets > Add Secret. |
||||
|
```yaml |
||||
|
# Example of data required in the Secret |
||||
|
# The csi-vsphere.conf key name is required, otherwise the installation will fail |
||||
|
csi-vsphere.conf: | |
||||
|
[Global] |
||||
|
cluster-id = "<cluster-id>" |
||||
|
user = "<username>" |
||||
|
password = "<password>" |
||||
|
port = "<port>" |
||||
|
insecure-flag = "<insecure-flag>" |
||||
|
|
||||
|
[VirtualCenter "<host>"] |
||||
|
datacenters = "<dc-1>, <dc-2>, ..." |
||||
|
``` |
||||
|
More information on CSI vSphere configuration [here](https://vsphere-csi-driver.sigs.k8s.io/driver-deployment/installation.html#create_k8s_secret). |
||||
|
|
||||
|
### <B>Option 2</b>: Create a Secret using kubectl |
||||
|
|
||||
|
Replace placeholders with actual values, and execute the following: |
||||
|
```bash |
||||
|
# The csi-vsphere.conf key name is required, otherwise the installation will fail |
||||
|
cat <<EOF | kubectl apply -f - |
||||
|
apiVersion: v1 |
||||
|
kind: Secret |
||||
|
type: Opaque |
||||
|
metadata: |
||||
|
name: <secret-name> |
||||
|
namespace: <charts-namespace> |
||||
|
stringData: |
||||
|
csi-vsphere.conf: | |
||||
|
[Global] |
||||
|
cluster-id = "<cluster-id>" |
||||
|
user = "<username>" |
||||
|
password = "<password>" |
||||
|
port = "<port>" |
||||
|
insecure-flag = "<insecure-flag>" |
||||
|
|
||||
|
[VirtualCenter "<host>"] |
||||
|
datacenters = "<dc-1>, <dc-2>, ..." |
||||
|
EOF |
||||
|
``` |
||||
|
|
||||
|
More information on managing Secrets using kubectl [here](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/). |
||||
|
|
||||
|
## Migration |
||||
|
|
||||
|
The CSI migration feature is only available for vSphere 7.0 U1. |
||||
|
Follow the steps [here](https://rancher.com/docs/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/out-of-tree/vsphere-volume-migration/?#) for CSI Migration. |
||||
|
|
||||
|
## Upgrade Kubernetes version after Migration |
||||
|
Follow the steps in the [Upgrading Kubernetes](https://rancher.com/docs/rancher/v2.5/en/cluster-admin/upgrading-kubernetes/) guide to upgrade the Kubernetes version after migration. |
||||
|
|
||||
|
**Important:** If you have a workload with a bound volume attached to it during an upgrade, you will see the pods get redeployed and the old pods will get stuck in a **Terminating** state. You can safely delete the old stuck pods by executing the command `kubectl delete pod <pod_id> --force=true --timeout=120s --grace-period=0` to fix this problem, but it's imperative you verify before deleting a pod that there is a new active pod corresponding to the old pod you want to delete. Deleting an old pod before a new one has been deployed could result in loss of data and bound volumes (Due to `--grace-period=0`). |
||||
@ -0,0 +1,23 @@ |
|||||
|
# vSphere Container Storage Interface (CSI) |
||||
|
|
||||
|
[vSphere Container Storage Interface (CSI)](https://github.com/kubernetes-sigs/vsphere-csi-driver) is a specification designed to enable persistent storage volume management on Container Orchestrators (COs) such as Kubernetes. The specification allows storage systems to integrate with containerized workloads running on Kubernetes. Using CSI, storage providers, such as VMware, can write and deploy plugins for storage systems in Kubernetes without a need to modify any core Kubernetes code. |
||||
|
|
||||
|
CSI allows volume plugins to be installed on Kubernetes clusters as extensions. Once a CSI compatible volume driver is deployed on a Kubernetes cluster, users can use the CSI to provision, attach, mount, and format the volumes exposed by the CSI driver. |
||||
|
|
||||
|
The CSI driver for vSphere is `csi.vsphere.vmware.com`. |
||||
|
|
||||
|
## Support Matrix |
||||
|
|
||||
|
| Rancher vSphere CSI | Minimum Kubernetes Release | Maximum Kubernetes Release | |
||||
|
| ------------------- | -------------------------- | -------------------------- | |
||||
|
| 2.3.1 | 1.19.x | 1.21.x | |
||||
|
| 2.3.0 | 1.19.x | 1.21.x | |
||||
|
| 2.1.1 | | 1.19.x | |
||||
|
| 2.1.000 | | 1.19.x | |
||||
|
|
||||
|
## Prerequisites |
||||
|
|
||||
|
- vSphere 6.7 U3+ |
||||
|
- Kubernetes v1.14+ |
||||
|
- Out-of-tree vSphere Cloud Provider Interface (CPI) |
||||
|
- A Secret on your Kubernetes cluster that contains vSphere CSI configuration and credentials (Refer to `README` or `Detailed Descriptions`) |
||||
@ -0,0 +1,134 @@ |
|||||
|
questions: |
||||
|
- variable: vCenter.configSecret.generate |
||||
|
label: Generate CSI vSphere Config Secret |
||||
|
description: Generates a Secret that contains a CSI vSphere config and credentials (If the option to generate it is enabled, credentials will be visible in the API to authorized users) |
||||
|
type: boolean |
||||
|
default: true |
||||
|
required: true |
||||
|
group: vCenter Configuration |
||||
|
show_subquestion_if: true |
||||
|
subquestions: |
||||
|
- variable: vCenter.host |
||||
|
label: vCenter Host |
||||
|
description: IP address or FQDN of the vCenter |
||||
|
type: string |
||||
|
|
||||
|
- variable: vCenter.datacenters |
||||
|
description: Comma-separated list of paths to data centers. E.g "<dc1-path>, <dc2-path>, ..." |
||||
|
label: Data Centers |
||||
|
type: string |
||||
|
|
||||
|
- variable: vCenter.username |
||||
|
label: Username |
||||
|
description: Username for vCenter |
||||
|
type: string |
||||
|
|
||||
|
- variable: vCenter.password |
||||
|
label: Password |
||||
|
description: Password for vCenter |
||||
|
type: password |
||||
|
|
||||
|
- variable: vCenter.configSecret.name |
||||
|
label: CSI vSphere Config Secret Name |
||||
|
description: Name of the Secret that contains a CSI vSphere config and credentials (Will not be visible in the API. More info in the README) |
||||
|
type: string |
||||
|
group: vCenter Configuration |
||||
|
show_if: "vCenter.configSecret.generate=false" |
||||
|
|
||||
|
- variable: csiMigration.enabled |
||||
|
label: Enable CSI Migration |
||||
|
description: Enable migration of volumes provisioned by in-tree vSphere provider to CSI (Available for vSphere 7.0 U1+ only) |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: csiAuthCheck.enabled |
||||
|
label: Enable authorization checks on operations involving datastores |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: onlineVolumeExtend.enabled |
||||
|
label: Enable Online Volume Extend |
||||
|
description: Enable expansion of PVCs that are in use by a Pod or mounted in a Node (Available for vSphere 7.0 U2+ only) |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: triggerCsiFullsync.enabled |
||||
|
label: Enable CSI Full Sync |
||||
|
description: Keeps CNS up to date with Kubernetes volume metadata information (such as PVs, PVCs, pods, and so on) |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: asyncQueryVolume.enabled |
||||
|
label: Enable Async Query Volume |
||||
|
description: Improves retrieval of volume information |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: improvedCsiIdempotency.enabled |
||||
|
label: Enable Improved CSI Idempotency |
||||
|
description: Enhances driver to ensure volume operations are idempotent |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: improvedVolumeTopology.enabled |
||||
|
label: Enable Improved Volume Topology |
||||
|
description: Allows using the topology feature without the need to mount vSphere credentials in the CSI node daemonset |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Driver Configuration |
||||
|
|
||||
|
- variable: csiController.csiResizer.enabled |
||||
|
label: Enable CSI Volume Resizer |
||||
|
description: This feature is available for vSphere 7.0 U1+ only |
||||
|
type: boolean |
||||
|
default: false |
||||
|
group: Storage |
||||
|
|
||||
|
- variable: storageClass.enabled |
||||
|
default: true |
||||
|
label: Create Storage Class |
||||
|
description: Create a storageClass with the vSphere CSI provisioner |
||||
|
type: boolean |
||||
|
required: true |
||||
|
show_subquestion_if: true |
||||
|
group: Storage |
||||
|
subquestions: |
||||
|
- variable: storageClass.name |
||||
|
label: Storage Class Name |
||||
|
default: "vsphere-csi-sc" |
||||
|
type: string |
||||
|
|
||||
|
- variable: storageClass.isDefault |
||||
|
label: Default Storage Class |
||||
|
description: Set the Storage Class as the default |
||||
|
default: true |
||||
|
type: boolean |
||||
|
|
||||
|
- variable: storageClass.allowVolumeExpansion |
||||
|
label: Allow Volume Expansion |
||||
|
description: Allows resizing the volume by editing the corresponding PVC object (Available for vSphere 7.0+ only) |
||||
|
default: false |
||||
|
type: boolean |
||||
|
|
||||
|
- variable: storageClass.storagePolicyName |
||||
|
label: Storage Policy Name |
||||
|
description: Name of the Storage Policy created in vCenter |
||||
|
type: string |
||||
|
|
||||
|
- variable: storageClass.datastoreURL |
||||
|
label: Data Store URL |
||||
|
description: URL of the data store to use for new volumes (If unspecified, any data store that matches the request will be selected). |
||||
|
type: string |
||||
|
|
||||
|
- variable: csiNode.prefixPath |
||||
|
label: Prefix Path for `/var/lib/kubelet` |
||||
|
description: For some operating systems including RancherOS, RKE prefixes `/var/lib/kubelet` with `/opt/rke`. Add the prefix path of the location of /var/lib/kubelet |
||||
|
type: string |
||||
|
default: "" |
||||
|
group: Node Configuration |
||||
@ -0,0 +1,7 @@ |
|||||
|
{{- define "system_default_registry" -}} |
||||
|
{{- if .Values.global.cattle.systemDefaultRegistry -}} |
||||
|
{{- printf "%s/" .Values.global.cattle.systemDefaultRegistry -}} |
||||
|
{{- else -}} |
||||
|
{{- "" -}} |
||||
|
{{- end -}} |
||||
|
{{- end -}} |
||||
@ -0,0 +1,14 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
apiVersion: v1 |
||||
|
data: |
||||
|
"csi-migration": {{ .Values.csiMigration.enabled | quote }} |
||||
|
"csi-auth-check": {{ .Values.csiAuthCheck.enabled | quote }} |
||||
|
"online-volume-extend": {{ .Values.onlineVolumeExtend.enabled | quote }} |
||||
|
"trigger-csi-fullsync": {{ .Values.triggerCsiFullsync.enabled | quote }} |
||||
|
"async-query-volume": {{ .Values.asyncQueryVolume.enabled | quote }} |
||||
|
"improved-csi-idempotency": {{ .Values.improvedCsiIdempotency.enabled | quote }} |
||||
|
"improved-volume-topology": {{ .Values.improvedVolumeTopology.enabled | quote }} |
||||
|
kind: ConfigMap |
||||
|
metadata: |
||||
|
name: internal-feature-states.csi.vsphere.vmware.com |
||||
|
namespace: {{ .Release.Namespace }} |
||||
@ -0,0 +1,208 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: Deployment |
||||
|
apiVersion: apps/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-controller |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
spec: |
||||
|
replicas: 1 |
||||
|
selector: |
||||
|
matchLabels: |
||||
|
app: vsphere-csi-controller |
||||
|
template: |
||||
|
metadata: |
||||
|
labels: |
||||
|
app: vsphere-csi-controller |
||||
|
role: vsphere-csi |
||||
|
spec: |
||||
|
serviceAccountName: vsphere-csi-controller |
||||
|
{{- if .Values.csiController.nodeSelector }} |
||||
|
nodeSelector: |
||||
|
{{- with .Values.csiController.nodeSelector }} |
||||
|
{{- toYaml . | nindent 8 }} |
||||
|
{{- end }} |
||||
|
{{- else }} |
||||
|
affinity: |
||||
|
nodeAffinity: |
||||
|
requiredDuringSchedulingIgnoredDuringExecution: |
||||
|
nodeSelectorTerms: |
||||
|
- matchExpressions: |
||||
|
# RKE node selector label |
||||
|
- key: node-role.kubernetes.io/controlplane |
||||
|
operator: In |
||||
|
values: |
||||
|
- "true" |
||||
|
- matchExpressions: |
||||
|
# RKE2 node selector label |
||||
|
- key: node-role.kubernetes.io/control-plane |
||||
|
operator: In |
||||
|
values: |
||||
|
- "true" |
||||
|
{{- end }} |
||||
|
{{- if .Values.csiController.tolerations }} |
||||
|
tolerations: |
||||
|
{{- with .Values.csiController.tolerations }} |
||||
|
{{- toYaml . | nindent 8 }} |
||||
|
{{- end }} |
||||
|
{{- else }} |
||||
|
tolerations: |
||||
|
- key: node-role.kubernetes.io/master |
||||
|
operator: Exists |
||||
|
effect: NoSchedule |
||||
|
# Rancher specific change: These tolerations are added to account for RKE1 and RKE2 taints |
||||
|
- key: node-role.kubernetes.io/controlplane |
||||
|
effect: NoSchedule |
||||
|
value: "true" |
||||
|
- key: node-role.kubernetes.io/control-plane |
||||
|
effect: NoSchedule |
||||
|
value: "true" |
||||
|
- key: node-role.kubernetes.io/etcd |
||||
|
effect: NoExecute |
||||
|
value: "true" |
||||
|
{{- end }} |
||||
|
dnsPolicy: "Default" |
||||
|
containers: |
||||
|
- name: csi-attacher |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.csiAttacher.repository }}:{{ .Values.csiController.image.csiAttacher.tag }}" |
||||
|
args: |
||||
|
- "--v=4" |
||||
|
- "--timeout=300s" |
||||
|
- "--csi-address=$(ADDRESS)" |
||||
|
- "--leader-election" |
||||
|
- "--kube-api-qps=100" |
||||
|
- "--kube-api-burst=100" |
||||
|
env: |
||||
|
- name: ADDRESS |
||||
|
value: /csi/csi.sock |
||||
|
volumeMounts: |
||||
|
- mountPath: /csi |
||||
|
name: socket-dir |
||||
|
{{- if .Values.csiController.csiResizer.enabled }} |
||||
|
- name: csi-resizer |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.csiResizer.repository }}:{{ .Values.csiController.image.csiResizer.tag }}" |
||||
|
args: |
||||
|
- "--v=4" |
||||
|
- "--timeout=300s" |
||||
|
- "--handle-volume-inuse-error=false" |
||||
|
- "--csi-address=$(ADDRESS)" |
||||
|
- "--kube-api-qps=100" |
||||
|
- "--kube-api-burst=100" |
||||
|
- "--leader-election" |
||||
|
env: |
||||
|
- name: ADDRESS |
||||
|
value: /csi/csi.sock |
||||
|
volumeMounts: |
||||
|
- mountPath: /csi |
||||
|
name: socket-dir |
||||
|
{{- end }} |
||||
|
- name: vsphere-csi-controller |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.repository }}:{{ .Values.csiController.image.tag }}" |
||||
|
args: |
||||
|
- "--fss-name=internal-feature-states.csi.vsphere.vmware.com" |
||||
|
- "--fss-namespace=$(CSI_NAMESPACE)" |
||||
|
imagePullPolicy: "Always" |
||||
|
env: |
||||
|
- name: CSI_ENDPOINT |
||||
|
value: unix:///csi/csi.sock |
||||
|
- name: X_CSI_MODE |
||||
|
value: "controller" |
||||
|
- name: X_CSI_SPEC_DISABLE_LEN_CHECK |
||||
|
value: "true" |
||||
|
- name: X_CSI_SERIAL_VOL_ACCESS_TIMEOUT |
||||
|
value: 3m |
||||
|
- name: VSPHERE_CSI_CONFIG |
||||
|
value: "/etc/cloud/csi-vsphere.conf" |
||||
|
- name: LOGGER_LEVEL |
||||
|
value: "PRODUCTION" # Options: DEVELOPMENT, PRODUCTION |
||||
|
- name: INCLUSTER_CLIENT_QPS |
||||
|
value: "100" |
||||
|
- name: INCLUSTER_CLIENT_BURST |
||||
|
value: "100" |
||||
|
- name: CSI_NAMESPACE |
||||
|
valueFrom: |
||||
|
fieldRef: |
||||
|
fieldPath: metadata.namespace |
||||
|
volumeMounts: |
||||
|
- mountPath: /etc/cloud |
||||
|
name: vsphere-config-volume |
||||
|
readOnly: true |
||||
|
- mountPath: /csi |
||||
|
name: socket-dir |
||||
|
ports: |
||||
|
- name: healthz |
||||
|
containerPort: 9808 |
||||
|
protocol: TCP |
||||
|
- name: prometheus |
||||
|
containerPort: 2112 |
||||
|
protocol: TCP |
||||
|
livenessProbe: |
||||
|
httpGet: |
||||
|
path: /healthz |
||||
|
port: healthz |
||||
|
initialDelaySeconds: 10 |
||||
|
timeoutSeconds: 3 |
||||
|
periodSeconds: 5 |
||||
|
failureThreshold: 3 |
||||
|
- name: liveness-probe |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.livenessProbe.repository }}:{{ .Values.csiController.image.livenessProbe.tag }}" |
||||
|
args: |
||||
|
- "--v=4" |
||||
|
- "--csi-address=/csi/csi.sock" |
||||
|
volumeMounts: |
||||
|
- name: socket-dir |
||||
|
mountPath: /csi |
||||
|
- name: vsphere-syncer |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.vsphereSyncer.repository }}:{{ .Values.csiController.image.vsphereSyncer.tag }}" |
||||
|
args: |
||||
|
- "--leader-election" |
||||
|
- "--fss-name=internal-feature-states.csi.vsphere.vmware.com" |
||||
|
- "--fss-namespace=$(CSI_NAMESPACE)" |
||||
|
imagePullPolicy: "Always" |
||||
|
ports: |
||||
|
- containerPort: 2113 |
||||
|
name: prometheus |
||||
|
protocol: TCP |
||||
|
env: |
||||
|
- name: FULL_SYNC_INTERVAL_MINUTES |
||||
|
value: "30" |
||||
|
- name: VSPHERE_CSI_CONFIG |
||||
|
value: "/etc/cloud/csi-vsphere.conf" |
||||
|
- name: LOGGER_LEVEL |
||||
|
value: "PRODUCTION" # Options: DEVELOPMENT, PRODUCTION |
||||
|
- name: INCLUSTER_CLIENT_QPS |
||||
|
value: "100" |
||||
|
- name: INCLUSTER_CLIENT_BURST |
||||
|
value: "100" |
||||
|
- name: CSI_NAMESPACE |
||||
|
valueFrom: |
||||
|
fieldRef: |
||||
|
fieldPath: metadata.namespace |
||||
|
volumeMounts: |
||||
|
- mountPath: /etc/cloud |
||||
|
name: vsphere-config-volume |
||||
|
readOnly: true |
||||
|
- name: csi-provisioner |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiController.image.csiProvisioner.repository }}:{{ .Values.csiController.image.csiProvisioner.tag }}" |
||||
|
args: |
||||
|
- "--v=4" |
||||
|
- "--timeout=300s" |
||||
|
- "--csi-address=$(ADDRESS)" |
||||
|
- "--kube-api-qps=100" |
||||
|
- "--kube-api-burst=100" |
||||
|
- "--leader-election" |
||||
|
- "--default-fstype=ext4" |
||||
|
# needed only for topology aware setup |
||||
|
#- "--feature-gates=Topology=true" |
||||
|
#- "--strict-topology" |
||||
|
env: |
||||
|
- name: ADDRESS |
||||
|
value: /csi/csi.sock |
||||
|
volumeMounts: |
||||
|
- mountPath: /csi |
||||
|
name: socket-dir |
||||
|
volumes: |
||||
|
- name: vsphere-config-volume |
||||
|
secret: |
||||
|
secretName: {{ .Values.vCenter.configSecret.name }} |
||||
|
- name: socket-dir |
||||
|
emptyDir: {} |
||||
@ -0,0 +1,13 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: ClusterRoleBinding |
||||
|
apiVersion: rbac.authorization.k8s.io/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-controller-binding |
||||
|
subjects: |
||||
|
- kind: ServiceAccount |
||||
|
name: vsphere-csi-controller |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
roleRef: |
||||
|
kind: ClusterRole |
||||
|
name: vsphere-csi-controller-role |
||||
|
apiGroup: rbac.authorization.k8s.io |
||||
@ -0,0 +1,57 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: ClusterRole |
||||
|
apiVersion: rbac.authorization.k8s.io/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-controller-role |
||||
|
rules: |
||||
|
- apiGroups: [""] |
||||
|
resources: ["nodes", "persistentvolumeclaims", "pods", "configmaps"] |
||||
|
verbs: ["get", "list", "watch"] |
||||
|
- apiGroups: [""] |
||||
|
resources: ["persistentvolumeclaims/status"] |
||||
|
verbs: ["patch"] |
||||
|
- apiGroups: [""] |
||||
|
resources: ["persistentvolumes"] |
||||
|
verbs: ["get", "list", "watch", "create", "update", "delete", "patch"] |
||||
|
- apiGroups: [""] |
||||
|
resources: ["events"] |
||||
|
verbs: ["get", "list", "watch", "create", "update", "patch"] |
||||
|
- apiGroups: ["coordination.k8s.io"] |
||||
|
resources: ["leases"] |
||||
|
verbs: ["get", "watch", "list", "delete", "update", "create"] |
||||
|
- apiGroups: ["storage.k8s.io"] |
||||
|
resources: ["storageclasses", "csinodes"] |
||||
|
verbs: ["get", "list", "watch"] |
||||
|
- apiGroups: ["storage.k8s.io"] |
||||
|
resources: ["volumeattachments"] |
||||
|
verbs: ["get", "list", "watch", "patch"] |
||||
|
- apiGroups: ["cns.vmware.com"] |
||||
|
resources: ["triggercsifullsyncs"] |
||||
|
verbs: ["create", "get", "update", "watch", "list"] |
||||
|
- apiGroups: ["cns.vmware.com"] |
||||
|
resources: ["cnsvspherevolumemigrations"] |
||||
|
verbs: ["create", "get", "list", "watch", "update", "delete"] |
||||
|
- apiGroups: ["apiextensions.k8s.io"] |
||||
|
resources: ["customresourcedefinitions"] |
||||
|
verbs: ["get", "create", "update"] |
||||
|
- apiGroups: ["storage.k8s.io"] |
||||
|
resources: ["volumeattachments/status"] |
||||
|
verbs: ["patch"] |
||||
|
- apiGroups: ["cns.vmware.com"] |
||||
|
resources: ["cnsvolumeoperationrequests"] |
||||
|
verbs: ["create", "get", "list", "update", "delete"] |
||||
|
- apiGroups: [ "snapshot.storage.k8s.io" ] |
||||
|
resources: [ "volumesnapshots" ] |
||||
|
verbs: [ "get", "list" ] |
||||
|
- apiGroups: [ "snapshot.storage.k8s.io" ] |
||||
|
resources: [ "volumesnapshotclasses" ] |
||||
|
verbs: [ "watch", "get", "list" ] |
||||
|
- apiGroups: [ "snapshot.storage.k8s.io" ] |
||||
|
resources: [ "volumesnapshotcontents" ] |
||||
|
verbs: [ "create", "get", "list", "watch", "update", "delete", "patch"] |
||||
|
- apiGroups: [ "snapshot.storage.k8s.io" ] |
||||
|
resources: [ "volumesnapshotcontents/status" ] |
||||
|
verbs: [ "update", "patch" ] |
||||
|
- apiGroups: [ "cns.vmware.com" ] |
||||
|
resources: [ "csinodetopologies" ] |
||||
|
verbs: ["get", "update", "watch", "list"] |
||||
@ -0,0 +1,6 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: ServiceAccount |
||||
|
apiVersion: v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-controller |
||||
|
namespace: {{ .Release.Namespace }} |
||||
@ -0,0 +1,20 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
apiVersion: v1 |
||||
|
kind: Service |
||||
|
metadata: |
||||
|
name: vsphere-csi-controller |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
labels: |
||||
|
app: vsphere-csi-controller |
||||
|
spec: |
||||
|
ports: |
||||
|
- name: ctlr |
||||
|
port: 2112 |
||||
|
targetPort: 2112 |
||||
|
protocol: TCP |
||||
|
- name: syncer |
||||
|
port: 2113 |
||||
|
targetPort: 2113 |
||||
|
protocol: TCP |
||||
|
selector: |
||||
|
app: vsphere-csi-controller |
||||
@ -0,0 +1,8 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
apiVersion: storage.k8s.io/v1 # For k8s 1.17 use storage.k8s.io/v1beta1 |
||||
|
kind: CSIDriver |
||||
|
metadata: |
||||
|
name: csi.vsphere.vmware.com |
||||
|
spec: |
||||
|
attachRequired: true |
||||
|
podInfoOnMount: false |
||||
@ -0,0 +1,156 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: DaemonSet |
||||
|
apiVersion: apps/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-node |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
spec: |
||||
|
selector: |
||||
|
matchLabels: |
||||
|
app: vsphere-csi-node |
||||
|
updateStrategy: |
||||
|
type: "RollingUpdate" |
||||
|
rollingUpdate: |
||||
|
maxUnavailable: 1 |
||||
|
template: |
||||
|
metadata: |
||||
|
labels: |
||||
|
app: vsphere-csi-node |
||||
|
role: vsphere-csi |
||||
|
spec: |
||||
|
serviceAccountName: vsphere-csi-node |
||||
|
hostNetwork: true |
||||
|
dnsPolicy: "ClusterFirstWithHostNet" |
||||
|
containers: |
||||
|
- name: node-driver-registrar |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiNode.image.nodeDriverRegistrar.repository }}:{{ .Values.csiNode.image.nodeDriverRegistrar.tag }}" |
||||
|
args: |
||||
|
- "--v=5" |
||||
|
- "--csi-address=$(ADDRESS)" |
||||
|
- "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)" |
||||
|
- "--health-port=9809" |
||||
|
env: |
||||
|
- name: ADDRESS |
||||
|
value: /csi/csi.sock |
||||
|
- name: DRIVER_REG_SOCK_PATH |
||||
|
value: {{ .Values.csiNode.prefixPath }}/var/lib/kubelet/plugins/csi.vsphere.vmware.com/csi.sock |
||||
|
volumeMounts: |
||||
|
- name: plugin-dir |
||||
|
mountPath: /csi |
||||
|
- name: registration-dir |
||||
|
mountPath: /registration |
||||
|
ports: |
||||
|
- containerPort: 9809 |
||||
|
name: healthz |
||||
|
livenessProbe: |
||||
|
httpGet: |
||||
|
path: /healthz |
||||
|
port: healthz |
||||
|
initialDelaySeconds: 5 |
||||
|
timeoutSeconds: 5 |
||||
|
- name: vsphere-csi-node |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiNode.image.repository }}:{{ .Values.csiNode.image.tag }}" |
||||
|
args: |
||||
|
- "--fss-name=internal-feature-states.csi.vsphere.vmware.com" |
||||
|
- "--fss-namespace=$(CSI_NAMESPACE)" |
||||
|
imagePullPolicy: "Always" |
||||
|
env: |
||||
|
- name: NODE_NAME |
||||
|
valueFrom: |
||||
|
fieldRef: |
||||
|
fieldPath: spec.nodeName |
||||
|
- name: CSI_ENDPOINT |
||||
|
value: unix:///csi/csi.sock |
||||
|
- name: MAX_VOLUMES_PER_NODE |
||||
|
value: "0" # Maximum number of volumes that controller can publish to the node. If value is not set or zero Kubernetes decide how many volumes can be published by the controller to the node. |
||||
|
- name: X_CSI_MODE |
||||
|
value: "node" |
||||
|
- name: X_CSI_SPEC_REQ_VALIDATION |
||||
|
value: "false" |
||||
|
- name: X_CSI_SPEC_DISABLE_LEN_CHECK |
||||
|
value: "true" |
||||
|
# needed only for topology aware setups |
||||
|
#- name: VSPHERE_CSI_CONFIG |
||||
|
# value: "/etc/cloud/csi-vsphere.conf" # here csi-vsphere.conf is the name of the file used for creating secret using "--from-file" flag |
||||
|
- name: LOGGER_LEVEL |
||||
|
value: "PRODUCTION" # Options: DEVELOPMENT, PRODUCTION |
||||
|
- name: CSI_NAMESPACE |
||||
|
valueFrom: |
||||
|
fieldRef: |
||||
|
fieldPath: metadata.namespace |
||||
|
securityContext: |
||||
|
privileged: true |
||||
|
capabilities: |
||||
|
add: ["SYS_ADMIN"] |
||||
|
allowPrivilegeEscalation: true |
||||
|
volumeMounts: |
||||
|
# needed only for topology aware setups |
||||
|
#- name: vsphere-config-volume |
||||
|
# mountPath: /etc/cloud |
||||
|
# readOnly: true |
||||
|
- name: plugin-dir |
||||
|
mountPath: /csi |
||||
|
- name: pods-mount-dir |
||||
|
mountPath: {{ .Values.csiNode.prefixPath }}/var/lib/kubelet |
||||
|
# needed so that any mounts setup inside this container are |
||||
|
# propagated back to the host machine. |
||||
|
mountPropagation: "Bidirectional" |
||||
|
- name: device-dir |
||||
|
mountPath: /dev |
||||
|
- name: blocks-dir |
||||
|
mountPath: /sys/block |
||||
|
- name: sys-devices-dir |
||||
|
mountPath: /sys/devices |
||||
|
ports: |
||||
|
- name: healthz |
||||
|
containerPort: 9808 |
||||
|
protocol: TCP |
||||
|
livenessProbe: |
||||
|
httpGet: |
||||
|
path: /healthz |
||||
|
port: healthz |
||||
|
initialDelaySeconds: 10 |
||||
|
timeoutSeconds: 5 |
||||
|
periodSeconds: 5 |
||||
|
failureThreshold: 3 |
||||
|
- name: liveness-probe |
||||
|
image: "{{ template "system_default_registry" . }}{{ .Values.csiNode.image.livenessProbe.repository }}:{{ .Values.csiNode.image.livenessProbe.tag }}" |
||||
|
args: |
||||
|
- "--v=4" |
||||
|
- "--csi-address=/csi/csi.sock" |
||||
|
volumeMounts: |
||||
|
- name: plugin-dir |
||||
|
mountPath: /csi |
||||
|
volumes: |
||||
|
# needed only for topology aware setups |
||||
|
#- name: vsphere-config-volume |
||||
|
# secret: |
||||
|
# secretName: vsphere-config-secret |
||||
|
- name: registration-dir |
||||
|
hostPath: |
||||
|
path: {{ .Values.csiNode.prefixPath }}/var/lib/kubelet/plugins_registry |
||||
|
type: Directory |
||||
|
- name: plugin-dir |
||||
|
hostPath: |
||||
|
path: {{ .Values.csiNode.prefixPath }}/var/lib/kubelet/plugins/csi.vsphere.vmware.com |
||||
|
type: DirectoryOrCreate |
||||
|
- name: pods-mount-dir |
||||
|
hostPath: |
||||
|
path: {{ .Values.csiNode.prefixPath }}/var/lib/kubelet |
||||
|
type: Directory |
||||
|
- name: device-dir |
||||
|
hostPath: |
||||
|
path: /dev |
||||
|
- name: blocks-dir |
||||
|
hostPath: |
||||
|
path: /sys/block |
||||
|
type: Directory |
||||
|
- name: sys-devices-dir |
||||
|
hostPath: |
||||
|
path: /sys/devices |
||||
|
type: Directory |
||||
|
tolerations: |
||||
|
- effect: NoExecute |
||||
|
operator: Exists |
||||
|
- effect: NoSchedule |
||||
|
operator: Exists |
||||
@ -0,0 +1,14 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: RoleBinding |
||||
|
apiVersion: rbac.authorization.k8s.io/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-node-binding |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
subjects: |
||||
|
- kind: ServiceAccount |
||||
|
name: vsphere-csi-node |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
roleRef: |
||||
|
kind: Role |
||||
|
name: vsphere-csi-node-role |
||||
|
apiGroup: rbac.authorization.k8s.io |
||||
@ -0,0 +1,10 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: Role |
||||
|
apiVersion: rbac.authorization.k8s.io/v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-node-role |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
rules: |
||||
|
- apiGroups: [""] |
||||
|
resources: ["configmaps"] |
||||
|
verbs: ["get", "list", "watch"] |
||||
@ -0,0 +1,6 @@ |
|||||
|
# Source: https://github.com/kubernetes-sigs/vsphere-csi-driver |
||||
|
kind: ServiceAccount |
||||
|
apiVersion: v1 |
||||
|
metadata: |
||||
|
name: vsphere-csi-node |
||||
|
namespace: {{ .Release.Namespace }} |
||||
@ -0,0 +1,9 @@ |
|||||
|
{{- if .Values.vCenter.configSecret.generate -}} |
||||
|
apiVersion: v1 |
||||
|
kind: Secret |
||||
|
metadata: |
||||
|
name: {{ .Values.vCenter.configSecret.name }} |
||||
|
namespace: {{ .Release.Namespace }} |
||||
|
data: |
||||
|
csi-vsphere.conf: {{ tpl .Values.vCenter.configSecret.configTemplate . | b64enc | quote }} |
||||
|
{{- end -}} |
||||
@ -0,0 +1,17 @@ |
|||||
|
{{- if .Values.storageClass.enabled -}} |
||||
|
apiVersion: storage.k8s.io/v1 |
||||
|
kind: StorageClass |
||||
|
metadata: |
||||
|
name: {{ .Values.storageClass.name | quote }} |
||||
|
annotations: |
||||
|
storageclass.kubernetes.io/is-default-class: {{ .Values.storageClass.isDefault | quote }} |
||||
|
provisioner: csi.vsphere.vmware.com |
||||
|
allowVolumeExpansion: {{ .Values.storageClass.allowVolumeExpansion }} |
||||
|
parameters: |
||||
|
{{- if .Values.storageClass.datastoreURL }} |
||||
|
datastoreURL: {{ .Values.storageClass.datastoreURL | quote }} |
||||
|
{{- end }} |
||||
|
{{- if .Values.storageClass.storagePolicyName }} |
||||
|
storagepolicyname: {{ .Values.storageClass.storagePolicyName | quote }} |
||||
|
{{- end }} |
||||
|
{{- end -}} |
||||
@ -0,0 +1,95 @@ |
|||||
|
vCenter: |
||||
|
host: "" |
||||
|
port: 443 |
||||
|
insecureFlag: "1" |
||||
|
clusterId: "" |
||||
|
datacenters: "" |
||||
|
username: "" |
||||
|
password: "" |
||||
|
configSecret: |
||||
|
name: "vsphere-config-secret" |
||||
|
generate: true |
||||
|
configTemplate: | |
||||
|
[Global] |
||||
|
cluster-id = {{ required ".Values.vCenter.clusterId must be provided" (default .Values.vCenter.clusterId .Values.global.cattle.clusterId) | quote }} |
||||
|
user = {{ .Values.vCenter.username | quote }} |
||||
|
password = {{ .Values.vCenter.password | quote }} |
||||
|
port = {{ .Values.vCenter.port | quote }} |
||||
|
insecure-flag = {{ .Values.vCenter.insecureFlag | quote }} |
||||
|
|
||||
|
[VirtualCenter {{ .Values.vCenter.host | quote }}] |
||||
|
datacenters = {{ .Values.vCenter.datacenters | quote }} |
||||
|
|
||||
|
csiController: |
||||
|
csiResizer: |
||||
|
enabled: false |
||||
|
image: |
||||
|
repository: rancher/mirrored-cloud-provider-vsphere-csi-release-driver |
||||
|
tag: v2.3.0 |
||||
|
csiAttacher: |
||||
|
repository: rancher/mirrored-sig-storage-csi-attacher |
||||
|
tag: v3.2.0 |
||||
|
csiResizer: |
||||
|
repository: rancher/mirrored-k8scsi-csi-resizer |
||||
|
tag: v1.1.0 |
||||
|
livenessProbe: |
||||
|
repository: rancher/mirrored-k8scsi-livenessprobe |
||||
|
tag: v2.2.0 |
||||
|
vsphereSyncer: |
||||
|
repository: rancher/mirrored-cloud-provider-vsphere-csi-release-syncer |
||||
|
tag: v2.3.0 |
||||
|
csiProvisioner: |
||||
|
repository: rancher/mirrored-sig-storage-csi-provisioner |
||||
|
tag: v2.2.0 |
||||
|
nodeSelector: {} |
||||
|
# Uncomment below toleration if you need an aggressive pod eviction in case when |
||||
|
# node becomes not-ready or unreachable. Default is 300 seconds if not specified. |
||||
|
# tolerations: |
||||
|
# - key: node.kubernetes.io/not-ready |
||||
|
# operator: Exists |
||||
|
# effect: NoExecute |
||||
|
# tolerationSeconds: 30 |
||||
|
# - key: node.kubernetes.io/unreachable |
||||
|
# operator: Exists |
||||
|
# effect: NoExecute |
||||
|
# tolerationSeconds: 30 |
||||
|
|
||||
|
# Internal features |
||||
|
csiMigration: |
||||
|
enabled: false |
||||
|
csiAuthCheck: |
||||
|
enabled: false |
||||
|
onlineVolumeExtend: |
||||
|
enabled: false |
||||
|
triggerCsiFullsync: |
||||
|
enabled: false |
||||
|
asyncQueryVolume: |
||||
|
enabled: false |
||||
|
improvedCsiIdempotency: |
||||
|
enabled: false |
||||
|
improvedVolumeTopology: |
||||
|
enabled: false |
||||
|
|
||||
|
csiNode: |
||||
|
prefixPath: "" |
||||
|
image: |
||||
|
repository: rancher/mirrored-cloud-provider-vsphere-csi-release-driver |
||||
|
tag: v2.3.0 |
||||
|
nodeDriverRegistrar: |
||||
|
repository: rancher/mirrored-k8scsi-csi-node-driver-registrar |
||||
|
tag: v2.1.0 |
||||
|
livenessProbe: |
||||
|
repository: rancher/mirrored-k8scsi-livenessprobe |
||||
|
tag: v2.2.0 |
||||
|
|
||||
|
storageClass: |
||||
|
enabled: true |
||||
|
allowVolumeExpansion: false |
||||
|
name: "vsphere-csi-sc" |
||||
|
isDefault: true |
||||
|
storagePolicyName: "" |
||||
|
datastoreURL: "" |
||||
|
|
||||
|
global: |
||||
|
cattle: |
||||
|
systemDefaultRegistry: "" |
||||
Loading…
Reference in new issue