---
title: "Install the DE Prerequisites on the Kubernetes cluster (5.4.x and 5.3.x)"
slug: "install-the-de-prerequisites-on-the-kubernetes-cluster-5-4-x-and-5-3-x"
description: "Install the VAST Zarf DataEngine package with our step-by-step guide, ensuring Kubernetes connectivity and proper storage class setup for optimal performance."
tags: ["DataEngine"]
updated: 2026-10-28T02:14:35Z
published: 2026-10-28T02:14:35Z
canonical: "docs.vastdata.com/install-the-de-prerequisites-on-the-kubernetes-cluster-5-4-x-and-5-3-x"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Install the DE Prerequisites on the Kubernetes cluster (5.4.x and 5.3.x)

> [!NOTE]
> ℹ️ Info
> 
> **Outcome:** Required namespaces and the VAST DataEngine operator are running on the Kubernetes cluster.
> 
> **Before you begin:** Install RKE2, configure kubectl, and obtain the VAST Zarf DataEngine package for your supported VAST AI OS release.
> 
> **Previous:** [Install the VAST CSI Driver](/develop/docs/installing-vast-csi-driver) | **Next:** [Connect DataEngine to the Kubernetes Cluster and Registry](/develop/docs/connecting-vast-dataengine-to-kubernetes-cluster-and-registry)

## Install the VAST Zarf DataEngine package

Ensure you have a valid *Kubeconfig* file for connecting to the Kubernetes cluster.

```bash
echo $KUBECONFIG
```

Example Output

```plaintext
/home/vastdata/.kubeconfig/rke2.yaml
```

### Verify the inotify limits

To check current limits:

```bash
sysctl fs.inotify.max_user_instances; sysctl fs.inotify.max_user_watches
```

Example Output

```plaintext
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 61656
```

To increase the limits temporarily:

```plaintext
sudo sysctl fs.inotify.max_user_instances=8192; sudo sysctl fs.inotify.max_user_watches=524288
```

Expected Output

```plaintext
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 524288
```

To make the change permanent:

```bash
echo "fs.inotify.max_user_instances=8192" | sudo tee -a /etc/sysctl.conf; echo "fs.inotify.max_user_watches=524288" | sudo tee -a /etc/sysctl.conf; sudo sysctl -p
```

Expected Output

```plaintext
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=524288
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 524288
```

### Obtain the DataEngine package

Find your VAST AI OS release version

![image-20260810-155530.png](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image-20260810-155530.png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A18%3A51Z&se=2026-11-08T10%3A18%3A51Z&sr=c&sp=r&sig=ab7rb4te9Sgo1d%2BFouULbxWrmVRD74xsczkaTjPmcYY%3D)

**If you are a VAST SE:**

Open a DataEngine OCTO request using the internal form.

**If you are a VAST Partner or Customer:**

Provide your VAST Account Team with the version string from above and they will generate a URL for you.

The URL generated in this example is:

```plaintext
https://vastdata-releases.s3.amazonaws.com/release_bundles/service-packs/release-5.4.3-sp5/dataengine/dataengine_offline-2459535.vast.tar.gz?AWSAccessKeyId=AKIARTN2ZMSY5WYRAXHB&Signature=vHag%2By8%2B91kewGLKk3D%2BG0yDqF4%3D&Expires=1786982326
```

### Download the package to the manager node

```bash
wget "https://vastdata-releases.s3.amazonaws.com/release_bundles/service-packs/release-5.4.3-sp5/dataengine/dataengine_offline-2459535.vast.tar.gz?AWSAccessKeyId=AKIARTN2ZMSY5WYRAXHB&Signature=vHag%2By8%2B91kewGLKk3D%2BG0yDqF4%3D&Expires=1786982326" -O dataengine_offline-2459535.vast.tar.gz
```

### Extract and initialize the Zarf package

Extract the VAST Zarf DataEngine package

```bash
mkdir -p zarf && tar -xzvf dataengine_offline-2459535.vast.tar.gz -C zarf
```

use ls -l to verify files have been extracted

```bash
cd zarf; ls -l
```

Example Output

```bash
total 2498024
-rw------- 1 vastdata vastdata  706464698 May 20 14:29 vast-builder.tar.gz
drwxrwxr-x 3 vastdata vastdata       4096 Aug 10 20:02 vast-cli
-rwxrwxr-x 1 vastdata vastdata  171475128 Aug 14  2025 zarf
-rw-rw-r-- 1 vastdata vastdata   81193896 Oct  9  2025 zarf-init-amd64-v0.60.0.tar.zst
-rw-rw-r-- 1 vastdata vastdata   73371902 Sep 17  2025 zarf-init-upstream-amd64-v0.60.0.tar.zst
-rw-rw-r-- 1 vastdata vastdata 1525440916 May 20 14:28 zarf-package-dataengine-amd64-1.0.0.tar.zst
```

> [!NOTE]
> ℹ️ Info
> 
> If you are running a version of VAST Cluster prior to 5.4.1-sp2, run the below command before you proceed:

```bash
if [ -f vast-zarf-mutator-amd64-v0.60.0.tar.zst ]; then
  mv vast-zarf-mutator-amd64-v0.60.0.tar.zst zarf-init-amd64-v0.60.0.tar.zst
fi
```

### Initialize Zarf with the VAST CSI driver

Use kubectl to see storage classes

```bash
kubectl get  storageclass
```

Example Output

```plaintext
NAME                 PROVISIONER        RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
vast-nfs (default)   csi.vastdata.com   Delete          Immediate           true                   7d4h
```

run `zarf init` with the following options :

```bash
./zarf init --architecture amd64 zarf-init-amd64-v0.60.0.tar.zst --set REGISTRY_HPA_AUTO_SIZE=true --confirm --log-level debug --storage-class=vast-nfs
```

This can take a few minutes depending on available resources.

Example Output

```plaintext
<snip>
2026-08-11 14:49:23 INF init complete. To get credentials for Zarf deployed services run `zarf tools get-creds`
```

### Initialize Zarf with local storage

If you do not have the VAST CSI Driver:

Installing local-path

Apply the manifest from Rancher's repository and annotate the `local-path` resource to mark it as the default `StorageClass`.

**Deploy Local Path Provisioner**

```plaintext
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
```

**Set** `local-path` **as the Default StorageClass**

Annotate the `local-path` StorageClass so Kubernetes automatically assigns it to new PersistentVolumeClaims (PVCs):

```plaintext
kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
```

run `zarf init` with the following options :

```bash
./zarf init --architecture amd64 zarf-init-amd64-v0.60.0.tar.zst --set REGISTRY_HPA_AUTO_SIZE=true --confirm --log-level debug --storage-class=local-path
```

This can take a few minutes depending on available resources.

Example Output

```plaintext
<snip>
2026-08-11 14:49:23 INF init complete. To get credentials for Zarf deployed services run `zarf tools get-creds`
```

### Create and label the Kubernetes namespaces

Add the following labels to the Kubernetes namespace:

```plaintext
create_namespace_for_zarf(){
local namespace=$1
kubectl create ns $namespace || true
kubectl label namespace $namespace zarf.dev/vast=mutate
}
create_namespace_for_zarf vast-dataengine
create_namespace_for_zarf knative-eventing
create_namespace_for_zarf knative-serving
create_namespace_for_zarf vast-pipelines
```

Expected Output

```plaintext
namespace/vast-dataengine created
namespace/vast-dataengine labeled
namespace/knative-eventing created
namespace/knative-eventing labeled
namespace/knative-serving created
namespace/knative-serving labeled
```

Verify that you have all of the namespaces

```plaintext
kubectl get ns
```

Example Output

```plaintext
NAME               STATUS   AGE
default            Active   62m
knative-eventing   Active   58s
knative-serving    Active   57s
kube-node-lease    Active   62m
kube-public        Active   62m
kube-system        Active   62m
vast-csi           Active   32m
vast-dataengine    Active   58s
zarf               Active   6m53s
```

### Install the VAST DataEngine operator

Run `zarf package deploy` with the following options:

```plaintext
./zarf package deploy --architecture amd64 zarf-package-dataengine*.tar.zst --confirm --log-level debug
```

Example Output

```plaintext
2026-08-11 15:03:39 INF running health checks chart=vast-serverless
2026-08-11 15:03:40 DBG done processing Helm chart name=vast-serverless duration=1m7.483781224s
2026-08-11 15:03:40 DBG done deploying component name=vast-serverless duration=2m34.996325919s
2026-08-11 15:03:40 DBG deployment complete duration=5m28.57834808s
```

Validate Pods are created

```bash
kubectl get pods -n knative-eventing
kubectl get pods -n vast-dataengine
```

Example Output

```plaintext
$ kubectl get pods -n knative-eventing
NAME                                                           READY   STATUS      RESTARTS   AGE
eventing-controller-97d8fd766-t8n48                            1/1     Running     0          86s
eventing-webhook-6c8f9fb4f9-gjzc8                              1/1     Running     0          85s
imc-controller-7cdc9cc88-mgr2k                                 1/1     Running     0          80s
imc-dispatcher-574bfd84c7-bl595                                1/1     Running     0          80s
job-sink-5574cb78d8-vjbdg                                      1/1     Running     0          86s
kafka-broker-dispatcher-0                                      1/1     Running     0          108s
kafka-broker-receiver-75547cc84d-jwkr8                         1/1     Running     0          108s
kafka-controller-8486fcc78d-dnb9s                              1/1     Running     0          74s
kafka-controller-post-install-eventing-1.17.1-lpm62            0/1     Completed   0          72s
kafka-source-dispatcher-0                                      1/1     Running     0          70s
kafka-webhook-eventing-6bc57cf945-9dw6g                        1/1     Running     0          73s
knative-kafka-storage-version-migrator-eventing-1.17.1-2dbc4   0/1     Completed   0          72s
mt-broker-controller-7d7c874d4d-wgwng                          1/1     Running     0          78s
mt-broker-filter-7795c7468c-jw94q                              1/1     Running     0          79s
mt-broker-ingress-9b46b84f6-kjd9n                              1/1     Running     0          78s
storage-version-migration-eventing-eventing-1.17.1-t4mwg       0/1     Completed   0          77s

$ kubectl get pods -n vast-dataengine
NAME                                                READY   STATUS    RESTARTS       AGE
keda-admission-webhooks-95c8dc6f4-jcj9q             1/1     Running   0              5m14s
keda-operator-846d96d678-nd8mj                      1/1     Running   1 (5m8s ago)   5m14s
keda-operator-metrics-apiserver-575c49b475-g5l2j    1/1     Running   0              5m14s
knative-operator-55f4bd4679-s26pf                   1/1     Running   0              6m26s
operator-webhook-6cdd696bcf-c4t4r                   1/1     Running   0              6m26s
vast-operator-controller-manager-86f5795d56-4l2wl   1/1     Running   0              2m10s
vast-telemetries-collector-56db6d769b-9jd2d         1/1     Running   0              115s
vast-telemetries-collector-56db6d769b-dxpff         1/1     Running   0              115s
vast-telemetries-collector-56db6d769b-km9rh         1/1     Running   0              115s
vast-telemetries-collector-56db6d769b-lp4zs         1/1     Running   0              2m10s
vast-telemetries-collector-56db6d769b-xffv2         1/1     Running   0              115s$
$
```