Migrate from NFS Export to VAST NFS

Overview

The NFS connector scans the source NFS export over NFSv3, stores the resulting index under a shared index path, and copies files with rsync to the destination export with POSIX ACLs preserved.

The following capabilities are supported for NFS migrations:

• To determine when data synchronization occurs:
  • Periodic Resync
• To manage which items are being synchronized:
  • Exclude List
  • Mirror Deletions
• To create a catalog of items being synchronized:
  • Sync Catalog
• To configure SyncEngine processes:
  • Configure Connector Ports for Managed NFS Environments
  • NFS Rate Limits
  • Sync Catalog Rate Limits

Complete these steps to set up migration from a source NFS export to a destination NFS export:

1. Ensure that the prerequisites are met.

2. Create two NFS connectors: one for source NFS export and the other for destination NFS export.

3. Create an index connector to connect to the sync catalog database.

   Note: This step is only required if you are going to create a sync catalog for the migration.

4. Create a migration.

5. Start the migration.

   Note: This step is only required when the migration does not have periodic resynchronization enabled.

6. Monitor migration progress and completion.

7. If necessary, troubleshoot.

The following limitations apply:

• SyncEngine does not preserve NFSv4.1 ACLs.

Features

Periodic Resync

In addition to starting a migration once it is created, SyncEngine can be configured to automatically run a resynchronization at user-supplied intervals.

• In SyncEngine UI, set the SyncMode to Automatic and enter an interval (in hours) in the Sync Interval field.

• In SyncEngine CLI, include the --sync-mode=automatic and --sync-interval <interval> options when submitting the migration create command.

For migrations set to Automatic synchronization mode, the first synchronization starts immediately after the migration is created, and the subsequent synchronizations are performed periodically with the specified interval. A subsequent synchronization cannot be triggered manually.

To be able to manually start a synchronization on demand, set SyncMode to Manual.

Exclude List

You can supply a list of files or directories to be skipped during migration.

The list can contain file paths or wildcard patterns, one path or pattern per line. Supported wildcards are: * (zero or more characters), ? (one character), and [seq] (one character from the sequence or range). The wildcards can appear anywhere in the path.

For example:

/mnt/source/projects/logs/*.log
/mnt/source/projects/data/temp/*
/mnt/source/projects/reports/image[1-2000].jpg

Do not add trailing slashes to directory paths, e.g. specify /path/dir (but not /path/dir/).

Common patterns include:

Starting with SyncEngine 3.4.1, only absolute paths can be specified in the exclude list.

To supply an exclude list:

• In SyncEngine UI, upload the exclude list file in the Exclusion list field when creating a migration.
• In SyncEngine CLI:

  • Include the --exclusion-file <file path> option on the migration create command, or

  • Run this command after the migration is created:

      migration exclusion-file upload <file path> --migration-id=<migration ID>
    

The exclude list file must use the UTF-8 encoding.

The following limitation applies:

• An exclude list can contain up to 10,000 lines.

Mirror Deletions

You can determine how SyncEngine handles the items in the source store that have been deleted since the previous migration run.

By default, SyncEngine does not delete such items from the destination. If you want to have them deleted, configure the migration to mirror source deletions at the destination:

• In SyncEngine UI, select the Mirror source deletions on the destination option when creating a migration.

• In SyncEngine CLI, include the --enable-deletes=true option when submitting the migration create command.

Note: Deletion of items at the destination cannot be undone by disabling this setting.

Sync Catalog

Note: This feature is available starting with SyncEngine 3.4.1.

You can set up a dedicated Postgres database to keep track of items being migrated from an NFS export to VAST NFS.

The sync catalog is optional; it is created in addition to the migration scan index (which is maintained on a dedicated VAST cluster index view).

Caution: Do not use the SyncEngine control-plane database to store the sync catalog.

To create a sync catalog:

1. Create an index connector to connect to the database that will store the sync catalog.
2. Attach the index connector to the NFS migration:
   • In SyncEngine UI, select the connector in the Index connector field in migration settings and specify the database namespace in the Catalog namespace field. By default, namespace public is used.
   • In SyncEngine CLI, include the --index-connector-id and --index-catalog options on the migration create command.

Configure Connector Ports for Managed NFS Environments

When you configure the NFS connector, you can specify ports to be used by SyncEngine in a managed NFS environment where the RPC portmapper (TCP port 111) is not reachable. To do so:

• In SyncEngine UI, use the NFS Port and Mount Port options in connector settings.
• In SyncEngine CLI, include the --nfs-port and --nfs-mount-port options on the connector create command.

NFS Rate Limits

Note: This feature is available starting with SyncEngine 3.4.1.

You can set request rate limits to prevent SyncEngine from causing excessive load on the NFS store:

• Cap the overall number of requests (per second) that SyncEngine sends to the NFS store.
• Restrict throughput when transferring the files.

Each of the limits applies to the aggregate amount of requests or throughput by all workers used by the connector. Each limit is enforced independently of the other limit. On reaching a limit, workers wait and retry so that synchronization continues after the pause.

By default, no request rate limits are set.

To set rate limits:

• In SyncEngine UI, use the API calls per second and Bytes per second fields in NFS connector settings.
• In SyncEngine CLI, include the --api-calls-per-second and --bytes-per-second options on the connector create command.

Sync Catalog Rate Limits

Note: This feature is available starting with SyncEngine 3.4.1.

You can set request rate limits to prevent SyncEngine from causing excessive load on the Postgres database that stores the sync catalog:

• Cap the overall number of requests (per second) that SyncEngine sends to the database.

This limit applies to the aggregate amount of requests by all workers used by the connector. On reaching the limit, workers wait and retry so that synchronization continues after the pause.

By default, no request rate limit is set.

To set the request rate limit:

• In SyncEngine UI, use the API calls per second field in index connector settings.
• In SyncEngine CLI, include the --api-calls-per-second on the connector create command.

Before You Begin

Ensure that the prerequisites are met:

Source NFS Store

• IPs to reach the source NFS server are known.

VAST Cluster

Create View Policies

Create a view policy that will control access to the destination data view and the index view. You can use a single view policy for both, or create two different policies.

In the view policy:

• Select the NFS security flavor.
• Set NFS host-based rules to nosquash for the worker IPs.
• Set Name length limit to Native Protocol Limit.
• Set Allowed Characters to Native Protocol Limit.

Create Destination View

Create an Element Store view exposing the destination VAST NFS export. In the view settings:

• Select the NFSv3 protocol.
• Select the view policy you created.

Create Index View

Create an Element Store view exposing the VAST NFS export that will be used to maintain scan index during the migration.

When running a migration, SyncEngine creates a directory named migration_<id> under the index view to store scan index files.

In the view settings:

• Select the NFSv3 protocol.
• Select the view policy you created.

Note: The index cannot reside under a source or destination NFS export, on a local disk or in a host-local directory.

SyncEngine

• The SyncEngine control plane and at least one worker are up and running.

• At least one worker label of a deployed worker is known. (The worker label is set when you deploy the worker.)

• Hosts that run workers can reach the source and destination NFS servers.

• The source NFS export, destination NFS export and the index view must be mounted on all workers (labeled for a particular NFS migration) using the same absolute path names.

  Tip: Run findmnt -T <path> -t nfs,nfs4 to verify.

• The SyncEngine worker must have read access to the source NFS export and read/write access to the destination NFS export and the index view.

Mount Source, Destination and Index on Worker Hosts

Below are examples of the required mounts for NFS to NFS migrations.

1. Mount the source:

    sudo mkdir -p /mnt/nfs-source
    sudo mount -t nfs -o vers=3 <source NFS server>:/export/path /mnt/nfs-source
   

2. Mount the destination:

    sudo mkdir -p /mnt/nfs-dest
    sudo mount -t nfs -o vers=3 <VAST virtual IP>:/view/path /mnt/nfs-dest
   

3. Mount the index:

    sudo mkdir -p /mnt/syncengine-index
    sudo mount -t nfs -o vers=3 <VAST virtual IP>:/syncengine-index /mnt/syncengine-index
   

4. Verify that findmnt returns the mounts:

    findmnt -T /mnt/nfs-source -t nfs,nfs4
    findmnt -T /mnt/nfs-dest -t nfs,nfs4
    findmnt -T /mnt/syncengine-index -t nfs,nfs4
   

5. Ensure that the mounts are visible inside the worker.

Workers only see host paths that are bind-mounted into the worker container. When you deploy a worker, include every top-level directory that holds your NFS mounts (default is often /mnt). For example, if mounts are under /mnt and /data, deploy the worker with both paths in the worker volume list (e.g. VOLUMES=/mnt,/data).
After deploy, from inside a worker shell (or by checking the host mounts the container inherits), confirm the same paths exist and remain NFS mounts. If you mount a new export on the host after the worker started, restart the worker (or remount) so the container sees it.

Database for Sync Catalog

This is only required if you are going to create a sync catalog for the migration.

• A dedicated Postgres instance (recommended) or database. You'll need to supply the host, port, database name, and user credentials to connect to the catalog. The user must be assigned a role that allows creating and using database objects in the specified namespace.

  Note: Ensure that authentication by username and password is allowed.

• Network connectivity from the SyncEngine control plane and worker hosts to the Postgres endpoint.

Caution: Do not use the SyncEngine control-plane database to store the sync catalog.

Create NFS Connector

An NFS connector is used to connect to a source NFS export or a destination VAST NFS export.

SyncEngine UI

1. In the Connectors page, click New Connector.
   A wizard dialog appears to walk you through the connector creation steps.

2. In the Type & name step:

   • Select the NFS connector type.
   • In the Connector name field, enter a unique name for the connector.
     The name can contain up to 63 characters, including uppercase and lowercase English letters, digits, periods (.), hyphens (-) and underscores (_).

3. In the Endpoint step:

   • Under NFS Endpoint, enter the index path and NFS endpoint IPs:

     Field	Description
     Index Path	Enter the absolute mount path of the index view created in Before you begin. For NFS to VAST NFS migrations, the same index path must be specified in both NFS connectors (source and destination).
     NFS Endpoint IPs	Enter a comma-separated list of NFS server endpoint IPs.

   • Under NFS Ports, specify ports to be used by SyncEngine in a managed NFS environment where the RPC portmapper (TCP port 111) is not reachable.

     Field	Description
     NFS Port	Enter the NFS daemon port. Port 2049 is commonly used for this purpose.
     Mount Port	Enter the mount port. Port 20048 is commonly used for this purpose.

4. In the Tunables step, optionally configure migration scan settings.

5. In the Rsync step, optionally configure rsync flags for NFS file transfers.

6. In the Rate limits step, set optional request rate limits for the connector:

   Field	Description
   API calls per second	Maximum allowed number of NFS RPC requests per second. Leave blank for unlimited (default).
   Bytes per second	Maximum allowed throughput (in bytes per second). Leave blank for unlimited (default).

7. In the Review step, review the settings you've made.
   If you want to make changes, click Back. Otherwise, click Create Connector.
   The connector is created and displayed in the Connectors page.

SyncEngine CLI

The following command creates an NFS connector named src-nfs to migrate from an NFS export reachable through <IP> using index path /mnt/syncengine-index:

syncengine-cli connector create \
  --type=nfs \
  --name=src-nfs \
  --index-path=/mnt/syncengine-index \
  --endpoint-ips=<IP>

Optional parameters:

--nfs-port <port>
--nfs-mount-port <port>
--block-size <bytes>
--batch-size-gb <GB>
--batch-file-count <count>
--task-flush-threshold <count>
--local-queue-timeout <seconds>
--scan-task-max-duration <seconds>
--local-queue-max-dirs <count>
--local-queue-split-size <count>
--rsync-archive
--rsync-compress
--rsync-checksum
--rsync-hard-links
--rsync-acls     # Required to have ACLs preserved
--rsync-xattrs
--rsync-no-perms
--rsync-no-owner
--rsync-no-group
--rsync-bwlimit <KB/s>
--rsync-timeout <seconds>
--api-calls-per-second <rate>
--bytes-per-second <bytes>

The output should be similar to the following:

Successfully created connector 'src-nfs' with ID 15

Tip: Note the connector ID in the output.

Create Index Connector

Note: This feature is available starting with SyncEngine 3.4.1.

An index connector is used to connect to the database that stores the migration's sync catalog.

SyncEngine UI

1. In the Connectors page, click New Connector.
   A wizard dialog appears to walk you through the connector creation steps.

2. In the Type & name step:

   • Select the Postgres connector type.
   • In the Connector name field, enter a unique name for the connector.
     The name can contain up to 63 characters, including uppercase and lowercase English letters, digits, periods (.), hyphens (-) and underscores (_).

3. In the Configure step, complete the fields:

   Field	Description
   Host	A hostname or IP to connect to the sync catalog database.
   Port	Port to connect to the sync catalog database (1-65535). By default, port 5432 is used.
   User	Database username.
   Password	Database password. On edit, leave blank to keep the existing password.
   Database	The name of the database used to store the sync catalog.

4. In the Rate limits step, set optional request rate limits for the connector:

   Field	Description
   API calls per second	Maximum allowed number of API calls per second. Leave blank for unlimited (default).

5. In the Review step, review the settings you've made.
   If you want to make changes, click Back. Otherwise, click Create Connector.
   The connector is created and displayed in the Connectors page.

SyncEngine CLI

The following command creates an index connector named nfs-catalog that connects to a Postgres database <catalog database> at <postgres host> using the credentials specified:

syncengine-cli connector create \
  --type=postgres \
  --name=nfs-catalog \
  --host=<Postgres host> \
  --username=<catalog user> \
  --password=<catalog password> \
  --database=<catalog database>

Optional parameters:

  --port=<port>
  --api-calls-per-second <number of calls>

The output should be similar to the following:

Successfully created connector 'nfs-catalog' with ID 15

Tip: Note the connector ID in the output.

Create Migration

Create a migration to copy files from a source mount path to a destination mount path on SyncEngine workers.

SyncEngine UI

1. In the Migrations page, click New Migration.
   A wizard dialog appears to walk you through the migration creation steps.

2. In the Source step, select the NFS connector that will be used to connect to the source NFS export.

3. In the Destination step, select the NFS connector that will be used to connect to the destination NFS export on the VAST cluster.

4. In the Paths step:

   • Under Migration Label, specify a SyncEngine worker label for the migration.
     The label is used to route migration jobs to the appropriate SyncEngine worker. One worker can be associated with one or more migration labels. Workers deployed with a certain label handle jobs for all migrations that have that label.
     You can create a new label or use an existing one.
     • To create a new label, select Create new label and enter a text string in the New Label field.
       The label can contain up to 128 characters, including uppercase and lowercase English letters, digits, periods (.), hyphens (-) and underscores (_).
     • To use an existing label, select Use existing label and enter the label string in the Existing Label field.
   • Under Paths, specify the paths to the migration source and destination.
     • Under Source Path, enter the absolute mount path of the NFS export to migrate data from. The export must be mounted at the same path on all workers with read permissions.
       You can specify a subdirectory under the mount root, for example: /mnt/nfs-source/projectA.
     • In the Destination Path field, enter the absolute mount path of the destination NFS export. The export must be mounted at the same path on all workers with write permissions.
   • Under Sync catalog, optionally attach a database that will store the migration's sync catalog:
     • In the Index connector field, select the index connector used to connect to the sync catalog database. If None (file index_path only) is left, sync catalog is not created.
     • In the Catalog namespace field, specify the database namespace for the sync catalog. By default, namespace public is used.

5. In the Sync mode step:

   • Under Sync Mode, specify if you want to run the migration once or periodically:
     • To run the migration once, select Manual in the SyncMode field.
       Migration will be started after you submit the migration job manually.
     • To run the migration automatically at certain intervals, select Automatic in the Sync Mode field and enter an interval (in hours) in the Sync Interval field. The default interval is 24 hours.
   • Under Include & exclude lists: optionally add an exclude list for the migration:
     • Use the Exclusion list field to upload a file listing specific file paths to be skipped during the migration. To learn more about this feature, see Exclude list.

6. In the What you're copying step:

   • Under What you're copying:
     • Review the settings you've made. If necessary, click Back and change as needed.
   • Under Delete propagation:
     • Decide how SyncEngine handles the files that have been deleted since the previous migration run.
       By default, SyncEngine does not delete such files from the destination NFS export. If you want to have them deleted, select Mirror source deletions on the destination.

7. In the Per-task timeouts and retries step, optionally configure suitable timeouts for the migration jobs.

8. In the Review step, review the settings you've made.
   If you want to make changes, click Back. Otherwise, click Create Migration.
   The migration is created and displayed in the Migration page.

SyncEngine CLI

The following command creates a migration from /mnt/nfs-source to /mnt/nfs-dest. The migration is to be accomplished by workers labeled with MYFIRSTMIGRATION using connectors with connector IDs 12 and 14.

syncengine-cli migration create \
  --source-connector-id=12 \
  --destination-connector-id=14 \
  --source-path=/mnt/nfs-source \
  --destination-path=/mnt/nfs-dest \
  --label=MYFIRSTMIGRATION

Optional parameters:

--index-connector-id=<index connector ID>
--index-catalog=<database namespace for sync catalog>
--sync-mode=manual
--sync-mode=automatic --sync-interval <hours>
--enable-deletes={true|false}
--exclusion-file <path to file>

The CLI creates a migration and displays its properties.

Tip: Note the migration ID in the output.

Start Migration

Manual migration start is only relevant for migrations that do not have periodic resynchronization configured. For migrations with periodic resynchronization, the migration starts automatically upon creation.

SyncEngine UI

Navigate to the Migrations page and click the start button for the migration you want.

SyncEngine CLI

Run the command:

 syncengine-cli migration start --migration-id=<ID>

The output should be similar to the following:

Starting sync job for migration: MYFIRSTMIGRATION
   ✓ Sync job created successfully!
   Job ID: 2
   Status: pending

Monitor Migration

As the migration progresses, the migration/job statuses change as follows: Not Started (or init) -> Waiting for Worker (or pending) -> Syncing -> Synced.

When complete, the destination mount path will have files and directories copied from the source mount path. To verify, compare file counts or spot-check files on the destination NFS path from a worker host.

SyncEngine UI

Check the migration status in the Migrations page.

SyncEngine CLI

Run the command:

syncengine-cli migration status --migration-id=<migration ID>

The output should be similar to the following:

=== Migration Status: MYFIRSTMIGRATION ===
ID: 125
Status: synced
Source: /mnt/nfs-source
Destination: /mnt/nfs-dest
Sync Mode: manual
Last Sync: 2026-07-15 14:06:08

Settings:
  Error Retry: 0
  Task Retry: 0

Sync Jobs: 1

Recent Sync Jobs:
+----+-----------+------------------+-------------+-------+------+
| ID |  STATUS   |    START TIME    |  DURATION   | FILES | SIZE |
+----+-----------+------------------+-------------+-------+------+
| 96 | completed | 2026-07-15 14:06 | PT4.623401S |     1 | 29 B |
+----+-----------+------------------+-------------+-------+------+

Pause, Resume, Abort Migration

To pause a running migration:

• In SyncEngine UI, use the pause control in the Migrations page.
• In SyncEngine CLI, run the migration pause command with one of the following options:
  • --migration-id <migration ID> to pause a migration.
  • --sync-job-id <sync job ID> to pause a synchronization job.

To resume a paused migration:

• In SyncEngine UI, use the resume control in the Migrations page.
• In SyncEngine CLI, run the migration resume command with one of the following options:
  • --migration-id <migration ID> to resume a migration.
  • --sync-job-id <sync job ID> to resume a synchronization job.

To abort a migration:

• In SyncEngine UI, use the abort control in the Migrations page.
• In SyncEngine CLI, run the migration abort --migration-id <migration ID> command.

Troubleshoot