--- title: "7. VAST Client NFS Multipathing" slug: "7-vast-client-nfs-multipathing" description: "Boost your file system performance with VAST's NFS multipathing for high-speed, scalable AI training and HPC solutions." updated: 2026-10-28T19:54:28Z published: 2026-10-28T19:54:28Z canonical: "docs.vastdata.com/7-vast-client-nfs-multipathing" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # 7. VAST Client NFS Multipathing VAST provides a high-performance NFS interface enhanced with multipathing, enabling clients to communicate with multiple VAST server nodes (CNodes) in parallel. This is achieved through the VAST NFS client, which builds on the standard Linux kernel NFS driver and provides multipathing support for both TCP and RDMA transport modes. Multipathing enables: - Load-balanced traffic across multiple VIPs. - Parallel data paths for improved throughput. - Support for multiple client-side NICs to utilize all available bandwidth. **Note**: The VAST NFS client is optional and best used in high-performance scenarios with large, sequential I/O—like AI training or HPC—where it enables multi-path, RDMA, or GPU Direct for higher throughput. For general-purpose workloads or many small files, it offers only a small benefit. ## Mounting with Multipathing The VAST Enhanced NFS client introduces several mount options to leverage multipathing fully: **NFSv3**: ```plaintext sudo mount -o vers=3,nconnect=34,pconnect=4,mdconnect=2,spread_reads,spread_writes,noextend,lookupcache=pos,remoteports=dns localports=192.168.0.100~192.168.0.200 vast.example.com:/export /mnt/vast ``` **NFSv4**: ```plaintext sudo mount -o vers=4,nconnect=32,pconnect=4,remoteports=dns localports=192.168.0.100~192.168.0.200 vast.example.com:/export /mnt/vast ``` | Option | Description | | --- | --- | | nconnect= | Sets the total number of TCP connections per mount. These connections are distributed across VIPs. **Note**: 1. Use 8 connections for every 100Gbps with TCP (e.g., nconnect=32 for 400Gbps). 2. Use 2 connections for every 100Gbps with RDMA (and a minimum of 8 to reach good load balancing). | | pconnect= | Controls the number of connections made by each client. The nconnect parameter still controls the total number of connections. For example, passing nconnect=32 and pconnect=4 will create 8 clients with 4 connections each, all to the same destination address, and regardless of how big is remoteports. (**Note**: pconnect applies to vastnfs 4.5 or later) When using mdconnect=2, there are two connections out of nconnect settings that are dedicated to metadata, hence the values of 34 (32 + 2) | | remoteports | Enables connections to multiple VIPs (server-side IPs). **Note**: It is recommended to use remoteport=dns, so the client can issue DNS queries with the mount target DNS name and retrieve an array of addresses. | | localports | Specifies multiple client-side IPs for outgoing traffic (NFSv3 only). | | spread_reads | Distributes read operations across all connections to increase throughput. Only applies to NFSv3 | | spread_writes | Distributes write operations across all connections. Only applies to NFSv3 | | lookupcache=pos | Reduces the delay in discovering newly created files on other clients by enabling positive lookup caching, which is useful when one client waits for another to create a file. **Note**: 1. Recommended for distributed applications where one client creating a file should be immediately visible to another. 2. Not recommended for home directories or source code repositories. | | mdconnect=2 | Enables dedicated metadata channels, preventing metadata operations from being blocked behind large read/write I/O. Note: Requires vastnfs 4.0.34 or later. `mdconnect `**Only applies to NFSv3** | | noextend | Turns off the default extend-to-page optimization in writes. Useful for specific NFSv3 workflows with multiple clients. No tradeoffs; safe default. | | nocto (optional): | Disables Close-to-Open (CTO) consistency checks to optimize read/write latency for NFSv3. **Note**: Recommended primarily for single-host workloads (e.g., dedicated home directories or temporary local scratch space). Disabling CTO cache validation means the client does not guarantee that data written by one host is immediately visible to another, which can result in stale reads or inconsistent file states across nodes. | --- ## NFSv3: `nconnect `& `mdconnect `Partition When nconnect=34 & mdconnect=2, the 34 total connections are split as follows: - **2 dedicated connections** (from the 34-connection) are used exclusively for metadata operations. - **The remaining 32 connections** are dedicated to regular Data I/O. ![NFSv3 connections](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image(216).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A06%3A16Z&se=2026-11-08T10%3A06%3A16Z&sr=c&sp=r&sig=sq5bXxfu6IW5XQxbHneOowDFBugqeOzD6EcQ1exUMT4%3D) #### **NFSv3 Data I/O Multipathing:** - Data connections are distributed based on pconnect. For example, with pconnect=4 and nconnect - mdconnect = 32, traffic is spread across **8 VIPs** (32 Data Connections split across 8 target VIPs, providing up to 4 parallel connections per VIP). ![NFSv3 VIP connections](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image(217).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A06%3A16Z&se=2026-11-08T10%3A06%3A16Z&sr=c&sp=r&sig=sq5bXxfu6IW5XQxbHneOowDFBugqeOzD6EcQ1exUMT4%3D) #### NFSv3 Data Striping (`spread_reads` / `spread_writes`): - Stripes I/O for individual files across all 8 VIPs/CNodes concurrently, maximizing single-file bandwidth and bypassing the 1-file = 1-VIP limitation. ![NFSv3 VIPs to CNodes](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image(218).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A06%3A16Z&se=2026-11-08T10%3A06%3A16Z&sr=c&sp=r&sig=sq5bXxfu6IW5XQxbHneOowDFBugqeOzD6EcQ1exUMT4%3D) ## NFSv4: Understanding `nconnect `& `pconnect `Allocation - No `spread_reads` / `spread_writes` support in NFSv4. - No `mdconnect `support for NFSv4. #### NFSv4 Data I/O Multipathing: - With `nconnect=8` & `pconnect=2`, the 8 total connections are split across 4 unique Target VIPs (VIP 1, VIP 2, VIP 3, VIP 4) with 2 parallel connections per VIP. ![NFSv4 connections](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image(219).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A06%3A16Z&se=2026-11-08T10%3A06%3A16Z&sr=c&sp=r&sig=sq5bXxfu6IW5XQxbHneOowDFBugqeOzD6EcQ1exUMT4%3D) #### NFSv4 File-to-VIP Mapping (1-File = 1-VIP Limitation): - Each file is pinned to a single VIP, but can leverage both parallel connections to its assigned VIP (pconnect=2) if multiple processes or open instances request I/O to that same file. ![NFSv4 file to VIP mapping](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/image(220).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A06%3A16Z&se=2026-11-08T10%3A06%3A16Z&sr=c&sp=r&sig=sq5bXxfu6IW5XQxbHneOowDFBugqeOzD6EcQ1exUMT4%3D) ## VAST NFS `readahead `considerations VAST NFS Client 4.5.7 and later introduces an enhanced readahead implementation with a per-thread sequential access detector. The client can identify multiple threads performing sequential reads from different offsets within the same file and independently optimize readahead for each thread. This closely matches how SGLang loads SafeTensors model weights in parallel using multiple worker threads. As a result, VAST NFS significantly improves page cache prefetch efficiency during model initialization, reducing model load times by approximately 5–10× for SGLang workloads. For more information on readahead configuration: [VAST Quick NFS Read Ahead Tuning](/kb/docs/vast-quick-nfs-read-ahead-tuning) ## Additional Resources - [**Full Driver Documentation**](https://vastnfs.vastdata.com/docs/4.0/ChangeLog.html) - [**Multipathing Quick Start Guide**](/kb/docs/quickstart-multipathing-nfs-client-download-and-install) - [**VAST Quick NFS Read Ahead Tuning**](/kb/docs/vast-quick-nfs-read-ahead-tuning) - [**VAST NFS parameters**](https://vastnfs.vastdata.com/docs/4.0/usage/mount-params.html)