---
title: "How To Benchmark VAST Data with FIO"
slug: "how-to-benchmark-vast-data-with-fio"
description: "Benchmark VAST data performance efficiently using FIO for optimal storage solutions."
updated: 2026-10-27T19:37:32Z
published: 2026-10-28T18:28:29Z
canonical: "docs.vastdata.com/how-to-benchmark-vast-data-with-fio"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# How To Benchmark VAST Data with FIO

> [!NOTE]
> VAST provides [vperfcheck](/kb/docs/vperfcheck-automated-performance-testing) as an automated and easy-to-use performance testing tool for basic file and S3 object throughput, IOPS and latency benchmarks. Please consider using this tool before running custom test cases.

## Configure the load-generating clients

To achieve sufficient concurrency, run fio on many clients, all using the same VAST mount.

### Mount the VAST cluster

To evenly spread the load across all CNodes in a VAST Data cluster and ensure that all NICs on all CNodes are used, use the [VAST NFS driver](https://vastnfs.vastdata.com/docs/4.0/Intro.html) on each client and mount the cluster with multipath.

**Mount Option - *spread_reads, spread_writes***

These mount option parameters are not mandatory; the client can reach line rate with/without them. By default, all I/O to a specific file goes through a single connection. With TCP mounts, this may bottleneck applications reading/writing a single large file. By specifying spread_reads/spread_writes, the respective operation can use all connections, increasing throughput.

The vast majority of workloads benefit from spread_reads and spread_writes. The only known exceptions are single-thread applications reading files with O_DIRECT (not a default option). These applications have a single in-flight IO at a time, and having those go to the same connection/CNode results in lower latency (because of server-side prefetch).

For example:

```bash
sudo mkdir /mnt/vast
sudo mount -t nfs -o vers=3,nconnect=32,remoteports=172.200.201.1-172.200.201.16 172.200.201.1:/ /mnt/vast
mkdir -p /mnt/vast/fio
```

If you are unable to use the VAST NFS driver, mount the VAST Data cluster on each client to a unique CNode by a VIP on that CNode. You will need at least as many clients as you have CNodes. If your CNodes are dual-NIC, you will need 2x the clients.

```bash
sudo mkdir /mnt/vast
sudo mount -t nfs -o vers=3,nconnect=16 172.200.201.1:/ /mnt/vast
mkdir -p /mnt/vast/fio
```

### Start fio in server mode

Now start `fio` in server mode on each client and daemonize it:

```bash
fio --server --daemonize=/tmp/fio.pid
```

## Configure the job runner

On a host that could be a load-generating client, create the hosts.txt file and the job files.

### Generate hosts.txt file

The job runner needs to know which hosts will be driving the load. Create a `hosts.txt` file with each host’s hostname or IP address on a separate line. For example:

```plaintext
10.143.11.111
10.143.11.113
10.143.11.115
10.143.11.117
```

### Generate job files

In [client/server mode](https://fio.readthedocs.io/en/latest/fio_doc.html#client-server), `fio` can only use job files, not command line arguments. To facilitate running an array of tests, we can dynamically generate job files with a shell script:

```bash
#!/bin/bash

for bs in 1m 512k 128k 64k; do
    for pattern in write read randread randwrite; do
        echo "
[global]
group_reporting
create_on_open=1
refill_buffers=1
randrepeat=0
ioengine=libaio
iodepth=16
direct=1
numjobs=16
nrfiles=1
directory=/mnt/vast/fio

[10g_files]
readwrite=$pattern
bs=$bs
filesize=10g
" > ${bs}_${pattern}.fio
    done
done
```

The `[globals]` section includes the best practices for benchmarking VAST, such as doing direct writes, an async engine, and a high iodepth. Because each thread on each host will generate a file of the specified size, it’s easiest to have each job just generate one file (`nrfiles=1`) and vary the filesize and number of threads (`numjobs`). You may want to customize `numjobs` and `filesize` to change how much data is written/read for the job.

## Running the test

Now, we can run each job file with:

```bash
fio --client=hosts.txt 1m_write.fio
fio --client=hosts.txt 1m_read.fio
fio --client=hosts.txt 1m_randread.fio
fio --client=hosts.txt 1m_randwrite.fio
```

### Test sequencing

The job files generated from the script will all use the same data files to save time between tests. An artifact of this is that smaller block writes will fragment the file and impact larger block reads. To address this testing artifact, either:

1. Delete and recreate the `/mnt/vast/fio` directory between each test run
2. Run the tests in this sequence: blocksize high → low; sequential write, sequential read, random read, random write, or in psudocode:

```bash
for bs in 1m 512k 128k 64k; do
    for pattern in write read randread randwrite; do
        fio --client=hosts.txt ${bs}_${pattern}.fio
    done
done
```