This section will recommend designs and starting settings for applications on VAST S3 and NFS: client settings for throughput, how to lay out buckets and keys, how to combine S3 and NFS on the same data, and how to configure the common data tools. Recommendations are a starting point, based on benchmarks and deployments. Always measure with your own data before you rely on them.
What this section will contain
- S3 client settings for throughput: concurrency, connection pools, part sizes and thresholds for boto3, the AWS CLI, s3fs, PyArrow and S3A, per object-size profile.
- Spreading load over the cluster: how many virtual IPs and endpoints to use per client, DNS versus explicit VIP lists, and client counts for a target throughput.
- NFS mounts for each workload:
nconnect, multipath and RDMA choices, NFS version choice, and settings for AI training and checkpointing, HPC and analytics. - Bucket and key design: prefixes for parallel listing and processing, object sizes, and small-file strategies.
- Multiprotocol designs: choosing a security flavor for S3-and-NFS pipelines, identity mapping between S3 users and POSIX users, and producer-consumer patterns across protocols.
- Spark and Hadoop on VAST S3: committers, overwrite strategies and table formats.
- Data protection patterns: when to use versioning, object lock and legal holds, and their capacity cost.
Each page will say which VAST Cluster version and test setup its recommendations come from.
Where to find how it works
The rest of the VAST DataStore docs explain how each feature works and what each setting does: concepts, performance, large objects and multipart uploads, multiprotocol access, NFS for applications and the integrations. Best-practices pages link to those pages rather than repeat them.
Status
This section is being written. Until its pages are published, start with the pages above, and ask your VAST account team for guidance on your workload.