S3 compatibility

Applies to VAST Cluster 5.5.

VAST Cluster 5.5 supports a subset of the Amazon S3 REST API. This page lists the supported requests, the supported headers, the differences from AWS S3 that affect applications, and VAST's own additions. When a request isn't listed here, assume it isn't supported and test before you depend on it.

Protocol basics

Supported requests

The supported requests, with the IAM action each needs in a policy. Each request in the table was run on a 5.5 cluster, except PutBucketLogging, PutBucketNotificationConfiguration, PutBucketOwnershipControls, DeleteObjectTagging, the per-version tagging requests and AssumeRoleWithWebIdentity:

Headers

These request headers are supported: Host, Expect, Content-Length, Content-MD5, Content-Type, Expires, Transfer-Encoding, Connection, Range, If-Match, If-None-Match, x-amz-meta-*, x-amz-copy-source, x-amz-copy-source-if-match, x-amz-copy-source-if-none-match, x-amz-copy-source-if-modified-since, x-amz-copy-source-if-unmodified-since, x-amz-copy-source-range, x-amz-grant-*, x-amz-acl, x-amz-metadata-directive, x-amz-content-sha256, x-amz-decoded-content-length, x-amz-trailer, x-amz-checksum-*, x-amz-tagging, x-amz-tagging-directive, x-amz-bucket-object-lock-enabled, x-amz-object-ownership, x-amz-bypass-governance-retention, x-amz-object-lock-mode, x-amz-object-lock-retain-until-date, x-amz-object-lock-legal-hold, the CORS request headers, and the SSE-C headers (x-amz-server-side-encryption-customer-*).

Content-MD5 is checked on uploads and parts (a mismatch gets 400 BadDigest), and accepted on configuration requests (ACLs, policies, tagging, versioning, lock, logging, notifications, ownership) and DeleteObjects. Uploads with an x-amz-checksum-* header (CRC32, SHA-1 or SHA-256, including the CRC32 checksum that recent AWS SDKs add by default) are accepted, but responses don't return a checksum. A copy whose x-amz-copy-source-if-match doesn't match fails with 400 InvalidArgument, where AWS returns 412 PreconditionFailed.

Not supported. These requests fail: bucket encryption, replication, website, transfer acceleration and Requester Pays configuration (501 NotImplemented), PutPublicAccessBlock (400 MalformedXML), SelectObjectContent (404 NoSuchKey) and GetObjectAttributes (500 Internal Server Error).

Differences from AWS S3

Bucket configuration

• Lifecycle rules are set in VMS, not with the S3 API. Expiration, cleanup of incomplete multipart uploads and of expired delete markers are lifecycle rules that your platform team sets in VMS. Rules can apply to files over any protocol, not only objects. PutBucketLifecycleConfiguration isn't a supported request: the cluster accepts it and returns the configuration from GetBucketLifecycleConfiguration, but no lifecycle rule is created from it, so don't rely on it.
• Versioning can't be suspended, only enabled: Suspended gets 501 NotImplemented. Versioning needs a view with no protocol other than S3: enabling it on a view that also has NFS fails. No MFA Delete.
• Object locking needs a view with no protocol other than S3, can't be disabled and turns on versioning. Both retention modes work: in governance mode, a user with s3:BypassGovernanceRetention can delete a locked version early; in compliance mode, nobody can delete it, shorten its retention or switch it to governance before the date, not even with the bypass header. Legal holds work too (versioning and object lock).
• Object ownership defaults to ObjectWriter (ACLs enabled, the uploader owns the object), where AWS defaults to BucketOwnerEnforced. Ownership controls apply only to views with the S3 Native security flavor.
• Policies apply only with the S3 Native flavor. On views with another security flavor, identity and bucket policies are ignored and file permissions decide (concepts).
• CORS rules have no IDs: a rule with an ID gets 400 MalformedXML. A failed PutObject response doesn't carry Access-Control-Allow-Origin.
• Bucket logging and event notifications are supported; event notifications go to VAST Event Broker or a third-party Kafka broker (S3 bucket events).

Objects and requests

• Conditional writes (If-None-Match: * on PutObject and CompleteMultipartUpload) aren't supported in versioned buckets (501 NotImplemented). Conditional reads (If-Match on GET and HEAD) and conditional deletes are.
• ETags of objects written over NFS or SMB aren't MD5s of the data; they're computed from metadata (multiprotocol access).
• Multipart: part numbers run from 1 to 10,000, as on AWS; other numbers get 400 InvalidPart. Unlike AWS, the cluster doesn't enforce a minimum part size (parts of one byte are accepted) or a 5 GiB maximum for a part or a single PutObject (5 GiB + 1 byte was accepted for both). Keep to the AWS limits anyway (parts of 5 MiB to 5 GiB, except the last), so your code works with other S3 stores and with clients that check them. An absolute URL in the CompleteMultipartUpload request line isn't supported.
• Presigned POST: object names must be ASCII (a non-ASCII name gets 403 AccessDenied), and the POST policy is limited in size (limits).
• SSE-C: requests with SSE-C headers must use HTTPS (over HTTP they get 400 InvalidArgument); only AES256; SSE-C objects can't be read over NFS or SMB.
• Keys with //, a leading / or .//../ parts have file-system implications; some are refused unless the view policy allows them (concepts). With the default settings, keys with // or a leading / got 400 InvalidObjectName, and keys with ./ or ../ parts were accepted but had no file over NFS.

VAST additions

See also

• Limits.
• KB: Supported S3 Requests, Supported S3 HTTP Headers and the 5.5.0 release notes.