--- title: "Using the Boto3 S3 Service Resource" slug: "using-the-boto3-s3-service-resource-tg" tags: ["5.5"] updated: 2026-10-27T23:00:54Z published: 2026-10-27T23:00:54Z canonical: "docs.vastdata.com/using-the-boto3-s3-service-resource-tg" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Using the Boto3 S3 Service Resource The S3 resource class has actions that are S3 service operations that pre-fill some of the parameters for you. Actions can return three types of responses: - A low level response that is a python dictionary which you're responsible for handling - A resource - A list of resources Sub-resources are actions that create new instances of child resources. The parent resource’s identifiers get passed to the child resource. The S3 service resource has several sub resources, notably bucket and object, which enable you to construct buckets and objects in the buckets. Some resources have collections, which are groups of resources , allowing you to iterate, filter and manipulate resources. For example, a bucket can have a collection of objects that you can iterate through to access information such as which objects are in the bucket. ## Connecting to the Boto3 Resource Interface To connect to the S3 service using a resource, import the Boto3 module and then call Boto3's [resource()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource) method, specifying 's3' as the service name to create an instance of an S3 service resource. You must pass your VAST S3 credentials and other configurations as parameters into the resource() method. See the Boto3 [resource()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource) documentation page for a complete list and description of the configuration parameters that can be passed in the resource() method. See also [S3 Service Resource](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#service-resource) page. ### Required Parameters The following parameters must be passed in the [resource()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource) call since the endpoint URL and access key pair must be hardcoded: - *endpoint_url* (string). The endpoint URL can be any of the cluster's Virtual IPs, prefixed by *http://* if you are disabling SSL or *https://* if you are enabling SSL. For example, `http://198.51.100.255`, in which *198.51.100.255* is one of the cluster's VIPs. > [!NOTE] > Note > > To retrieve the cluster's virtual lPs: > > - In the VAST Web UI, open the menu (click ![menu_button.png](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/img-cefc1f3d9a04c4d52ab88c0aaf9f0192(71).png?sv=2026-02-06&spr=https&st=2026-11-08T09%3A18%3A43Z&se=2026-11-08T10%3A18%3A43Z&sr=c&sp=r&sig=D0dByaJmVeO8t8JJKIQU1cBLeCUNBlDE1NCAr7cYcLM%3D)), select Configuration and then select the Virtual IPs tab. The Virtual IPs list shows you which virtual IPs are configured on each CNode. > - In the VAST CLI, run the `vip list` command. - *aws_access_key_id* (string) and *aws_secret_access_key* (string). Pass these parameters your [S3 key pair](/docs-5-5/docs/managing-s3-user-permissions-tg#managing-s3-user-access-from-thevast-web-ui) values. If you are passing `signature_version=S3v4`, then you must pass *region_name* (string). You can use any string value for the region name. ### For HTTPS Connections For an HTTPS connection, pass parameters as follows in the [resource()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource) call: - Enable HTTPS by setting `use_ssl=True`. - If the default certificate trust store does not recognize the signer of the installed certificate, you can use the `verify` parameter to specify a non default path to the certificate trust store. If you're using a self signed certificate, you can point this to the certificate itself. For example: ```plaintext verify="path/to/client/cert.pem" ``` - Alternatively, you can use the `verify` parameter to disable verification: ```plaintext verify=False ``` ### For HTTP Connections Set `use_ssl=False`. ### Example In this example, some optional advanced session configurations are passed in. The endpoint URL, S3 user access key and secret key are hard coded, as needed. SSL is disabled, so the connection will be over HTTP. ```plaintext # Import the boto3 module import boto3 # Set advanced configuration parameters boto_params = {'s3': {'addressing_style': 'path'}, 'signature_version': 's3v4'} # Create a resource to the Amazon S3 service, passing configurations and credentials as parameters s3_resource = boto3.resource('s3', use_ssl=False,                                   endpoint_url=,                                   aws_access_key_id=,                                   aws_secret_access_key=,                                                       region_name=,                                   config=boto3.session.Config(**self.boto_params),                                           ) ``` > [!NOTE] > Note > > All further examples assume an S3 service resource named*s3_resource* was created. ## List All Buckets on the Server The [buckets](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ServiceResource.buckets) collection is the collection of all buckets on the S3 server and it has a method *all()* that creates an iterable of all the bucket resources in the collection. To return all the bucket resources on the server, you can Iterate through all the buckets returned by *buckets.all()*. The bucket resource has an identifier [*name*](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.name) and an attribute [*creation_date*](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.creation_date). ### Example In this example, we iterate through all the buckets on the server. We print out the name of each bucket using the identifier *name*, which must be set for each bucket. ```plaintext for bucket in s3_resource.buckets.all(): print (bucket.name) ``` ## Create a Bucket Once you have an instance of the S3 service resource, you can call the [create_bucket](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ServiceResource.create_bucket) action on the resource to create a bucket. This returns a new bucket resource. The operation is idempotent, so it will either create the bucket or return the existing bucket if it already exists. > [!NOTE] > Note > > You can optionally set ACL for the bucket while creating the bucket by passing optional parameters. For details of ACL support, see [Managing S3 Access Control Lists (ACLs)](/docs-5-5/docs/managing-s3-access-tg#managing-s3-access-control-lists-acls). For request syntax details, see the boto3 documentation for [create_bucket](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ServiceResource.create_bucket). ### Example In this example, we create a bucket called *mybucket* as a child resource of the S3 service resource *s3_resource*. ```plaintext s3_resource.create_bucket(Bucket="mybucket") ``` ## List Objects in a Bucket The collection [bucket.objects](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.objects) is the collection of all objects in a bucket and it has a method called all() that creates an iterable of all the object resources in the collection. To list all the objects in a bucket, you can iterate through the list of objects returned by *bucket.objects.all()*. The object resource has several identifiers and attributes that you could access this way, listed in the [object](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#object) resource documentation ### Example In this example, we instantiate an S3 bucket by its name "the bucket", and iterate through all of the objects in the bucket. We print out the name of each object, using the *key* identifier, and the last date each object was modified, using the *last_modified* attribute. ```plaintext s3_resource.Bucket("the_bucket") for s3_object in bucket.objects.all():    print(obj.key, obj.last_modified) ``` ## Delete a Bucket To delete a bucket, call the [delete()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.delete) action on the bucket resource. ### Example ```plaintext s3_resource.Bucket("the_bucket").delete() ``` ## Set the ACL of a Bucket Before setting ACL permissions, we recommend you read [Managing S3 Access Control Lists (ACLs)](/docs-5-5/docs/managing-s3-access-tg#managing-s3-access-control-lists-acls). Call the [put()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketAcl.put) action on a bucket's [BucketAcl](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#bucketacl) resource to set ACL permissions on the bucket. ### Example In this example, a user with VID 3 is granted full control permission to the bucket my_bucket owned by JDoe whose VID is 2. ```plaintext s3_resource.BucketAcl ('my_bucket').put( AccessControlPolicy={  'Grants': [        {           'Grantee': {                'ID': '3' ,                'Type': 'CanonicalUser'      },          'Permission':               'FULL_CONTROL'         },   ],      'Owner': {        'DisplayName': 'JDoe',        'ID': '2'      }   }, ) ``` ## Return the ACL of a bucket You can use the [grants](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketAcl.grants) attribute of the ACL resource to access grantee information, and the [owner](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketAcl.owner) attribute of the ACL resource to access owner information. ### Example In this example, we iterate through the ACL grants and print out the user name and permission of each grantee who has permission to access the bucket. ```plaintext acl = bucket.Acl() for grant in acl.grants:    print(grant['Grantee']['DisplayName'], grant['Permission']) ``` ## Create an Object You can call the [put_object()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.put_object) action on the [bucket](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#bucket) resource to add an object to the bucket. ### Example In this example, we add a file called "object_name" with a body "object data" to the "the_bucket" bucket, specifying a checksum value to store with the object. ```plaintext s3_resource.Bucket("the_bucket").put_object(Key="object_name", Body=b"object data", Metadata={ 'checksum': '1234567890'}) ``` ## Create a copy of an object The [copy_from()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.copy_from) action on the [object](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#object) resource creates a copy of an object already stored on the server. ### Example In this example, we create a new object called *file2* in bucket *bucket2* by copying an object called *file1* that is stored in bucket *bucket1*. *CopySource* is the only required parameter for *copy_from()*. In this case, we are specifying the source in string format. See [copy_from()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.copy_from) for full syntax details. ```plaintext s3_resource.Object('bucket2','file2').copy_from(CopySource='bucket1/file1') ``` ## Retrieve an Object The [get()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.get) action on the [object](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#object) resource retrieves an object. Use the *Range* parameter to download a specified range of bytes of an object. For more information about the HTTP Range header, go to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35. ### Example In this example, we retrieve the object "my_object" from the bucket "my_bucket" by calling the [get()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.get) action on the bucket. The response is a python dictionary, inside which is the 'Body', which is the body data in the format Streamingbody(). We read the content of the object by calling the read() function on the 'Body'. ```plaintext s3_resource.Object("my_bucket", "my_object").get()['Body'].read() ``` In this example, we download only the byte range 32-64 of the object "my_object" from the bucket "my_bucket". ```plaintext s3_resource.Object("my_bucket", "my_object").get(Range='bytes=32-64')['Body'].read() ``` ## Retrieve Object Metadata To retrieve metadata from an object without returning the object itself, use the [metadata](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.metadata) attribute on the [object](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#object) resource. ### Example ```plaintext s3_resource.Object("the_bucket", "object_name").metadata ``` ## Delete an Object To delete an object, call the [delete()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.delete) action on the [object](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#object) resource. ### Example ```plaintext s3_resource.Object("the_bucket", "object_name").delete() ``` ## Delete Multiple Objects To delete multiple objects in a bucket, call the [delete_objects()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.delete_objects) action on the [bucket](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#bucket) resource. ### Example ```plaintext s3_resource.Bucket("the_bucket").delete_objects(    Delete={        'Objects': [            {'Key': 'file1'},            {'Key': 'file2'},            {'Key': 'file3'},        ],    }, ) ``` ## Set ACL Permissions on an Existing Object Before setting ACL permissions, we recommend you read [Managing S3 Access Control Lists (ACLs)](/docs-5-5/docs/managing-s3-access-tg#managing-s3-access-control-lists-acls). To learn about VAST Cluster's support for S3 ACLs, read [Managing S3 Access Control Lists (ACLs)](/docs-5-5/docs/managing-s3-access-tg#managing-s3-access-control-lists-acls). The [ObjectAcl()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#objectacl) method on the S3 service resource creates a resource representing an S3 object's ACL. Alternatively, you can create the [ObjectAcl](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ObjectAcl) resource by calling the [Acl()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.Acl) method on the object resource. The [put()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.ObjectAcl.put) action on an *ObjectAcl* resource sets the ACL permissions for an object that already exists in a bucket. ###### Syntax Notes To grant permission to a user, specify the grantee with the following parameters: To grant permission to a predefined group, specify *Group* as the 'Type' and pass the group's URI as the 'URI': - For the All Users group: 'http://acs.amazonaws.com/groups/global/AllUsers' - For the Authenticated Users group: 'http://acs.amazonaws.com/groups/global/AuthenticatedUsers' ### Examples In this example, a user with VID 3 is granted full control permission to the object my_object in the bucket my_bucket owned by *JDoe* whose VID is 2. ```plaintext s3_resource.ObjectAcl('my_bucket', 'my_object').put(AccessControlPolicy = {      'Grants': [        {           'Grantee': {                'ID': '3' ,                'Type': 'CanonicalUser'      },          'Permission':               'FULL_CONTROL'         },   ],      'Owner': {        'DisplayName': 'JDoe',        'ID': '2'      }   }, ) ``` In this example, the predefined AUTHENTICATED_USERS group is granted WRITE permission to the object *my_object* in the bucket *my_bucket* owned by *JDoe* whose VID is 2.. ```plaintext #attempting group permission example - guessing to specify group name as "displayname" and that the name I used is sufficient and we don't need the long version s3_resource.ObjectAcl('my_bucket', 'my_object').put(AccessControlPolicy = {      'Grants': [        {       'Grantee':{       'Type ':'Group'       'URI': 'http://acs.amazonaws.com/groups/global/AuthenticatedUsers'       },      'Permission':'WRITE'         },   ],       'Owner': {        'DisplayName': 'JDoe',        'ID': '2'        }   }, ) ``` ## Return the ACL of an object You can use the [grants](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketAcl.grants) attribute of the ACL resource to access grantee information, and the [owner](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketAcl.owner) attribute of the ACL resource to access owner information. ### Example In this example, we iterate through the ACL grants and print out the user name and permission of each grantee who has permission to access the object. ```plaintext acl = object.Acl() for grant in acl.grants:    print(grant['Grantee']['DisplayName'], grant['Permission']) ``` ## Initiate a Multipart Upload Call the [initiate_multipart_upload()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.initiate_multipart_upload) action on the object resource to initiate a multipart upload and return an upload ID. After initiating the multipart upload, you then need to [upload all parts](/docs-5-5/docs/using-the-boto3-s3-service-resource-tg#upload-a-part-in-a-multipart-upload) and then [complete the upload](/docs-5-5/docs/using-the-boto3-s3-service-resource-tg#complete-a-multipart-upload). ## Abort a Multipart Upload To abort a multipart upload after it was initiated, call the [abort()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload.abort) action on the [MultipartUpload](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource. ## Complete a Multipart Upload To complete a multipart upload after it was initiated and all parts were uploaded, call the [complete()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload.complete) action on the [MultipartUpload](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource. Completing a multipart upload assembles the previously uploaded parts. ## Upload a part in a Multipart Upload To upload a part in a multipart upload that was already initiated, create a [MultipartUploadPart](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) sub resource of the [MultipartUpload](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource and call the [upload()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUploadPart.upload) action on the [MultipartUploadPart](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource. After uploading all parts, the upload needs to be [completed](/docs-5-5/docs/using-the-boto3-s3-service-resource-tg#complete-a-multipart-upload). ## Upload a part by copying data from an existing object as data source To upload a part of a multipart upload by copying data from an existing object as data source, create a [MultipartUploadPart](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) sub resource of the [MultipartUpload](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource and call the [copy_from()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUploadPart.copy_from) action on the [MultipartUploadPart](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload) resource. ## List Uploaded Parts for a Specific Multipart Upload The collection [parts](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.MultipartUpload.parts) available on a [MultipartUpload](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#multipartupload) resource is the collection of all parts in the multipart upload. It has a method called all() that creates an iterable of all the parts in the multipart upload. To list all parts in a multipart upload, you can iterate through the list of parts returned by *multipart_upload.parts.all()*. ## List Multipart Uploads The collection [multipart_uploads](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Bucket.multipart_uploads) is the collection of all multipart uploads in a bucket and it has a method called all() that creates an iterable of all the multipart uploads in the collection. To list all currently initiated multipart uploads in a bucket, you can iterate through the list of multipart uploads returned by *bucket.multipart_uploads.all()*.