C#: List All Objects in a S3 Bucket

AWS S3 is a great service for storing personal files or files related to a website, app, or business. The .NET AWS SDK is a simple, straight forward SDK for access your S3 file and all AWS services.

The code below can be used to list all objects within an S3 bucket. You can also add a key prefix if you want to limit the results.

The main concept to grasp with the code below is that the S3 SDK will not always return all objects within the bucket. If you have a very large bucket, you will only get a subset of objects (I think 1,000). When this happens, the SDK will give you a Continuation Token which can be used on a subsequent request to get the next batch of records. When there are no more records to get, the Continuation Token will be null.

copy

static public async Task<List<Amazon.S3.Model.S3Object>> S3__List_Objects(string bucket, string key_prefix = "")
{
	var s3_objects = new List<Amazon.S3.Model.S3Object>();


	//using an access key and secret key directly			
	var access_key = "YOUR_ACCESS_KEY";
	var secret_key = "YOUR_SECRET_KEY";
	var credentials = new BasicAWSCredentials(access_key, secret_key);
	using var client = new AmazonS3Client(credentials, Amazon.RegionEndpoint.USEast1);


	//using an AWS profile store in your user/.aws/config file
	//var profile_name = "YOUR_PROFILE_NAME";
	//var credentials = new StoredProfileAWSCredentials(profile_name);
	//using var client = new AmazonS3Client(credentials, Amazon.RegionEndpoint.USEast1);


	//running in AWS Lambda
	//using var client = new AmazonS3Client();


	var request = new Amazon.S3.Model.ListObjectsV2Request()
	{
		BucketName = bucket,
		Prefix = key_prefix,
	};
	

	do
	{
		var response = await client.ListObjectsV2Async(request);
		if (response.HttpStatusCode == System.Net.HttpStatusCode.OK)
		{
			s3_objects.AddRange(response.S3Objects);
			request.ContinuationToken = response.NextContinuationToken;
		}
		else
		{
			break;
		}
	} while (string.IsNullOrWhiteSpace(request.ContinuationToken) == false);


	return s3_objects;
}