Beginners guide and reference to Azure Blob Storage SDK v12 .Net C#

by Kaarthik · Published April 18, 2020 · Updated October 5, 2020

Azure Blob Storage is used across the globe by many people to store and retrieve data from the cloud. Blob storage service is mainly used for its data consistency, mutability, support blobs of different types and easy to use geo-redundancy. In this article, we are going to learn how to use the Azure blob storage SDK v12 in .Net with C# and this article can also be a guide for the beginners with blob storage service.

We have covered the commonly used operations of Azure Blob Storage service with SDK v12 in this article and grouped the related topics for your reference below

Azure Blob Storage Emulator and Explorer

You can use the Azure Blob Storage Emulator for learning purposes and can use the Azure Storage Explorer to view or manage files uploaded to the storage emulator. Please find below the download links

Azure Blob Storage Emulator: https://go.microsoft.com/fwlink/?linkid=717179&clcid=0x409 (Link is subjected to change by Microsoft at any time. At present this link provides direct download to storage emulator)

Azure Blob Storage Explorer: https://azure.microsoft.com/en-us/features/storage-explorer/

Source Code

You can download the full working solution from the GitHub at https://github.com/kaarthikin/reference_to_azure_blob_storage_sdk_v12 or view the functions of the code snippet mentioned in this article at https://github.com/kaarthikin/reference_to_azure_blob_storage_sdk_v12/blob/master/ReferenceToAzureBlobStorageSdkV12/BlobStorageService.cs

NuGet Package

The namespace of the blob storage service has been changed to Azure.Storage.Blobs. SDK v12.4.1 was the latest at the time of this article.

If you are using the package manager console to install the package then use the below command

Install-Package Azure.Storage.Blobs

If you are using package manager GUI then install the package shown in the below image.

The advantage of SDK v12 over the previous versions is that our 80% of the commonly used actions can be covered by the three major classes

BlobContainerClient - Covers all container related actions
BlobClient - Covers all blob related actions
BlobSasBuilder - Covers SAS token generation

Note : All the date-time values that are assigned to entities using this SDK should be of UTC format. For example, the date-time value used for SAS token generation or in the access-policy creation to a container should be in UTC format

Container Actions

1. Creating a storage container if it does not exist

Creating a blob container will be our initial step to get started working with the Azure Blob Storage service. The containers are created inside the Azure's storage account. You can learn to create a storage account at https://docs.microsoft.com/en-us/azure/storage/common/storage-account-create?tabs=azure-portal

Step 1: Initialize the BlobContainerClient object with the storage account connection string and the container name which has to be created.

Step 2: Call the method Create() or CreateIfNotExists() to create the container

Note: Container names can contain only lowercase letters, numbers, and hyphens, and must begin and end with a letter or a number. The name can't contain two consecutive hyphens.

public void CreateContainerIfDoesNotExists(string containerName)
{
	var container = new BlobContainerClient(connectionString, containerName);
	container.CreateIfNotExists();
}

2. Deleting a storage container if it exists

Deleting a storage container is a straightforward action. Initialize the BlobContainerClient and call the method DeleteIfExists() or Delete()

public void DeleteContainerIfExists(string containerName)
{
	new BlobContainerClient(connectionString, containerName).DeleteIfExists();
}

3. Add new or update existing access-policies to a container

Access policies are mainly used when we are generating a Shared Access Signature (SAS) token for an external user to access the contents inside the container.

Once we create a policy with a predefined set of permissions with start time and end time then the policy name can be directly used for token generation. The generated SAS token will have the lifetime and permissions based on the values we define while we are creating it.

Step 1: Initialize the BlobContainerClient with connection string and container name. We should make sure the container is already created.

Step 2: Initialize a list of BlobSignedIdentifier object

Step 3: Add BlobSignedIdentifier to the created list based on the number of access policies to be created

Both the properties of the BlobSignedIdentifier; Id and AccessPolicy has to be filled.

The Id is the name of the policy that has to be created
AccessPolicy (BlobAccessPolicy) defines the permissions, start and end time for the policy

The permissions property is of type String and the value which we set for this property should be in correct order such as read, add, and so on. If the order is changed the exception "XML specified is not syntactically valid. 400 - Invalid XML Document for the following reasons" will be thrown.

The permission string with full permissions and order is racwdl

r - Read
a - Add
c - Create
w - Write
d - Delete
l - List

Step 4: Invoke the method BlobContainerClient.SetAccessPolicy() and pass the newly created list of policies.

public void AddNewOrUpdateAccessPoliciesToContainer(string containerName)
{
	var container = new BlobContainerClient(connectionString, containerName);

	var accessPolicies = new List<BlobSignedIdentifier>();
	accessPolicies.Add(
		new BlobSignedIdentifier
		{
			Id = "Read_Write_Policy",
			AccessPolicy = new BlobAccessPolicy
			{
				Permissions = "rw",
				StartsOn = DateTime.UtcNow,
				ExpiresOn = DateTime.UtcNow.AddDays(1)
			}
		});
		
	accessPolicies.Add(new BlobSignedIdentifier
	{
		Id = "Full_Access_Policy",
		AccessPolicy = new BlobAccessPolicy
		{
			Permissions = "racwdl",
			StartsOn = DateTime.UtcNow,
			ExpiresOn = DateTime.UtcNow.AddDays(1)
		}
	});

	container.SetAccessPolicy(permissions: accessPolicies);
}

4. Remove access-policy from a container if it exists

We do not have a straightforward method to remove the access policy with its name. To remove an access policy the steps that must be followed are

Step 1: Initialize the BlobContainerClient with connection string and container name

Step 2: Call the method container.GetAccessPolicy() and get the already existing list of access policies

Step 3: Remove the required values from the list and call container.SetAccessPolicy() with the modified list

public void RemoveAccessPolicyIfExists(string containerName, string policyName)
{
	var container = new BlobContainerClient(connectionString, containerName);

	var exisitingPolicies = container.GetAccessPolicy().Value.SignedIdentifiers.ToList();
	var policyToBeRemoved = exisitingPolicies.SingleOrDefault(x => x.Id == policyName);

	if (policyToBeRemoved == null)
		return;

	exisitingPolicies.Remove(policyToBeRemoved);
	container.SetAccessPolicy(permissions: exisitingPolicies);

}

5. Set public access-policy to a container

Setting public level access container allows anyone without a SAS token to access the blobs in the container. The steps to be followed are

Step 1: Initialize the BlobContainerClient with connection string and container name

Step 2: Call the container.SetAccessPolicy() with PublicAccessType.BlobContainer

Use PulicAccessType.BlobContainer if you want to provide public access to both blobs and container or PublicAccessType.Blob if you want to provide public access only for blob

public void SetPublicAccessLevel(string containerName)
{
	var container = new BlobContainerClient(connectionString, containerName);
	container.SetAccessPolicy(PublicAccessType.BlobContainer); //Set PublicAccessType.Blob if public access is only for blob
}

6. Remove public access-policy from a container

Removing public access is similar to setting public access, execute the same steps for setting public access and pass the value to container.SetAccessPolicy() as PublicAccessType.None

public void RemovePublicAccessLevel(string containerName)
{
	var container = new BlobContainerClient(connectionString, containerName);
	container.SetAccessPolicy(PublicAccessType.None);
}

7. List all files or blobs from a container

The BlobContainerClient provides a direct method GetBlobs() to list all blobs under the container. The return response of GetBlobs() is an enumerable pageable object, iterate using foreach to loop to get its details.

public void ListAllBlobInCotainer(string containerName)
{
	var blobs=new BlobContainerClient(connectionString, containerName).GetBlobs();
	foreach(var blob in blobs)
	{
		Console.WriteLine(blob.Name);
	}
}

Blob Actions

1. Upload blob to a container

The BlobClient makes it easy for us to upload a file directly from a path in local storage or a stream.

Step 1: Initialize the BlobClient with a connection string, container name where the blob has to be uploaded and blob name in which the file name has to be stored.

Step 2: call the method blobClient.Upload() with the file path as string pointing to the file in your local storage.

You can use the property overwrite: true if you want to override the existing file.

public void UploadBlob(string containerName, string blobName, string filePath)
{
	var blob = new BlobClient(connectionString, containerName, blobName);
	blob.Upload(filePath, overwrite: true);
}

2. Upload blob to a folder in a container

To organize the files, we tend to create folders. We can also create folders in the blob storage container and organize our files in it. We do have separate methods or function calls to create the folders. The folders are automatically created based on the file name and deleted if no files exist in the folder.

The create a folder prefix the blob name passed to the Blob Client with the number of folders of your choice.

For example, if I need to upload to file temp.txt to folder "Testing" and inside "Testing" folder another folder called "TextFiles" then my blob name will look like

Testing/TextFiles/temp.txt

This will create 2 folders with hierarchy and place the file temp.txt in the TextFiles folder

3. Delete blob from a container

We have a direct method to delete the blob. Initialize the blob client with blob name to be deleted and call the method DeleteIfExists().

public void DeleteBlob(string containerName, string blobName)
{
	new BlobClient(connectionString, containerName, blobName).DeleteIfExists();
}

4. Download blob from a container

Like upload blob, we have multiple methods to download a blob. The simplest way is the initialize the blob client and call the method blobClient.DownloadTo() and provide the path to be download as an input parameter.

public void DownloadBlob(string containerName, string blobName, string pathToDownload)
{
	new BlobClient(connectionString, containerName, blobName).DownloadTo(pathToDownload);
}

5. Get blob properties (File size, content type, created on, updated on)

In many cases, we need to perform business actions on the file size, content type or date and time of the file modified.

To retrieve these metadata from the blob, initialize the blob client with the file for which you intend to get the properties and call the method GetProperties().Value.

From there you can retrieve all the required metadata like file name and so on.

public void GetBlobProperties(string containerName, string blobName)
{
	var blob = new BlobClient(connectionString, containerName, blobName);
	var properties = blob.GetProperties().Value;
	Console.WriteLine($"File size {ByteSize.FromBytes(properties.ContentLength).MebiBytes.ToString("#.##")} MB");
	Console.WriteLine($"Content type {properties.ContentType}");
	Console.WriteLine($"Created On {properties.CreatedOn}");
	Console.WriteLine($"Updated On {properties.LastModified}");
}

6. Copy blob from one blob storage container to another or from one account to another

The memory-optimized method to copy a blob from one container to another or from one account to another is to use the BlobClient's inbuild method StartCopyFromUri.

When the StartCopyFromUri is invoked, the request is queued to the blob storage service to perform the copy operation and blob service will internally copy the blob.

The steps to be followed are

Step 1: Initialize the BlobClient object based on the destination container or account where the blob has to be copied.

Step 2: Call the method StartCopyFromUri() with source blob URL

Step 3: The copy happens asynchronously wait for the copy to be completed by calling the WaitForCompletionAsync() method with the time interval to be probe the blob storage service for copy status.

The status code 200 from WaitForCompletionAsync() indicates the copy is completed.

Whenever you are copying the blobs within the storage containers under the same storage account, remove the SAS token from blob URLs and initialize the destination blob client with the connection string.

The connection string used in the destination blob client object will also be used to access the source file.

Make sure to remove the SAS token from the URL. If the SAS token is present, then the BlobClient will be initialized based on the SAS token even though the connection string is provided.

When copying files between two different accounts, the URL of the source and destination file must have a SAS token for the copy operation to take place. The connection string of one account cannot be shared with another one so the copy can take place only with the SAS token. SAS token is not required for the public containers.

public async Task CopyBlobFromOneContainerToAnother(string sourceUrl,string destinationUrl)
{
	var destBlobDetails = new BlobUriBuilder(new Uri(destinationUrl));
	var initiateCopy=new BlobClient(connectionString, destBlobDetails.BlobContainerName, destBlobDetails.BlobName).StartCopyFromUri(new Uri(sourceUrl)); //Start Copy queues copy operation to Azure Blob Storage/Emulator service and it happens in background. 
	var response = await initiateCopy.WaitForCompletionAsync(new TimeSpan(0, 0, 10), new System.Threading.CancellationToken()); //In-order to wait for copy function to complete call WaitForCompletionAsync with time interval to know the status. Here once in 10 seconds we are querying the storage service for copy status.


	if (response.GetRawResponse().Status == 200) //Status code 200 will be returned once copy is completed
		Console.WriteLine("Copy completed");
}

Shared Access Signature (SAS) Token Generation

1. Generate Ad Hoc SAS token for a blob

Shared Access Signature Token (SAS) plays a major role in providing access to the blobs of a private container to an external user. We can use the BlobSasBuilder class to generate the SAS token.

The BlobSasBuilder supports generating SAS token for both the blob storage container and the blobs inside it.

The BlobSasBuilder has a property called Resource, the values it accepts are "b" for SAS token at blob level and "c" for SAS token at the container level.

The steps to be followed to generate the token are

Step 1: Initialize the BlobSasBuilder with Container Name, Blob Name for which SAS token has to be generated with start time, end time and Resource.

Step 2: Set the permissions that have to be granted by calling SetPermissions() method of the BlobSasBuilder.

Step 3: Initialize the object StorageSharedKeyCredential with Account Name and Account Key

Step 4: Call the ToSasQueryParameters() of BlobSasBuilder and the StorageSharedKeyCredential object to get the SAS token

Step 5: Append the token to the Blob URL and share it with the user

public void GetAdHocBlobSasToken(string containerName, string blobName)
{
	var sasBuilder = new BlobSasBuilder()
	{
		BlobContainerName = containerName,
		BlobName = blobName,
		Resource = "b",//Value b is for generating token for a Blob and c is for container
		StartsOn = DateTime.UtcNow.AddMinutes(-2),
		ExpiresOn = DateTime.UtcNow.AddMinutes(10),
	};

	sasBuilder.SetPermissions(BlobSasPermissions.Read | BlobSasPermissions.Write); //multiple permissions can be added by using | symbol

	var sasToken = sasBuilder.ToSasQueryParameters(new StorageSharedKeyCredential(GetKeyValueFromConnectionString("AccountName"), GetKeyValueFromConnectionString("AccountKey")));

	Console.WriteLine($"{new BlobClient(connectionString, containerName, blobName).Uri}?{sasToken}");
}

2. Generate access-policy based SAS token for a blob

The access policies are created and assigned to a container. The BlobSasBuilder supports generating SAS token based on the assigned access-policy.

To generate the SAS token based on access policy, the property Identifier has to be set with the name of the access policy which is already created and assigned to the container.

When the token generation is based on access policy, the permissions or the start and end time should not be specified.

Even if you specify in the code, the SAS token will be generated.

But when you try to access the URL with the generated token you will be getting the error "Access policy fields can be associated with signature or SAS identifier but not both"

The permissions, start and end time will be picked from the access policy assigned to the container. In previous versions of SDK, it was allowed to mix up both the access policy and modifying the start or end time before the token generation, but starting with SDK v12 this is not allowed.

public void GetAccessPolicyBasedSasToken(string containerName, string blobName, string accessPolicyName)
{
var sasBuilder = new BlobSasBuilder()
{
	BlobContainerName = containerName,
	BlobName = blobName,
	Resource = "b",//Value b is for generating token for a Blob and c is for container
	Identifier = accessPolicyName
};

var sasToken = sasBuilder.ToSasQueryParameters(new StorageSharedKeyCredential(GetKeyValueFromConnectionString("AccountName"), GetKeyValueFromConnectionString("AccountKey")));
Console.WriteLine($"{new BlobClient(connectionString, containerName, blobName).Uri}?{sasToken}");
}

3. Generate Ad Hoc SAS token for a container

Generating SAS token for a container is the same as generating a token for blob using the BlobSasBuilder. The only difference is that we will not be providing the blob name, and the resource property must be set to "c".

public void GetAdHocContainerSasToken(string containerName)
{
	var sasBuilder = new BlobSasBuilder()
	{
		BlobContainerName = containerName,
		Resource = "c", //Value b is for generating token for a Blob and c is for container
		StartsOn = DateTime.UtcNow.AddMinutes(-2),
		ExpiresOn = DateTime.UtcNow.AddMinutes(10),
	};

	sasBuilder.SetPermissions(BlobContainerSasPermissions.Read | BlobContainerSasPermissions.Write | BlobContainerSasPermissions.List); //multiple permissions can be added by using | symbol

	var sasToken = sasBuilder.ToSasQueryParameters(new StorageSharedKeyCredential(GetKeyValueFromConnectionString("AccountName"), GetKeyValueFromConnectionString("AccountKey")));


	//Console.WriteLine($"{new BlobContainerClient(connectionString, containerName).Uri}?{sasToken}");
	Console.WriteLine($"{new BlobContainerClient(connectionString, containerName).Uri}?{sasToken}&restype=container&comp=list");

	/* Note : If you want to list the items inside container and view those details in a browser based on the generated SAS token
	 * then two additional query parameters has to be appended to the token
	 * the Query parameters are "restype=container&comp=list"
	 */
}

4. Generate access-policy based SAS token for a container

Generating SAS token for a container based on access policy is the same as generating SAS token for blob based on access policy. The only difference is we will not be providing the blob name and the resource property must be set to "c".

public void GetAccessPolicyBasedContainerSasToken(string containerName,string accessPolicy)
{
var sasBuilder = new BlobSasBuilder()
{
BlobContainerName = containerName,
Resource = "c", //Value b is for generating token for a Blob and c is for container
StartsOn = DateTime.UtcNow.AddMinutes(-2),
ExpiresOn = DateTime.UtcNow.AddMinutes(10),
};

sasBuilder.SetPermissions(BlobContainerSasPermissions.Read | BlobContainerSasPermissions.Write | BlobContainerSasPermissions.List); //multiple permissions can be added by using | symbol

var sasToken = sasBuilder.ToSasQueryParameters(new StorageSharedKeyCredential(GetKeyValueFromConnectionString("AccountName"), GetKeyValueFromConnectionString("AccountKey")));


//Console.WriteLine($"{new BlobContainerClient(connectionString, containerName).Uri}?{sasToken}");
Console.WriteLine($"{new BlobContainerClient(connectionString, containerName).Uri}?{sasToken}&restype=container&comp=list");

/* Note : If you want to list the items inside container and view those details in a browser based on the generated SAS token
* then two additional query parameters has to be appended to the token
* the Query parameters are "restype=container&comp=list"
*/
}

Other Actions

1. Get account name, container name, and blob name from URL

To get the account, container or blob name from then URL we can use the BlobUriBuilder class.

Initialize the class with the URI for which details have to be fetched and use its properties to get the required names.

This operation is very light weighted and will not call the blob storage service to retrieve the details.

public void GetAccountContainerAndBlobNameFromUrl(string Url)
{
	var details = new BlobUriBuilder(new Uri(Url));

	Console.WriteLine($"Account Name {details.AccountName}");
	Console.WriteLine($"Container Name {details.BlobContainerName}");
	Console.WriteLine($"Blob Name {HttpUtility.UrlDecode(details.BlobName)}"); //When URL has spaces it will be encoded as %20, decode the value to get correct file name
}

2. Get account name and account key from a connection string

The Azure blob storage connection string has the account name, key and endpoint URL embedded in it, we don't want to store them separately in variables or DB and parse at runtime whenever it is required

The below code snippet shows converting the connection string into a dictionary and reading the values based on a key.

private string GetKeyValueFromConnectionString(string key)
{
IDictionary<string, string> settings = new Dictionary<string, string>();
var splitted = connectionString.Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries);

foreach (var nameValue in splitted)
{
var splittedNameValue = nameValue.Split(new char[] { '=' }, 2);
settings.Add(splittedNameValue[0], splittedNameValue[1]);
}

return settings[key];
}

Comments2
Pingbacks0

heedong0612 says:

February 25, 2021 at 5:58 pm

YOU ARE SERIOUSLY THE BEST!!!

Douglas says:

August 6, 2021 at 5:31 pm

Holy Gzzz.. this is such a valuable content!!

Translate

Beginners guide and reference to Azure Blob Storage SDK v12 .Net C#

Container Actions

Blob Actions

Shared Access Signature (SAS) Token Generation

Other Actions

Azure Blob Storage Emulator and Explorer

Source Code

NuGet Package

Container Actions

1. Creating a storage container if it does not exist

2. Deleting a storage container if it exists

3. Add new or update existing access-policies to a container

4. Remove access-policy from a container if it exists

5. Set public access-policy to a container

6. Remove public access-policy from a container

7. List all files or blobs from a container

Blob Actions

1. Upload blob to a container

2. Upload blob to a folder in a container

3. Delete blob from a container

4. Download blob from a container

5. Get blob properties (File size, content type, created on, updated on)

6. Copy blob from one blob storage container to another or from one account to another

Shared Access Signature (SAS) Token Generation

1. Generate Ad Hoc SAS token for a blob

2. Generate access-policy based SAS token for a blob

3. Generate Ad Hoc SAS token for a container

4. Generate access-policy based SAS token for a container

Other Actions

1. Get account name, container name, and blob name from URL

2. Get account name and account key from a connection string

You may also like...

2 Responses

Leave a ReplyCancel reply

Search

Recent Posts

Beginners guide and reference to Azure Blob Storage SDK v12 .Net C#

Container Actions

Blob Actions

Shared Access Signature (SAS) Token Generation

Other Actions

Azure Blob Storage Emulator and Explorer

Source Code

NuGet Package

Container Actions

1. Creating a storage container if it does not exist

2. Deleting a storage container if it exists

3. Add new or update existing access-policies to a container

4. Remove access-policy from a container if it exists

5. Set public access-policy to a container

6. Remove public access-policy from a container

7. List all files or blobs from a container

Blob Actions

1. Upload blob to a container

2. Upload blob to a folder in a container

3. Delete blob from a container

4. Download blob from a container

5. Get blob properties (File size, content type, created on, updated on)

6. Copy blob from one blob storage container to another or from one account to another

Shared Access Signature (SAS) Token Generation

1. Generate Ad Hoc SAS token for a blob

2. Generate access-policy based SAS token for a blob

3. Generate Ad Hoc SAS token for a container

4. Generate access-policy based SAS token for a container

Other Actions

1. Get account name, container name, and blob name from URL

2. Get account name and account key from a connection string

Share this:

Like this:

Related

You may also like...

Create Repository Workspace, Load Components, Set Flow Table and Accept Change sets – IBM RTC Plain Java API

Custom Authorization Policy Provider with Custom Authorize Attribute in Asp .Net Core 2.2 and above

OpenSocial Gadget Development for IBM RTC

2 Responses

Leave a ReplyCancel reply

Search

Recent Posts