Azure Storage Archives

Migrating files into Microsoft Azure Virtual Machines

We use Microsoft’s AzCopy to move large amounts of data from external sources into Microsoft Azure Storage.

The typical scenario for us is a customer who is moving onto an Azure Virtual Machine, and wants their data stored on that machine.

AzCopy is a versatile command line utility that allows you to move files from another PC or Server into Azure Storage, and then into your Azure virtual machine.

When migrating data to an Azure VM, the solution looks like this:

To get started, you need to install AzCopy from here.
Next, create an Azure Storage Account. You can do this in either the old portal or the new portal.
Once you’ve created a storage account, you’ll need to create a storage container. Here is a quick guide for the old portal and the new portal.

Now that you’ve installed AzCopy, created a storage account and a container, you can put together your AzCopy commands. Typically you’ll create two commands, one that uploads your data into Azure Storage, and the other downloads your data from Azure Storage into your Azure Virtual Machine.

Here’s an example that will move local data from E:\NAS into Azure Storage, and then download it to E:\NAS on the destination virtual machine:

To Azure Blob Storage

AzCopy /Source:E:\NAS /Dest:https://<storageaccountname>.blob.core.windows.net/<containername> /DestKey:<LongStorageAccountKey> /S /V:C:\temp\NASDrive.log

From Azure Blob Storage

AzCopy /Source:https://<storageaccountname>.blob.core.windows.net/<containername> /Dest:E:\NAS /SourceKey:<LongStorageAccountKey> /S /V:C:\temp\NASDrive.log

These commands will also create a log file under C:\temp\NASDrive.log

Running the AzCopy commands

Open Command Prompt and navigate to the location where AzCopy was installed. Typically this is under “C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy”

Paste in your first command and run it to start the upload. In the meantime, you can install AzCopy on the destination server.

Once the upload is complete, go to the destination server and run the second command to download the data from Azure Blob Storage.

If you experience errors

Occasionally you may experience errors in the AzCopy transfer (usually the upload), where certain files will fail. The solution for this is usually to append the /NC: parameter and run the command again. The /NC: parameter limits the number of concurrent connections to Azure storage. I usually set it to /NC:5, where 5 is the maximum number of concurrent files that will be uploaded. The upper limit of concurrent connections is 512.

Updated command for uploading to Azure Blob Storage

AzCopy /Source:E:\NAS /Dest:https://<storageaccountname>.blob.core.windows.net/<containername> /DestKey:<LongStorageAccountKey> /S /V:C:\temp\NASDrive.log /NC:5

If you’re rerunning the command, you will be asked whether you want to skip files that already exist. Choose to skip All.

Here’s the results of the second transfer. You may notice that the number of files transferred in this image is different to the number of files that failed in the previous image. In this case, we removed some unnecessary files before restarting the upload.

AzCopy Documentation

For more information on AzCopy, see the documentation here.

December 4, 2015/0 Comments/by Elliot Munro

How to create an Azure Storage Container in the old portal

Azure, Azure Storage

Here’s how to create an Azure Storage Account Container in the old Microsoft Azure portal at https://manage.windowsazure.com, assuming you already have an Azure subscription and existing Azure Storage account.

You will need an existing storage account. See this guide for how to create one.
Login to https://manage.windowsazure.com
Open your existing storage account.
Click Containers then click Create a container:
Choose a Name and Access type for it:
Click the tick and wait for the container to be created. It should just take a few seconds.
You can now refer to this container from other services using https://<yourstorageaccountname>.blob.core.windows.net/<yourcontainername>
Depending on the Access type you chose (eg. private), you may need your storage account name and primary key to access this container. See the end of this guide for how to retrieve these.

December 4, 2015/0 Comments/by Elliot Munro

How to create an Azure Storage Container in the new portal

Azure, Azure Storage

Here’s how to create an Azure Storage Account Container, assuming you already have an Azure subscription and existing Azure Storage account.

Firstly you’ll need a storage account. See this guide for how to create one.
Open your storage account by signing into https://portal.azure.com
Click Blobs
Click Containers in the Blob service blade.
Create the new container by clicking ‘+ Container’ then choosing a Name and Access type for it:
You can now refer to this container from other services using https://<yourstorageaccountname>.blob.core.windows.net/<yourcontainername>
Depending on the Access type you chose, you may need your storage account name and primary key. See the end of this guide for how to retrieve these.

December 4, 2015/0 Comments/by Elliot Munro

How to create an Azure Storage Account in the new portal

Azure, Azure Storage

Creating an Azure Storage account through the new Azure Portal is quite simple. Here’s how you do it.

To create an Azure Storage account in the old portal, follow this quick guide instead.

Log into your Azure Subscription at https://portal.azure.com
Click New, Data + Storage, Storage Account
Choose Create
Type a name for your storage account.

This will be the storage account name that you’ll use to access your storage.

Choose a pricing tier.

This is the replication level of your storage across Azure’s physical data centre locations. Different levels of replication offer greater redundancy in case of a datacentre outage.

Choose a Resource Group,

This groups your related services together in Azure. If you don’t have any other services running yet, click Create a new resource group

Choose the subscription

This will add the storage account under the chosen subscription, you may only have one.

Choose the location of the data

This is the physical location of the datacentres that will hold your storage account. If you’ve chosen Geo Replication, your data will also be replicated outside of this location.

Choose whether you want to use Diagnostics

Diagnostics will disable the regular monitoring charts and alerts for your storage resource, though will send diagnostic data into a storage account for your own monitoring.
Now, click Create.
Wait a few moments for your storage account to be created. Once completed, you can open your brand new Azure storage account!
To make use of this Storage account, click ‘Keys‘ on the settings blade to retrieve the storage account name and primary access key.

December 4, 2015/0 Comments/by Elliot Munro

How to create an Azure Storage Account in the old portal

Azure, Azure Storage

Creating an Azure Storage account through the current portal (non-preview) is quite simple. Here’s how you do it.

(To create an Azure Storage account in the new portal, follow this quick guide instead.)

Login to https://manage.windowsazure.com
Click Storage
Click New
Click Quick Create, then choose a name for the storage account.
Choose the location you want your data to be stored, make it close to you or your other Azure resources.
Choose the Azure Subscription you want the Storage Account associated with.
Choose the replication level for your storage, this will affect pricing, though different levels of replication offer greater redundancy in case of a datacentre outage.
Click Create Storage Account.

Your storage account will show as ‘Creating’ in the storage window.Wait for it to change to Online
Once it’s created you can open it up. You now have a new Azure Storage account! Click Manage Access Keys at the bottom of the page to retrieve the Storage account keys. You’ll usually only need the Storage account name and Primary Key to make use of your storage account.