Microsoft Azure for Beginners : Moving Your Data to Azure Storage? Let Azure Storage Mover Do the Heavy Lifting – Part 16

Azure Storage Mover: A managed migration service for Azure Storage.

In a world increasingly dependent on digital data, it is essential to store this data securely and reliably. Azure Storage provides a powerful and versatile storage platform for this purpose. However, migrating data to Azure Storage can be challenging, especially when dealing with large amounts of data and complex migrations. Fortunately, Azure Storage Mover offers a managed migration service that helps businesses migrate data to Azure Storage easily and quickly.

What is Azure Storage Mover?

Azure Storage Mover is a managed migration service for Azure Storage that helps businesses migrate data to Azure Storage easily and quickly. With Azure Storage Mover, you can migrate data from local, network, and other cloud storage providers to Azure Storage.

How does Azure Storage Mover work?

Azure Storage Mover is easy to use and requires no specific technical knowledge. Here are the steps to migrate data to Azure Storage with Azure Storage Mover:

• Create a new Azure Storage account: Go to the Azure Portal and create a new Azure Storage account
• Create a migration project: Go to the Azure Storage Mover service and create a new migration project. Enter the required information, such as the project name, source, and target storage
• Set migration settings: Configure the migration settings, such as the transfer rate and the number of parallel transfers

• Start Migration: Click “Start Migration,” and Azure Storage Mover will automatically migrate the data to Azure Storage

Benefits of Azure Storage Mover

Azure Storage Mover offers many benefits for companies looking to migrate data to Azure Storage, including:

• Easy to use: Azure Storage Mover is easy to use and requires no specific technical knowledge
• Fast migration: Azure Storage Mover uses advanced transfer algorithms and parallel transfers to migrate data to Azure Storage quickly
• Managed migration: Azure Storage Mover provides a managed migration experience that helps businesses migrate data quickly and easily
• Data protection: Azure Storage Mover ensures data protection during migration using advanced security features and encryption
• Flexibility: Azure Storage Mover supports migrations from various sources to Azure Storage, including local storage, network storage, and other cloud storage providers

There are several situations in which companies could use Azure Storage Mover to migrate data to Azure Storage. Here are some examples:

• Migrating from local storage to Azure Storage: If you store data on a local server and want to migrate it to Azure Storage, you can use Azure Storage Mover to migrate the data quickly and easily.
• Migrating from network storage to Azure Storage: If you store data on network storage and want to migrate it to Azure Storage, you can use Azure Storage Mover to migrate the data quickly and easily.
• Migrating cloud storage to Azure Storage: If you store data with another cloud storage provider and want to migrate it to Azure Storage, you can use Azure Storage Mover to migrate the data quickly and easily.

Cost of Azure Storage Mover

The cost of Azure Storage Mover depends on several factors, such as the amount of data you are migrating, the source, and the target storage. In general, the cost of Azure Storage Mover is affordable, and you can easily view and manage it through the Azure Portal. You should check the Azure pricing page for more information on the cost of Azure Storage Mover.

Roadmap of Azure Storage Mover

Microsoft is continuously improving Azure Storage Mover and offering new features and enhancements. Here is an overview of current and planned features for Azure Storage Mover:

• Support for migrations from multiple sources: Azure Storage Mover currently supports migrations from a single source to a single target storage. In the future, it will support migrations from multiple sources to a single target storage
• Support for migrations to different target stores: Azure Storage Mover currently supports migrations to a single target storage. In the future, it will support migrations to multiple target stores
• Migration speed improvements: Azure Storage Mover uses advanced transfer algorithms and parallel transfers to quickly migrate data to Azure Storage. Further improvements in migration speed will be made in the future
• Security improvements: Azure Storage Mover currently uses advanced security features and encryption to protect data during migration. Further security enhancements will be made in the future

 

---

Testing out Azure Storage Mover

I recently noticed that Azure Storage Mover is now Generally Available. This is a service I’ve not used before, but migrations of Storage to Azure is something I am often involved with, so I was keen to test out this service, and explore the setup and usage with a test migration.  In this post I will run through an overview of Storage Mover, how it is setup, and also run through a simple test migration.

---

What is Azure Storage Mover?

Azure Storage Mover is a service that enables the migration of files and folders to Azure Storage, whilst minimizing downtime for your workload. Storage Mover can be used for a variety of migration requirements and scenarios.

Azure Storage Mover is a Hybrid Cloud Service – so it comprises both a Cloud and Infrastructure component, and to use it we need to deploy both components.

---

What are the Supported Sources and Targets for a Move?

At the current time Azure Storage Mover enables the migration of NFS shares to an Azure Blob Container. For an overview of the supported sources and targets, see here: https://learn.microsoft.com/en-gb/azure/storage-mover/service-overview#supported-sources-and-targets.

---

Setup & Testing

To test out the Service, I will be using a simple setup, comprising of:

• An Azure Subscription

• AN Azure Storage Account with a Blob Container

• An NFS Share hosted on a Windows Virtual Machine (within my Lab)

• A Storage Mover Agent deployed within my Lab

To start testing – I will first browse to my NFS share, and show that there is some data within:

NFS Share – showing data within.

I also have an Azure Storage Account with a Blob Container setup – this is currently empty, and will be the target destination I use for the data above:

Storage Account setup with a Container called “migration01”.

Creating the Storage Mover

Next we need to deploy the Azure Storage Mover Resource – this is done from within the Azure Portal:

Storage Mover setup – using search

Click on “Create storage mover”:

Storage Mover setup – Creation

We can then fill in a few basic details to begin:

Storage Mover setup – Basic details

There is also an option to add Monitoring to Storage Mover – which enables metrics and logs to be collected:

Storage Mover setup – Log Analytics

Once this is done we can move onto the “Review + create” pane, and then create the Resource. Once created, we can see our Storage Mover:

Storage Mover setup – Created Resource, ready for our agent.

We can now move onto the Agent Deployment. The agent is how our Storage Mover Resource will be able to community with our on-premises storage shares and devices.

Agent Deployment

Storage Mover makes the Agent deployment a simple process – from the Storage Mover overview, click on “Register agent”, as shown below:

Registering our Storage Mover Agent

You’ll then be prompted to download the Agent:

Registering our Storage Mover Agent

The link shown above will take you to a Microsoft download page, that allows you to download the VHDX for the Agent VM. Once this has downloaded we need to deploy this within our environment. My Lab environment is using Hyper-V, so the process is simple. If you need guidance on deployment of the VHDX to Hyper-V, please see this guide here: https://learn.microsoft.com/en-gb/azure/storage-mover/agent-deploy?tabs=xdmshell.

At this point, it is also worth noting that the Storage Mover agent VM requires outbound unrestricted internet connectivity. There is also a table of CPU Cores and RAM recommendations that needs consideration, depending upon the scale of migration: https://learn.microsoft.com/en-gb/azure/storage-mover/agent-deploy?tabs=xdmshell#recommended-compute-and-memory-resources. A minimum of 20 GB of disk space is also required – with potentially more space for a large migration.

Once the VM is deployed, and has been booted, we can connect via SSH, using the default credentials of admin/admin, and we are prompted to update the password. Note, you can use Hyper-V to check the IP you need to connect to!

Connecting in via PuTTY

We can now start the process of Registering the agent. To do this, select “4” from the menu, which starts the Registration process. Once you select this option and press enter, you will be prompted to enter the following information:

• Tenant ID

• Subscription ID

• Resource Group Name

• Storage Mover Name

• Whether Private Link is required

Configuration via PuTTY

Once completed – you will see the following:

Agent configuration completed!

Our Agent is now Registered:

Agent online and ready for use!

We can now setup our Storage Target within Storage Mover, before we move onto the Migration.

Adding our Storage Endpoints

We can now add our Storage Endpoints to Storage Mover, so that we have our source and destination endpoints configured. We will start with our target endpoint – to do this, visit your Storage Mover in the Portal, and click on “Storage endpoints”, then “Create endpoint”, and select “Create target endpoint”:

Creating a Target Endpoint

We are then prompted to fill in some details – as you can see I have selected my Subscription, Storage Account, and Container, and provided a description:

Target Endpoint details

Once these details are added, click on “Create”. After a few moments, our endpoint will then be added:

Our Target Endpoint is added and ready for use

Our Target endpoint is now added, and we can now move on to adding our Source endpoint. To do this, click on the “Create endpoint” drop down and select “Create source endpoint”:

Adding a Source Endpoint

This will bring up a window for us to add the details of our source NFS Share:

Adding our Source NFS Share details

Once the details are added, we can click on “Create”, and our Source will be added:

Source endpoint added and ready to use

We can now move onto the Migration!

Migration

Once our Agents, and Source and Target endpoints are setup, Migration is a straightforward process using Storage Mover. We next need to create a Project, and then a Job within this project – which we can do from the main Storage Mover window. To get started click on “create project”:

Create a Storage Mover project

This will bring up the Project explorer window, where we need to click on “create project” again to start. We can then fill in basic details and click on “create”:

Creating our migration project

We can now create a job definition within this project:

Creating a Job definition

We can now fill in the job definition details – there are a few screens for this, which are shown below:

Step 1 – Creating our migration job

Step 2 – Selecting endpoint source

Step 3 – Selecting target endpoint

Step 4 – Migration settings

Step 5 – Review

Our job will now be setup – and we can review this from the Project explorer. Once we are happy with the details shown, we can click on “Start job”:

Start migration job

The job will then show as “Queued”, whilst the agent picks up the job:

Migration job queued

Once the operation is completed, you will see the overview screen that details how the job went. Note I added a few more files to my share to increase the number of files/objects since I took the “source” screenshot at the start of this post.

Migration job completed

We can now browse to our Storage Account and confirm that files are showing as expected. I can see the test files I added below:

Confirming Test migration files can be seen

This concludes my test migration. Obviously in a production scenario it’s likely to be significantly more data or shares, however this post hopefully demonstrates the simplicity with which migration can be done using Storage Mover – and in particular, how it can be used to manage a number of migration jobs across different sources and targets.

 

Azure Storage Mover–A managed migration service for Azure Storage

Share Azure Storage Mover–A managed migration service for Azure Storage on FacebookShare Azure Storage Mover–A managed migration service for Azure Storage on XShare Azure Storage Mover–A managed migration service for Azure Storage on LinkedIn

File storage is a critical part of any organization’s on-premises IT infrastructure. As organizations migrate more of their applications and user shares to the cloud, they often face challenges in migrating the associated file data. Having the right tools and services is essential to successful migrations.

Across workloads, there can be a wide range of file sizes, counts, types, and access patterns. In addition to supporting a variety of file data, migration services must minimize downtime, especially on mission-critical file shares.

In February of 2022, we launched the Azure file migration program that provides no-cost migration to our customers, via a choice of storage migration partners.

We added another choice for file migration with the general availability of Azure Storage Mover, which is a fully managed, hybrid migration service that makes migrating files and folders into Azure a breeze.

The key capabilities of Azure Storage Mover are:

Cloud-driven migrations

Managing copy jobs at scale without a coordinating service can be time consuming and error-prone. Individual jobs have to be monitored and any errors resolved. It’s hard to maintain comprehensive oversight to ensure a complete and successful migration of your data.

With Azure Storage Mover you can express your migration plan in Azure and when you are ready, conveniently start and track migrations right from the Azure portal, PowerShell, or CLI. This allows you to utilize Azure Storage Mover for a one-time migration project or for any repeated data replication needs.

Azure Storage Mover is a hybrid service with migration agents that you’ll deploy close to your source storage. All agents can be managed from the same place in Azure, even if they are deployed across the globe.

Scale and performance

Many aspects contribute to a high-performance migration service. Fast data movement through the Azure Storage REST protocol and a clear separation of the management path from the data path are among the most important. Each agent will send your files and folders directly to the target storage in Azure.

Directly sending the data to the target optimizes the performance of your migration because the data doesn’t need to be processed through a cloud service or through a different Azure region from where the target storage is deployed in. For example, this optimization is key for migrations that happen across geographically diverse branch offices that will likely target Azure Storage in their region.

What’s next for Storage Mover?

There are many steps in a cloud migration that need to happen before the first byte can be copied. A deep understanding of your data estate is essential to a balanced cloud solution design for your workloads.

When we combine that with a strategy to minimize downtime, and manage and monitor migration jobs at scale, then we’ve arrived at our vision for the Storage Mover service. This roadmap for this vision includes:

• Support for more sources and Azure Storage targets.
• More options to tailor a migration to your needs.
• Automatically loading possible sources into the service. That’s more than just convenience; it enables large-scale migrations and reduces mistakes from manual input.
• Deep insights about selected sources for a sound cloud solution design.
• Provisioning target storage automatically based on your migration plan.
• Running post-migration tasks such as data validation, enabling data protection, and completing migration of the rest of the workload, etc.

workloads, there can be a wide range of file sizes, counts, types, and access patterns. In addition to supporting a variety of file data, migration services must minimize downtime, especially on mission-critical file shares.

In February of 2022, we launched the Azure file migration program that provides no-cost migration to our customers, via a choice of storage migration partners.

Today, we are adding another choice for file migration with the preview launch of Azure Storage Mover, which is a fully managed, hybrid migration service that makes migrating files and folders into Azure a breeze.

The key capabilities of the Azure Storage Mover preview are:

NFS share to Azure blob container

With this preview release, we focus on the migration of an on-premises network file system (NFS) share to an Azure blob container. Storage Mover will support many additional source and target combinations over the coming months.

Cloud-driven migrations

Managing copy jobs at scale without a coordinating service can be time consuming and error-prone. Individual jobs have to be monitored and any errors resolved. It’s hard to maintain comprehensive oversight to ensure a complete and successful migration of your data.

With Azure Storage Mover you can express your migration plan in Azure and when you are ready, conveniently start and track migrations right from the Azure portal, PowerShell, or CLI. This allows you to utilize Azure Storage Mover for a one-time migration project or for any repeated data movement needs.

Azure Storage Mover is a hybrid service with migration agents that you’ll deploy close to your source storage. All agents can be managed from the same place in Azure, even if they are deployed across the globe.

Scale and performance

Many aspects contribute to a high-performance migration service. Fast data movement through the Azure Storage REST protocol and a clear separation of the management path from the data path are among the most important. Each agent will send your files and folders directly to the target storage in Azure.

Directly sending the data to the target optimizes the performance of your migration because the data doesn’t need to be processed through a cloud service or through a different Azure region from where the target storage is deployed in. For example, this optimization is key for migrations that happen across geographically diverse branch offices that will likely target Azure Storage in their region.

What’s next for Storage Mover?

There are many steps in a cloud migration that need to happen before the first byte can be copied. A deep understanding of your data estate is essential to a balanced cloud solution design for your workloads.

When we combine that with a strategy to minimize downtime, and manage and monitor migration jobs at scale, then we’ve arrived at our vision for the Storage Mover service. This roadmap for this vision includes:

• Support for more sources and Azure Storage targets.
• More options to tailor a migration to your needs.
• Automatically loading possible sources into the service. That’s more than just convenience; it enables large-scale migrations and reduces mistakes from manual input.
• Deep insights about selected sources for a sound cloud solution design.
• Provisioning target storage automatically based on your migration plan.
• Running post-migration tasks such as data validation, enabling data protection, and completing migration of the rest of the workload, etc.