Deploy an Azure Storage Mover agent

• 
  

The Azure Storage Mover service utilizes agents to perform the migration jobs you configure in the service. An agent is a virtual machine-based migration appliance that runs on a virtualization host. Ideally, your virtualization host is located as near as possible to the source storage to be migrated. Storage Mover can support multiple agents.

Because an agent is essentially a migration appliance, you interact with it through an agent-local administrative shell. The shell limits the operations you can perform on this machine, though network configuration and troubleshooting tasks are accessible.

Use of the agent in migrations is managed through Azure. Both Azure PowerShell and CLI are supported, and graphical interaction is available within the Azure portal. The agent is made available as a disk image compatible with either new Windows Hyper-V or VMware virtual machines (VMs).

This article guides you through the steps necessary to successfully deploy a Storage Mover agent VM.

Prerequisites

• A capable Windows Hyper-V or VMware host on which to run the agent VM.
  See the Recommended compute and memory resources section in this article for details about resource requirements for the agent VM.

 Note

At present, Windows Hyper-V and VMware are the only supported virtualization environments for your agent VM. Other virtualization environments have not been tested and are not supported.

Determine required resources for the VM

Like every VM, the agent requires available compute, memory, network, and storage space resources on the host. Although overall data size might affect the time required to complete a migration, it's generally the number of files and folders that drives resource requirements.

Network resources

The agent requires unrestricted internet connectivity.

Although no single network configuration option works for every environment, the simplest configuration involves the deployment of an external virtual switch. The external switch type is connected to a physical adapter and allows your host operating system (OS) to share its connection with all your virtual machines (VMs). This switch allows communication between your physical network, the management operating system, and the virtual adapters on your virtual machines. This approach might be acceptable for a test environment, but is likely not sufficient for a production server.

After the switch is created, ensure that both the management and agent VMs are on the same switch. On the WAN link firewall, outbound TCP port 443 must be open. Keep in mind that connectivity interruptions are to be expected when changing network configurations.

You can get help with creating a virtual switch for Hyper-V virtual machines in the Windows Server documentation. Consult the VMware support website for detailed guidance on creating a virtual switch for VMware-hosted VMs.

Recommended compute and memory resources

Migration scale*	Memory (RAM)	Virtual processor count cores (at 2 GHz min.)
1 million items	8 GiB	4 virtual cores
10 million items	8 GiB	4 virtual cores
30 million items	12 GiB	6 virtual cores
50 million items	16 GiB	8 virtual cores
100 million items	16 GiB	8 virtual cores

Number of items refers to the total number of files and folders in the source.

 Important

While agent VMs below minimal specs may work for your migration, they may not perform optimally and are not supported.

The Performance targets article contains test results from different source namespaces and VM resources.

Local storage capacity

At a minimum, the agent image needs 20 GiB of local storage. The amount required might increase if a large number of small files are cached during a migration.

Download the agent VM image

Images for agent VMs are hosted on Microsoft Download Center as a zip file. Download the file at https://aka.ms/StorageMover/agent and extract the agent virtual hard disk (VHD) image to your virtualization host.

Create the agent VM

The following steps describe the process for creating a VM using Microsoft Hyper-V. Consult the VMware support website for detailed guidance on creating a VMware-based VM.

1. Create a new VM to host the agent. Open Hyper-V Manager. In the Actions pane, select New and Virtual Machine... to launch the New Virtual Machine Wizard.

   

2. Within the Specify Name and Location pane, specify values for the agent VM's Name and Location fields. The location should match the folder where the VHD is stored, if possible. Select Next.

   

3. Within the Specify Generation pane, select the Generation 1 option.

   

    Important

   Only Generation 1 VMs are supported. This Linux image won't boot as a Generation 2 VM.

4. If you haven't already, determine the amount of memory you need for your VM. Enter this amount in the Assign Memory pane, noting that you need to enter the value in MiB. 1 GiB = 1024 MiB. Using the Dynamic Memory feature is fine.

   

5. Within the Configure Networking pane, select the Connection drop-down. From the list, choose the virtual switch that provides the agent with internet connectivity and select Next. For more information, see the Hyper-V virtual networking documentation for details.

   

6. Within the Connect Virtual Hard Disk pane, select the Use an existing Virtual Hard Disk option. In the Location field, select Browse and navigate to the VHD file that was extracted in the previous steps. Select Next.

   

7. Within the Summary pane, select Finish to create the agent VM.

   

8. After the new agent is successfully created, it will appear in the Virtual Machines pane within the Hyper-V Manager.

   

Change the default password

The agent is delivered with a default user account and password. Connect to the newly created agent and change the default password immediately after the agent is deployed and started.

From a machine on the same subnet as the agent, run an ssh command:

PowerShellssh <AgentIpAddress> -l admin

 Important

A newly deployed Storage Mover agent has a default password:
Local user: admin
Default password: admin

You're prompted and advised to change the default password immediately after you first connect to a newly deployed agent. Note down the new password, there's no process to recover it. Losing your password locks you out from the administrative shell. Cloud management doesn't require this local admin password. If the agent was previously registered, you can still use it for migration jobs. Agents are disposable. They hold little value beyond the current migration job they're executing. You can always deploy a new agent and use that instead to run the next migration job.

Bandwidth throttling

Take time to consider the amount of bandwidth a new machine uses before you deploy it to your network. An Azure Storage Mover agent communicates with a source share using the local network, and the Azure Storage service on the wide area network (WAN) link. In both cases, the agent uses all available network bandwidth.

 Important

The current Azure Storage Mover agent does not support bandwidth throttling schedules.

If bandwidth throttling is important to you, create a local virtual network with an internet connection and configure quality of service (QoS) settings. This approach allows you to expose the agent through the virtual network and to locally configure an unauthenticated network proxy server on the agent if needed.

Decommissioning an agent

When you no longer need a specific storage mover agent, you can decommission it. Decommissioning is a two-step process:

1. Unregister the agent from the storage mover resource.
2. Stop the agent VM on your virtualization host and then delete it.

Decommissioning an agent starts with unregistering the agent. There are three options to start the unregistration process:

• Agent administrative shell
• Azure portal
• PowerShell

You can unregister an agent using the administrative shell of the agent VM. The agent must be connected to the service and showing online both locally and through Azure portal and either Azure PowerShell or Azure CLI.

From a machine on the same subnet as the agent, run an ssh command:

PowerShellssh <AgentIpAddress> -l admin

 Important

A newly deployed Storage Mover agent has a default password:
Local user: admin
Default password: admin

You're prompted and advised to change the default password immediately after you first connect to a newly deployed agent. Note down the new password, there's no process to recover it. Losing your password locks you out from the administrative shell. Cloud management doesn't require this local admin password. If the agent was previously registered, you can still use it for migration jobs. Agents are disposable. They hold little value beyond the current migration job they're executing. You can always deploy a new agent and use that instead to run the next migration job.

StorageMoverAgent-AdministrativeShell1) System configuration

2) Network configuration
3) Service and job status
4) Unregister
5) Collect support bundle
6) Restart agent
7) Disk Cleanup
8) Exit

xdmsh> 4

Select the option 4) Unregister. You're prompted for confirmation.

 

Azure Data Box — Physical devices for data upload

→ Normally when we upload data to storage for small to medium-sized data uploading using the Internet is fine (say around 100 TB data).

→ If we want to upload large-size data say 500 TBs of data. Uploading data will take time maybe a week or two. This might affect business.

→ Microsoft offers physical devices for large data for faster upload.

→ Those who are from the Amazon cloud platform. This feature is similar to AWS Snow Family Service.

→ These are actual physical devices that are similar to hard disks that will ship to you when we order them.

→ We unpack them, load these data onto these devices and we ship them back to Azure, they will copy the data and put it into our storage account.

Type 1: Data Box — Somewhat size of a CPU with 100 TB storage

Type 2: Data Box Disk — Size of an SSD with 8 TB

Type 3: Data Box Heavy — Heavy size with 1 PB

→ Depending on our needs we can order these devices load the data and ship it back.

→ So once we courier these data centres to Azure, it will reflect in our Azure storage account.

→ There are also Edge Devices. Azure stack has Edge devices we can install in our data centre which is a Microsoft piece of hardware. It helps us do cloud stuff in our own data centre.

→ There is also a Virtual Data Box, which works like a data machine that has some agent software on it.

→ Let us see this service from Azure Cloud:

→ So basically we can import data from the cloud to us or export data to Azure.

→ The reason why we were using this was that online upload of large data was a problem, we can order them physically and make data upload and ship it back.

→ Microsoft also gives security guarantees, when data is uploaded it will be clean from devices.