OS images (classic)

OS images :

 Two types of images can be used in Azure: VM image and OS image. A VM image includes an operating system and all disks attached to a virtual machine when the image is created. This is the newer type of image. Before VM images were introduced, an image in Azure could have only a generalized operating system and no additional disks. A VM image that contains only a generalized operating system is basically the same as the original type of image, the OS image.

You can create your own images, based on a virtual machine in Azure, or a virtual machine running elsewhere that you copy and upload. If you want to use an image to create more than one virtual machine, you’ll need to prepare it for use as an image by generalizing it. To create a Windows Server image, run the Sysprep command on the server to generalize it before you upload the .vhd file. For details about Sysprep, see How to Use Sysprep: An Introduction. To create a Linux image, depending on the software distribution, you’ll need to run a set of commands that are specific to the distribution, as well as run the Azure Linux Agent.

Working with images

You can use the Azure Command-Line Interface (CLI) for Mac, Linux, and Windows or Azure PowerShell module to manage the images available to your Azure subscription. You also can use the Azure classic portal for some image tasks, but the command line gives you more options.

For information about using these tools with Resource Manager deployments, see Navigating and Selecting Azure Virtual Machine images with PowerShell and the Azure CLI.

For examples of using the tools in a classic deployment:

• For CLI, see "Commands to manage your Azure virtual machine images" in Using the Azure CLI for Mac, Linux, and Windows with Azure Service Management

• For Azure PowerShell, see the following list of commands. For an example of finding an image to create a VM, see "Step 3: Determine the ImageFamily" in Use Azure PowerShell to create and preconfigure Windows-based Virtual Machines

• Get all images:Get-AzureVMImagereturns a list of all the images available in your current subscription: your images as well as those provided by Azure or partners. This means you might get a large list. The next examples show how to get a shorter list.

• Get image families:Get-AzureVMImage | select ImageFamily gets a list of image families by showing strings ImageFamily property.

• Get all images in a specific family: Get-AzureVMImage | Where-Object {$_.ImageFamily -eq $family}

• Find VM Images: Get-AzureVMImage | where {(gm –InputObject $_ -Name DataDiskConfigurations) -ne $null} | Select -Property Label, ImageName this works by filtering the DataDiskConfiguration property, which only applies to VM Images. This example also filters the output to only the label and image name.

• Save a generalized image: Save-AzureVMImage –ServiceName "myServiceName" –Name "MyVMtoCapture" –OSState "Generalized" –ImageName "MyVmImage" –ImageLabel "This is my generalized image"

• Save a specialized image: Save-AzureVMImage –ServiceName "mySvc2" –Name "MyVMToCapture2" –ImageName "myFirstVMImageSP" –OSState "Specialized" -Verbose

[Azure.Tip] The OSState parameter is required if you want to create a VM image, which includes data disks as well as the operating system disk. If you don’t use the parameter, the cmdlet creates an OS image. The value of the parameter indicates whether the image is generalized or specialized, based on whether the operating system disk has been prepared for reuse.

• Delete an image: Remove-AzureVMImage –ImageName "MyOldVmImage"

Azure Availability Sets

 

Definition of Azure Availability Set

When you deploy new virtual machines (VMs), Azure, by default, does not possess the necessary information to identify dependencies among them. Hence, this may lead to a single point of failure in the hosted service due to an unexpected hardware fault or a maintenance operation.

An Azure Availability Set is a logical grouping capability that guarantees that the VMs included in the same group are isolated from each other. Availability Sets are based on two logical groupings: fault domains and update domains.

• VMs in the same fault domain share common power and network resources, similar to a rack at an on-premises datacenter.
• VMs belonging to the same update domain group can undergo planned and unplanned maintenance and simultaneous reboots. This configuration covers 99.95% of service-level agreements (SLAs).

When working with Availability Sets, a best practice is to create one Availability Set per workload, understanding the workload as a set of servers that work together running the same service (for example, Active Directory Domain Controllers).

Availability Sets and Availability Zones: What’s the Difference?

Depending on the SLAs required by different organizations, Azure includes different options for building and configuring high availability solutions. Availability Zones are unique physical locations within an Azure Region (set of datacenters). Each zone comprises one or more data centers equipped with independent power, cooling, and networking resources.

The main difference between Availability Zones and Availability Sets is that Availability Zones protect your resources from a potential datacenter failure. In contrast, Availability Sets only offer protection from hardware failures within a datacenter, thus increasing SLA coverage from 99.95% to 99.99%.

Steps to Set Up an Availability Set in Azure

To configure an Availability Set in Azure, two steps are required: creating the Availability Set and the assignment of VMs.

Create an Availability Set

To create an availability Set from the Azure portal, follow these steps:

1. Log on to your Azure account.

2. Click on Create a Resource, search for Availability Set, and click on the Create button.

3. Fill in the Subscription, Resource Group, Name, and Region fields according to your requirements. Next, specify the number of fault domains and update domains needed for this Availability Set. Azure automatically manages the distribution of virtual machines among the created domains, always ensuring the best possible configuration within a group. For this example, we configure two fault domains and two update domains.

4. Click on the Review + create button. Once the validation process is complete, click on the Create button.

5. Once the deployment is complete, you can explore the resource and review its features before assigning new virtual machines.

Availability Sets can also be created using Azure PowerShell.

Assign a Virtual Machine

You assign a VM to an Availability Set through the Create a virtual machine wizard. Under the Basics tab, locate the Availability options field. Select Availability set from the drop-down list, and choose the Availability Set resource needed for the VM being created. Complete the rest of the configuration fields according to your requirements.

Important things to note are that VMs can be included in an Availability Set only when deployed (existing machines cannot be assigned to an Availability Set). The Availability Set of a virtual machine can’t be changed after being created.

Read more regarding how Azure distributes virtual machines among existing fault and update domains.

Azure Kubernetes Service (AKS)

 Azure Kubernetes Service:

Microsoft Azure is a world-renown cloud platform for SMBs to large scale business, while Kubernetes is a modern-day approach that is rapidly becoming the regular methodology to manage cloud-native applications in a production environment. Azure Kubernetes Service (AKS) has brought both solutions together that allow customers to create fully-managed Kubernetes clusters quickly and easily.

AKS is an open-source fully managed container orchestration service that became available in June 2018 and is available on the Microsoft Azure public cloud that can be used to deploy, scale and manage Docker containers and container-based applications in a cluster environment.

Azure Kubernetes Service offers provisioning, scaling, and upgrades of resources as per requirement or demand without any downtime in the Kubernetes cluster and the best thing about AKS is that you don’t require deep knowledge and expertise in container orchestration to manage AKS.

AKS is certainly an ideal platform for developers to develop their modern applications using Kubernetes on the Azure architecture where Azure Container Instances are the pretty right choice to deploy containers on the public cloud. The Azure Container Instances help in reducing the stress on developers to deploy and run their applications on Kubernetes architecture.

To learn more about Kubernetes, check out Cloud Academy’s Introduction to Kubernetes. This is the second course in the Certified Kubernetes Administrator (CKA) Exam Preparation Learning Path and will teach you all about Kubernetes, including what it is and how to use it.

Azure Kubernetes Service Benefits:

Azure Kubernetes Service is currently competing with both Amazon Elastic Kubernetes Service (EKS) and Google Kubernetes Engine (GKE). It offers numerous features such as creating, managing, scaling, and monitoring Azure Kubernetes Clusters, which is attractive for users of Microsoft Azure. The following are some benefits offered by AKS:

• Efficient resource utilization: The fully managed AKS offers easy deployment and management of containerized applications with efficient resource utilization that elastically provisions additional resources without the headache of managing the Kubernetes infrastructure.
• Faster application development: Developers spent most of the time on bug-fixing. AKS reduces the debugging time while handling patching, auto-upgrades, and self-healing and simplifies the container orchestration. It definitely saves a lot of time and developers will focus on developing their apps while remaining more productive.
• Security and compliance: Cybersecurity is one of the most important aspects of modern applications and businesses. AKS integrates with Azure Active Directory (AD) and offers on-demand access to the users to greatly reduce threats and risks. AKS is also completely compliant with the standards and regulatory requirements such as System and Organization Controls (SOC), HIPAA, ISO, and PCI DSS.
• Quicker development and integration: Azure Kubernetes Service (AKS) supports auto-upgrades, monitoring, and scaling and helps in minimizing the infrastructure maintenance that leads to comparatively faster development and integration. It also supports provisioning additional compute resources in Serverless Kubernetes within seconds without worrying about managing the Kubernetes infrastructure.

Azure Kubernetes Service Features:

• Microsoft Azure offers Azure Kubernetes Service that simplifies managed Kubernetes cluster deployment in the public cloud environment and also manages health and monitoring of managed Kubernetes service. Customers can create AKS clusters using the Azure portal or Azure CLI and can manage the agent nodes.

  A template-based deployment using Terraform and Resource Manager templates can also be chosen to deploy the AKS cluster that manages the auto-configuration of master and worker nodes of the Kubernetes cluster. Some additional features such as advanced networking, monitoring, and Azure AD integration can also be configured. Let’s take a look into the features that Azure Kubernetes Service (AKS) offers:

Open-source environment with enterprise commitment:

• Microsoft has inducted the number of employees in last couple of years to make Kubernetes easier for the businesses and developers to use and participate in open-source projects and became the third giant contributor to make Kubernetes more business-oriented, cloud-native, and accessible by bringing the best practices and advanced learning with diverse customers and users to the Kubernetes community.

AZURE CLOUD SERVICE (classic)

Cloud Service (classic) :

 Azure Cloud Services is an example of a platform as a service (PaaS). Like Azure App Service, this technology is designed to support applications that are scalable, reliable, and inexpensive to operate. In the same way that App Service is hosted on virtual machines (VMs), so too is Azure Cloud Services. However, you have more control over the VMs. You can install your own software on VMs that use Azure Cloud Services, and you can access them remotely.

More control also means less ease of use. Unless you need the additional control options, it's typically quicker and easier to get a web application up and running in the Web Apps feature of App Service compared to Azure Cloud Services.

There are two types of Azure Cloud Services roles. The only difference between the two is how your role is hosted on the VMs:

• Web role: Automatically deploys and hosts your app through IIS.

• Worker role: Does not use IIS, and runs your app standalone.

For example, a simple application might use just a single web role, serving a website. A more complex application might use a web role to handle incoming requests from users, and then pass those requests on to a worker role for processing. (This communication might use Azure Service Bus or Azure Queue storage.)

As the preceding figure suggests, all the VMs in a single application run in the same cloud service. Users access the application through a single public IP address, with requests automatically load balanced across the application's VMs. The platform scales and deploys the VMs in an Azure Cloud Services application in a way that avoids a single point of hardware failure.

Even though applications run in VMs, it's important to understand that Azure Cloud Services provides PaaS, not infrastructure as a service (IaaS). Here's one way to think about it. With IaaS, such as Azure Virtual Machines, you first create and configure the environment your application runs in. Then you deploy your application into this environment. You're responsible for managing much of this world, by doing things such as deploying new patched versions of the operating system in each VM. In PaaS, by contrast, it's as if the environment already exists. All you have to do is deploy your application. Management of the platform it runs on, including deploying new versions of the operating system, is handled for you.

Azure Service fabric clusters

 

Fabric Cluster in Azure :

As an Azure Computing enthusiast, I am following the Service Fabric since the platform was available for private preview. The Service Fabric is a distributed platform that addresses significant challenges in managing cloud applications. i.e. Microservices, High-Density Web Services or self-host applications. The Azure Service Fabric avoids complex logistical problems around the infrastructure and service management. It mainly focuses on implementing critical, high-volume workload that is scalable, fault-tolerant, self-healing, stateless or stateful, fast deployable, resource balancing, self-optimising and manageable.

There are mainly two ways to provision the Service Fabric clusters,

• Azure Portal
• Azure Resource Manager Template

Create Service Fabric Cluster using the Azure Portal

Creating Service Fabric Cluster using Azure Portal is simple, though there are some tricky steps involved while setting up security and certificates. We would highlight them as we go along. Azure Portal is a useful tool especially if you are configuring for quick proof of concept or early environment. For production use (at Enterprise Scale), I would recommend Azure Resource Manager Templates.

Basic Configuration

Basic configuration would require Cluster Name, Operating System, Default VM Credentials, Subscription, Resource Group and Data Center Location. It is not a good idea to have the same username and password for all the VMs from a security perspective. However, it is fine for testing or development purposes.

Key configuration elements would be Operating System and Data Center Location.

Cluster Configuration & Node Types

Node Type configuration is one of the key decision points for your Service Fabric Cluster. Service Fabric Provisioning Orchestration would create some the VM Scale Set as equal to node types. At node type configuration, you can specify Node Type Name, Durability Tier, Machine Size, Reliability Tier and Initial VM Scale Set Capacity.

Durability Tier

The Durability Tier determines the minimum size of VM. I relate Durability term in this context as Up-Scaling your computing nodes, as well as there is an element of availability, too. Gold durability can be enabled on VM Size like D15_*, G5+ or equivalent, the similar constraint for Silver, too. The use case of the services should be the main driver for deciding durability tier.

Reliability Tier

The reliability tier configuration is more relevant to the High Availability requirement; the configuration value would run the system services with a count of target replica set. The configuration value would also determine the minimum number of nodes. However, bear in mind that there is no ceiling on numbers of VM (Azure limitation would apply).

I see them, as proper valuation as the minimal system should be in place. However, we can always scale-up or scale-out.

Node Types provides a physical segmentation within Service Fabric Cluster, you can consider separate node type for various drivers. i.e. Business Domain, Front-End Service Layer, Composite Service Layer, Core Service Layer, Back-End Service Layer, Service Profiling (High Throughput), Stateless Services (with lighter but higher more machines), Stateful (or Actor with High I/O but a smaller number of machines).

In a nutshell, it would provide much-needed flexibility to manage different enterprise services. The configuration parameters are same as Primary Node Type.

Custom Fabric Settings

You can configure runtime configuration values here, for more detail refer Customize Service Fabric cluster settings and Fabric Upgrade policy  .

Upgrade and Fabric Version

You can configure automatic Fabric Runtime upgrade or leave it for manual upgrade. A Azure Service Fabric cluster is a shared responsibility (as PaaS), you can choose a preferred update mode using Resource Manager template or the Azure Portal. For more information, please see Upgrade an Azure Service Fabric cluster  .

Security is key for any Public Cloud deployment. Service Fabric provides different configuration options i.e. Node-to-Node or Node-to-Client. You can consider some of the following,

• X.509 Certificate Security (using Azure Key Vault) Recommended
• X.509 Certificate Security (uploading .pfx directly and configuring through CD/CI pipeline to individual node)
• Windows Security (Azure Active Directory)

By click Show advanced setting link to expand other options available to the configuration. i.e. Secondary Certificate, Windows AD configuration. Secondary Certificate is necessary as it would make Key Rotation easier and straightforward. Refer add a secondary cluster certificate using the portal  for more detail.

$ResouceGroup = "blog.nilayparikh.com"
$VName = "XXXX"
$SubID = "0000000-0000-0000-0000-000000000000"
$locationRegion = "southuk"
$newCertName = "npblogdemosfcertificate"
$dnsName = "xxxxxxxx.uksouth.cloudapp.azure.com" #The certificate's subject name must match the domain used to access the Service Fabric cluster.
$localCertPath = "D:\MyCertificates" # location where you want the .PFX to be stored

Invoke-AddCertToKeyVault -SubscriptionId $SubID -ResourceGroupName $ResouceGroup -Location $locationRegion -VaultName $VName -CertificateName $newCertName -CreateSelfSignedCertificate -DnsName $dnsName -OutputPath $localCertPath

/* Output */
Name  : CertificateThumbprint
Value : 7D96DC096AXX98DCXXXXX85178AECD2AXXXX889

Name  : SourceVault
Value : /subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/blog.nilayparikh.com/providers/Microsoft.KeyVault/vaults/XXXX

Name  : CertificateURL
Value : https://XXXX.vault.azure.net:443/secrets/npblogdemosfcertificate/0000000000000000000000000000000

If you are considering deploying X.509 certificate through the Azure Key Vault then you need to tick Enable access to Azure Virtual Machine for deployment option on Advance Access Policies.

Service Fabric Advance Access Policies

Quick hack for creating self-signed certificate for non-production use,

Review and Create Service Fabric Cluster

Review the summary and click on Create, it may take long to provision a cluster. The Azure Portal will provision following,

• Load Balancer (per Node Type)
• Subnet (per Node Type)
• Public IP
• Virtual Network (optional with Resource Manager Template)
• Virtual Scale Sets (per Node Type)
• Virtual Machines (per Durability, Reliability and Cluster Configuration)
• Storage (per configuration, i.e. Logs)

That is it; it could take several minutes while Microsoft Azure provision you all the artefact that make Service Fabric Cluster.

Azure Batch account

 

Create a Batch account

Follow these steps to create a sample Batch account for test purposes. You need a Batch account to create pools and jobs. You can also link an Azure storage account with the Batch account. Although not required for this quickstart, the storage account is useful to deploy applications and store input and output data for most real-world workloads.

1. In the Azure portal, select Create a resource.

2. Type "batch service" in the search box, then select Batch Service.

   

3. Select Create.

4. In the Resource group field, select Create new and enter a name for your resource group.

5. Enter a value for Account name. This name must be unique within the Azure Location selected. It can contain only lowercase letters and numbers, and it must be between 3-24 characters.

6. Optionally, under Storage account, you can specify a storage account. Click Select a storage account, then select an existing storage account or create a new one.

7. Leave the other settings as is. Select Review + create, then select Create to create the Batch account.

When the Deployment succeeded message appears, go to the Batch account that you created.

Create a pool of compute nodes

Now that you have a Batch account, create a sample pool of Windows compute nodes for test purposes. The pool in this quickstart consists of two nodes running a Windows Server 2019 image from the Azure Marketplace.

1. In the Batch account, select Pools > Add.

2. Enter a Pool ID called mypool.

3. In Operating System, use the following settings (you can explore other options).

   Setting	Value
   Image Type	Marketplace
   Publisher	microsoftwindowsserver
   Offer	windowsserver
   Sku	2019-datacenter-core-smalldisk

4. Scroll down to enter Node Size and Scale settings. The suggested node size offers a good balance of performance versus cost for this quick example.

   Setting	Value
   Node pricing tier	Standard_A1_v2
   Target dedicated nodes	2

5. Keep the defaults for remaining settings, and select OK to create the pool.

Batch creates the pool immediately, but it takes a few minutes to allocate and start the compute nodes. During this time, the pool's Allocation state is Resizing. You can go ahead and create a job and tasks while the pool is resizing.

After a few minutes, the allocation state changes to Steady, and the nodes start. To check the state of the nodes, select the pool and then select Nodes. When a node's state is Idle, it is ready to run tasks.

Create a job

Now that you have a pool, create a job to run on it. A Batch job is a logical group of one or more tasks. A job includes settings common to the tasks, such as priority and the pool to run tasks on. The job won't have tasks until you create them.

1. In the Batch account view, select Jobs > Add.

2. Enter a Job ID called myjob.

3. In Pool, select mypool.

4. Keep the defaults for the remaining settings, and select OK.

Create tasks

Now, select the job to open the Tasks page. This is where you'll create sample tasks to run in the job. Typically, you create multiple tasks that Batch queues and distributes to run on the compute nodes. In this example, you create two identical tasks. Each task runs a command line to display the Batch environment variables on a compute node, and then waits 90 seconds.

When you use Batch, the command line is where you specify your app or script. Batch provides several ways to deploy apps and scripts to compute nodes.

To create the first task:

1. Select Add.

2. Enter a Task ID called mytask.

3. In Command line, enter cmd /c "set AZ_BATCH & timeout /t 90 > NUL". Keep the defaults for the remaining settings, and select Submit.

Repeat the steps above to create a second task. Enter a different Task ID such as mytask2, but use the same command line.

After you create a task, Batch queues it to run on the pool. When a node is available to run it, the task runs. In our example, if the first task is still running on one node, Batch will start the second task on the other node in the pool.

View task output

The example tasks you created will complete in a couple of minutes. To view the output of a completed task, select the task, then select the file stdout.txt to view the standard output of the task. The contents are similar to the following example:

The contents show the Azure Batch environment variables that are set on the node. When you create your own Batch jobs and tasks, you can reference these environment variables in task command lines, and in the apps and scripts run by the command lines.

Clean up resources

If you want to continue with Batch tutorials and samples, you can keep using the Batch account and linked storage account created in this quickstart. There is no charge for the Batch account itself.

You are charged for the pool while the nodes are running, even if no jobs are scheduled. When you no longer need the pool, delete it. In the account view, select Pools and the name of the pool. Then select Delete. After you delete the pool, all task output on the nodes is deleted.

When no longer needed, delete the resource group, Batch account, and all related resources. To do so, select the resource group for the Batch account and select Delete resource group.