Benefits of using Azure OpenShift

 

Benefits of using Azure OpenShift

Red Hat and Microsoft jointly engineer, operate, and support Azure Red Hat OpenShift. The table below summarizes the key benefits of Azure Red Hat OpenShift over OpenShift as IaaS. Azure OpenShift Benefits

Benefit	Description
Security	Enterprise-grade operations, security, and compliance. With SLA of 99.95% availability and PCI DSS, ISO 27001, HITRUST, SOC 2 Type II, and FedRAMP certifications.
Cloud-native Integrations	Promotes developer productivity with built-in CI/CD pipelines and effortlessly connects applications to hundreds of Azure services such as MySQL, PostgreSQL, Redis, Cosmos DB, etc.
Quick Startup	Start a highly available cluster quickly and scale your application demand changes.
Flexible Instance Types	Choice between standard, memory-optimized, and CPU-optimized application nodes.
Convenient Billing	Pay through an already configured Azure subscription.
Vertical Integration	A vertically integrated product with a core OS, so any updates or security vulnerabilities are addressed sooner and faster.

Azure OpenShift Cost

Azure OpenShift 4 has a minimum cluster size of three master nodes and three worker nodes. Both node types use Linux Azure VM pricing. Additionally, worker nodes have OpenShift license costs as well. With Azure OpenShift, these VM sizes are billed as part of the standard Azure subscription. Both master and worker nodes can use Azure's reduced on-demand and reserved instance pricing. For a complete list of supported VM sizes, see Azure Red Hat OpenShift pricing

Azure OpenShift Deployment Options

There are two ways to run OpenShift on Azure:

• OpenShift Container Platform (OCP) deployed on a virtual machine
• Azure Red Hat OpenShift deployed through Azure portal or Azure CLI.

OpenShift running on an Azure VM

OpenShift Container Platform on virtual machines is deployed through cloud.redhat.com/openshift. With this deployment model, OpenShift must be installed and set up on a VM, and you must bring your own license.  There are no Azure-specific operations, integrations, or billing benefits in this case. With this option, using OpenShift on Azure is the same as running it on your own hardware or another infrastructure as a service (IaaS) platform like GCP or AWS. 

Azure OpenShift 

With this deployment model, you can deploy Azure Red Hat OpenShift through the Azure portal or the Azure CLI. It is fully managed and supported by both Red Hat and Azure. Since this is part of the Azure service stack, the billing integrates into your Azure subscription, including a license.

How to Deploy Azure OpenShift 

Now, let’s walk through how you can deploy Azure Red Hat OpenShift using the Azure portal and Azure CLI. 

Machine learning for Kubernetes sizing

LEARN MORE

	Visualize Utilization Metrics	Set Resource Requests & Limits	Set Requests & Limits with Machine Learning	Identify mis-sized containers at a glance & automate resizing	Get Optimal Node Configuration Recommendations
Kubernetes
Kubernetes + Densify

Prerequisites

Before we get started, you’ll need to:

• Ensure Azure Red Hat is available in your region. 
• Make sure you have a pay-as-you-go Azure account, not a free entry-level account. The free version does not cover third-party license costs.
• Have enough Azure quota for the standard dsv3 family of vCPUs. 

Note that the initial quota allowed is ten vCPUs. You’ll need at least 40. Use the normal Azure process to request a large quota by going to the “Usage + quotas” option available under the Azure portal and requesting an additional quota for the instance type you will use to deploy the Azure OpenShift cluster.  You can find a list of supported Quotas and other requirements for Azure OpenShift here. 

Begin the process in the Red Hat OpenShift management console

1. First, log in to your Red Hat OpenShift console using your Red Hat account at https://console.redhat.com/. Once logged in, you will find many options to interact with Red Hat software. The RedHat OpenShift console displays a variety of submenus.

   

   The RedHat OpenShift console displays a variety of submenus.

2. Select Red Hat OpenShift Cluster Manager.
3. Then click the "Create Cluster"  which will display many different options to create a new cluster.
4. Click “Try it on Azure” for the Azure Red Hat OpenShift offering. You will see detailed instructions for quickly deploying a production-grade OpenShift cluster standing up on Azure. We’ll follow these instructions here.
5. Get your pull secret from the Red Hat management console for Azure to begin deployment. It is located under “Run it yourself” below the Managed services offering.

   

   The Red Hat management console displays several options for deploying OpenShift on different IaaS platforms.

6. Choose the “Installer-provisioned infrastructure” option.

   

   The Red Hat management console provides two options for Azure OpenShift

7. Click “Download pull secret” to download the pull secret. This step will save the pull secret to your local machine.

   

   The Red Hat management console allows users to download their pull secret.

Prep Azure for installation

Next, we’ll prep Azure for installation. We can also do this through the Azure portal, but for this article, we’ll use the Azure CLI as it makes deployment easier. 

1. First, we need to register the Red Hat OpenShift resource on our subscription along with Microsoft’s compute, storage, and authorization resource providers. Run the following commands to register required resources:

   
       az provider register -n Microsoft.RedHatOpenShift –wait 
       az provider register -n Microsoft.Compute –wait
       az provider register -n Microsoft.Storage –wait
       az provider register -n Microsoft.Authorization –wait                            
                               

2. Next, run these commands to configure environment variables:

   
       LOCATION=eastus #location of your cluster
       RESOURCEGROUP=arorg #name of the resource group where you want to create your cluster
       CLUSTER=cluster #name of your cluster                                                      
                               

3. Now, we’ll create a resource group with a location. Use these commands to create the group:

   
       az group create 
       --name $RG
       --location $LOCATION                                                                              
                               

   You should see output similar to this:

   
       {
       "id": "/subscriptions/<guid>/resourceGroups/aro-rg",
       "location": "eastus",
       "name": "aro-rg",
       "properties": {
         "provisioningState": "Succeeded"
       },
       "type": "Microsoft.Resources/resourceGroups"
       }                                                                                                
                               

   Note that in Azure, a resource group is a logical group in which Azure resources are deployed and managed. We have to set a location for this. This location is where the resource group metadata is stored and sets the default location for the resources created in this resource group. The resource created in a resource group is not limited to the location of the resource group.
4. Next, we’ll configure networking for our cluster. We will create a virtual network that will have two empty subnets. The first subnet will be for the control pane, and the other for the workers. We will build all this networking inside the resource group we created earlier. Run these commands to configure the cluster networking:

   
       az network vnet create 
       --resource-group $RG 
       --name arovnet 
       --address-prefixes 20.0.0.0/22                                                                                                                       
                               

   You should see output similar to:

   
       {
       "newVNet": {
         "addressSpace": {
           "addressPrefixes": [
             "20.0.0.0/22"
           ]
         },
         "dhcpOptions": {
           "dnsServers": []
         },
         "id": "/subscriptions/<guid>/resourceGroups/aro-rg/providers/Microsoft.Network/virtualNetworks/aro-vnet",
         "location": "eastus",
         "name": "arovnet",
         "provisioningState": "Succeeded",
         "resourceGroup": "arorg",
         "type": "Microsoft.Network/virtualNetworks"
       }                                                                                                                                             
                               

5. After creating the virtual network, we need to create the two subnets we discussed earlier. We’ll also explicitly set a service endpoint, and that’s because Azure guarantees a secure and direct route when we do that. Use this command to create the master subnet:

   
       az network vnet subnet create 
         --resource-group $RG
         --vnet-name arovnet 
         --name master-subnet 
         --address-prefixes 20.0.0.0/23 
         --service-endpoints Microsoft.ContainerRegistry                                                                                                                                               
                               

6. Next, use the following command to create the worker subnet:

   
       az network vnet subnet create 
         --resource-group $RG 
         --vnet-name arovnet 
         --name workersubnet 
         --address-prefixes 20.0.2.0/23 
         --service-endpoints Microsoft.ContainerRegistry                                                                                                                                                                       
                               

7. Now, disable the subnet private endpoint policies for that master subnet. It is required for the service to be able to connect to and manage the cluster, and if we’re using the portal to do it, it would be the default. These are the CLI commands required.

   
       az network vnet subnet update 
         --name mastersubnet 
         --resource-group $RG 
         --vnet-name arovnet 
         --disable-private-link-service-network-policies true                                                                                                                                                                                               
                               

8. Create a cluster using the aro create command. You can run this command to see the options available for the aro create command.

   aro create –help | more                                                                                                                                                                                             
                               

   There are plenty of options available, including the ability to set the VM sizes, set up advanced network configuration, worker sizing, etc., to name a few. To create the cluster, run the commands below. It references the pull secret we downloaded earlier through the Red Hat management console.

   
       az network vnet subnet update 
         --name mastersubnet 
         --resource-group $RG 
         --vnet-name arovnet 
         --disable-private-link-service-network-policies true                                                                                                                                                                                                                
                               

   
       az aro create 
         --resource-group $RESOURCEGROUP 
         --name $CLUSTER 
         --vnet arovnet 
         --master-subnet mastersubnet 
         --worker-subnet workersubnet
         --pull-secret @pull-secret.txt                                                                                                                                                                                                                                    
                               

   Installation typically takes around 30-35 minutes, depending on your region. You can watch the deployment progress on the Azure portal or through the debug command in CLI. When the installation completes, you’ll see the following list of resources deployed on your Azure, as shown in the screenshot below.

   

   Resources displayed in the Azure portal.

9. Once the cluster is reporting complete, we need to connect to it. Use this command to retrieve the credentials for the cluster:

   
       az aro list-credentials \
         --name $CLUSTER 
         --resource-group $RG                                                                                                                                                                                                                                
                               

10. Running this command retrieves the details for the cluster.

    
        az aro show 
          --name $CLUSTER 
          --resource-group $RG 
          --query “consoleProfile.url” -o tsv                                                                                                                                                                                                                                                      
                                

    The output will include a URL to connect to the cluster we created.
11. Use the credentials extracted from the list-credentials command to log in to the Red Hat Management portal.

Pick the ideal instance type for your workload using an ML-powered visual catalog map

See How It Works

Now you have full cluster access for advanced customization and management. It also gives you complete control over upgrades and life cycle management.

Storage classes are displayed in the Red Hat OpenShift management Portal.

You also get direct access to Azure storage, compute, and auto-scaling options, among others.

Azure-related options are displayed in the Red Hat OpenShift management Portal.

How to Configure Azure OpenShift Monitoring

Another excellent integration with Azure is being able to configure Azure RedHat OpenShift version 4 with container insight. After deployment, Azure Insights automatically displays the container. It is displayed under “Unmonitored clusters”, as shown in the screenshot below.

A container is displayed in the “Unmonitored clusters” section of the Azure portal.

Prerequisites

Before we proceed, make sure:

• Your CLI has access to bash4
• Helm is installed
• The OC command is installed

Identify under/over-provisioned K8s resources and use Terraform to auto-optimize

WATCH 3-MIN VIDEO

Running the install script and configuring monitoring

1. Microsoft provides a monitoring script that makes it very easy to enable monitoring. We start by setting up the resource id, resource group, and cluster name, with the command below:

   export ResourceId=”subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroup/rg-name/providers/Microsoft.RedHatOpenShift/OpenShiftClusters/clustername” 
                                   bash enable-monitoring.sh -- resource-id $ResourceId                                                                                                                                                                                                                                                             
                               

2. After running the script, CLI will prompt you to grant access by authentication.

   

3. Go to the highlighted URL and enter the provided authentication code
4. Once the script runs, the cluster will be under monitored clusters.

   

   A container is displayed in the “Monitored clusters” section of the Azure portal.

5. Click on the cluster name to access all the various configured dashboards. Depending on the deployment region, it will take around 5-10 minutes for the data to become available.

   

6. 
   

 

Azure Kubernetes Service Control Plane

The Sumo Logic app for Azure Kubernetes Service (AKS) - Control Plane provides visibility into the AKS control plane with operational insights into the API server, scheduler, control manager, and worker nodes. The app's preconfigured dashboards display resource-related metrics for Kubernetes deployments, clusters, namespaces, pods, containers, and daemonsets.

Azure Kubernetes Service (AKS) is a Kubernetes environment with clusters managed by Azure. AKS simplifies deploying and managing container-based applications, while automatically provisioning, upgrading, and scaling resources as needed.

Supported versions​

The following are the minimum supported requirements for this application:

Name	Supported versions
Kubernetes	1.10 and later
AKS	1.12.8

Log types​

The AKS - Control Plane app collects logs for the following Azure Kubernetes Services:

• kube-apiserver - The API server exposes the underlying Kubernetes APIs. This component provides the interaction for management tools, such as kubectl or the Kubernetes dashboard.
• kube-scheduler - The Scheduler determines what nodes can run the workload when you create or scale applications and then starts them.
• kube-controller-manager - The Controller Manager oversees a number of smaller controllers that perform actions, such as replicating pods and handling node operations.

Sample log messages​

kube-apiserver

{
  "operationName":"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
  "category":"kube-apiserver",
  "resourceId":"/SUBSCRIPTIONS/C111111-DXXX-4XXX-AXXX-900000000/RESOURCEGROUPS/AG-AKS-RG/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/AG-AKS-CLUSTER",
  "properties":
  {
    "log":"I0624 20:14:59.855669       1 wrap.go:47] PUT /api/v1/namespaces/kube-system/endpoints/kube-scheduler?timeout=10s:(9.05251ms) 200 [hyperkube/v1.12.8 (linux/amd64) kubernetes/a89f8c1/leader-election 172.31.1.1:48110]",
    "stream":"stderr",
    "pod":"kube-apiserver-796bd9b775-xqk5s",
    "containerID":"2d6cac1300da3226323fd1b936fe8278b87cba2b7a1bbd9c8401da6f8e786f5e"
  },
  "time":"2019-06-24T20:14:59.000Z"
}

kube-controller-manager

{
"operationName":"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
"category":"kube-controller-manager",
"resourceId":"/SUBSCRIPTIONS/C111111-DXXX-4XXX-AXXX-900000000/RESOURCEGROUPS/AG-AKS-RG/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/AG-AKS-CLUSTER",
"properties":
{
  "log":"I0624 07:27:25.9763861 event.go:221] Event(v1.ObjectReference{Kind:\"DaemonSet\",Namespace:\"kube-system\", Name:\"kube-proxy\", UID:\"2dfb3905-7dac-11e9-b60d-0a58ac1f01f6\",APIVersion:\"apps/v1\", ResourceVersion:\"4150266\", FieldPath:\"\"}): type: 'Normal'reason: 'SuccessfulCreate' Created pod: kube-proxy-xhmv7",
  "stream":"stderr",
  "pod":"kube-controller-manager-59fd65c5bd-694kh",
  "containerID":"667b540db41b66e914ca2ed496e0bef6d4a0b73fc832f5d5eba958d8a56a5e93"
},
"time":"2019-06-24T07:27:25.000Z"
}

kube-scheduler

{
"operationName":"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
"category":"kube-scheduler",
"resourceId":"/SUBSCRIPTIONS/C111111-DXXX-4XXX-AXXX-900000000/RESOURCEGROUPS/AG-AKS-RG/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/AG-AKS-CLUSTER",
"properties":
{
  "log":"W0622 22:50:25.377565       1 reflector.go:256] k8s.io/client-go/informers/factory.go:131:watch of *v1.StorageClass ended with: too old resource version: 3828720 (3970094)",
  "stream":"stderr",
  "pod":"kube-scheduler-744886667c-cxnvk",
  "containerID":"6093b28d82e1f850fb5a9f59ed8c31aa7179e8a3907449dcd2450a6605341a60"
},
"time":"2019-06-22T22:50:25.000Z"
}

Sample queries​

kube-apiserver

_sourceCategory="azure/aks" "kube-apiserver"
| json "properties.log", "category", "time", "properties.pod", "resourceId" as log, category, time, pod, resourceId
| where category ="kube-apiserver"
| parse regex field=log "(?<severity>W|I|F|E)(?<tt>[\S]+) (?<times>[\d:.]+)[\s]+(?<log_msg>.*)"
| parse regex field=resourceId "RESOURCEGROUPS\/(?<resource_grp>[\S]+)\/PROVIDERS\/MICROSOFT\.CONTAINERSERVICE\/MANAGEDCLUSTERS\/(?<cluster>[\S]+)"
| timeslice 1h
| count by _timeslice, severity
| transpose row _timeslice column severity
| fillmissing timeslice(1h)

kube-controller-manager

_sourceCategory="azure/aks" ("kube-controller-manager")
| json "properties.log", "category", "time", "properties.pod", "resourceId" as log, category, time, pod, resourceId
| where category ="kube-controller-manager"
| parse regex field=log "(?<severity>W|I|F|E)(?<tt>[\S]+) (?<times>[\d:.]+)[\s]+(?<log_msg>.*)"
| parse regex field=resourceId "RESOURCEGROUPS\/(?<resource_grp>[\S]+)\/PROVIDERS\/MICROSOFT\.CONTAINERSERVICE\/MANAGEDCLUSTERS\/(?<cluster>[\S]+)"
| timeslice 1h
| count by _timeslice, severity
| transpose row _timeslice column severity
| fillmissing timeslice(1h)

kube-scheduler

_sourceCategory="azure/aks" "kube-scheduler"
| json "properties.log", "category", "time", "properties.pod", "resourceId" as log, category, time, pod, resourceId
| where category ="kube-scheduler"
| parse regex field=log "(?<severity>W|I|F|E)(?<tt>[\S]+) (?<times>[\d:.]+)[\s]+(?<log_msg>.*)"
| parse regex field=resourceId "RESOURCEGROUPS\/(?<resource_grp>[\S]+)\/PROVIDERS\/MICROSOFT\.CONTAINERSERVICE\/MANAGEDCLUSTERS\/(?<cluster>[\S]+)"
| timeslice 1h
| count by _timeslice, severity
| transpose row _timeslice column severity
| fillmissing timeslice(1h)

Collecting Logs for the Kubernetes and AKS - Control Plane​

The Sumo Logic Kubernetes app works in conjunction with the AKS - Control Plane app and allows you to monitor worker node logs, as well as metrics for the Azure monitor and worker nodes.

Collecting logs and installing the Kubernetes app​

The Sumo Logic Kubernetes app provides the services for managing and monitoring Kubernetes worker nodes. You must set up collection and install the Kubernetes app before configuring collection for the AKS - Control Plane app. You will configure log and metric collection during this process.

To set up collection and install the Kubernetes app, follow the instructions in this document.

Collecting logs for the AKS - Control Plane app​

This section walks you through the process of configuring a pipeline to send logs from Azure Monitor to Sumo Logic.

1. To set up the logs collection in Sumo Logic, refer to Azure Event Hubs Source for Logs.

   When you configure the event hubs source, plan your source category to ease the querying process. A hierarchical approach allows you to make use of wildcards. For example: Azure/AKS/ControlPlane/Logs.

   Enable the Kubernetes master node logs in Azure Kubernetes Service to send logs to an event hub created in the previous step.

2. Push logs from Azure Monitor to Event Hub.

   1. Sign in to Azure Portal.
   2. Go to Kubernetes Services. Select your AKS cluster from which you want to collect logs.
   3. In the Monitoring Section, the Diagnostic Settings blade displays any existing settings. Click Edit Setting if you want to change your existing settings, or click Add diagnostic setting to add a new one. You can have a maximum of three settings.
   4. Enter a name.
   5. Check the Stream to an event hub box and click Event hub / Configure.
   6. Select an Azure subscription.
   7. Event bub namespace. If you have chosen Method 1 (Azure Event Hubs Source) for collecting logs, select the EventHubNamespace created manually, or else if you have chosen Method 2 (Collect logs from Azure monitor using Azure functions), then select SumoAzureLogsNamespace<UniqueSuffix> namespace created by the ARM template.
   8. Event hub name (optional). If you have chosen Method 1 (Azure Event Hub Source) for collecting logs, select the event hub name, which you created manually, or if you have chosen Method 2 (Collect logs from Azure monitor using Azure functions), then select insights-operational-logs.
   9. Select RootManageSharedAccessKey from Select event hub policy name dropdown.
   10. Select the checkbox for log types under Categories which you want to ingest.
       
   11. Click Save.

Installing the AKS Control Plane app​

Now that you have set up collection for AKS, you can install the Sumo Logic app for AKS and access the pre-configured Kubernetes dashboards for visibility into your AKS environment from a single-pane-of-glass.

All the dashboards are linked to the Kubernetes views so they can be easily accessed by clicking the Cluster in the navigation pane of the tab.

To install the app:

1. Select App Catalog.
2. In the Search Apps field, search for and then select your app.
3. Optionally, you can scroll down to preview the dashboards included with the app. Then, click Install App (sometimes this button says Add Integration).
4. On the next configuration page, under Select Data Source for your App, complete the following fields:
   • Data Source. Select one of the following options:
     • Choose Source Category and select a source category from the list; or
     • Choose Enter a Custom Data Filter, and enter a custom source category beginning with an underscore. For example, _sourceCategory=MyCategory.
   • Folder Name. You can retain the existing name or enter a custom name of your choice for the app.
   • All Folders (optional). Default location is the Personal folder in your Library. If desired, you can choose a different location and/or click New Folder to add it to a new folder.
5. Click Next.
6. Look for the dialog confirming that your app was installed successfully.
   

Once an app is installed, it will appear in your Personal folder or the folder that you specified. From here, you can share it with other users in your organization. Dashboard panels will automatically start to fill with data matching the time range query received since you created the panel. Results won't be available immediately, but within about 20 minutes, you'll see completed graphs and maps.

Viewing the AKS Control Plane dashboards​

Filter with template variables​

Template variables provide dynamic dashboards that rescope data on the fly. As you apply variables to troubleshoot through your dashboard, you can view dynamic changes to the data for a fast resolution to the root cause. For more information, see the Filter with template variables page.

You can use template variables to drill down and examine the data on a granular level.

AKS Controller Manager​

The AKS Controller Manager dashboard provides a high-level view of severity types and trends, along with details on scale operations, pod creation and deletion, and recent error messages.

Use this dashboard to:

• Find pod and scale operations performed by controller manager.
• Find the severity of various controller manager events and analyze fatal and erroneous controller manager operation events.

AKS API Server​

The AKS API Server dashboard provides insights into API server severity events and trends, autoscaler and status code trends, top problem URLs, and a list of recent error messages.

Use this dashboard to:

• Understand the status codes of requests made to Kube API Server.
• Review the top 10 URLs with problem status codes.
• Review the severity of various Kube API Server events, and analyze any fatal or erroneous events of Kube API Server operations.
• Find spikes or abnormal activity in the status codes of auto-scaler operations.

AKS Scheduler​

The AKS Scheduler dashboard provides a high-level view of severity types and trends for the Kube scheduler, as well as a detailed list of error messages.

Use this dashboard to:

• Find the severity of various Kube scheduler events.
• Analyze fatal or erroneous events of Kube scheduler operations.