Tuesday, 30 July 2024

Manage a Datadog - An Azure Native ISV Service resource

 

Manage a Datadog - An Azure Native ISV Service resource

Resource overview

To see details of your Datadog resource, select Overview in the left pane.

Datadog resource overview

The details include:

  • Resource group name
  • Location/Region
  • Subscription
  • Tags
  • Single sign-on link to Datadog organization
  • Datadog offer/plan
  • Billing term

It also provides links to Datadog dashboards, logs, and host maps.

The overview screen provides a summary of the resources sending logs and metrics to Datadog.

  • Resource type - Azure resource type.
  • Total resources - Count of all resources for the resource type.
  • Resources sending logs - Count of resources sending logs to Datadog through the integration.
  • Resources sending metrics - Count of resources sending metrics to Datadog through the integration.

Reconfigure rules for metrics and logs

To change the configuration rules for metrics and logs, select Metrics and Logs in the left pane.

Screenshot showing how to modify the configuration of logs and metrics for the Datadog resource.

For more information, see Configure metrics and logs.

View monitored resources

To see the list of resources emitting logs to Datadog, select Monitored Resources in the Resource menu.

View resources monitored by Datadog

You can filter the list of resources by resource type, subscription, resource group name, location, and whether the resource is sending logs and metrics. Only the resource types supported by Azure Monitor are monitored. Check out the list of resource types supported by Azure Monitor.

The column Logs to Datadog indicates whether the resource is sending logs to Datadog. If the resource isn't sending logs, this field indicates why logs aren't being sent to Datadog. The reasons could be:

  • Resource doesn't support sending logs. Only resources types with monitoring log categories can be configured to send logs to Datadog.
  • Limit of five diagnostic settings reached. Each Azure resource can have a maximum of five diagnostic settings. For more information, see diagnostic settings.
  • Error. The resource is configured to send logs to Datadog, but is blocked by an error.
  • Logs not configured. Only Azure resources that have the appropriate resource tags are configured to send logs to Datadog.
  • Region not supported. The Azure resource is in a region that doesn't currently support sending logs to Datadog.
  • Datadog agent not configured. Virtual machines without the Datadog agent installed don't emit logs to Datadog.

Monitor multiple subscriptions

While monitoring resources in a subscription is straightforward, observing multiple subscriptions, each with their own set of resources becomes tricky. Instead, you can monitor multiple subscriptions by linking them to a single Datadog resource that is tied to a Datadog organization. This provides a single pane of glass view for all resources across multiple subscriptions.

To manage multiple subscriptions that you want to monitor, select Monitored Subscriptions in the Datadog organization configurations section of the Resource menu.

Screenshot showing Monitored Subscriptions selected in the Resource menu.

From Monitored Subscriptions in the Resource menu, select the Add Subscriptions. The Add Subscriptions experience that opens and shows the subscriptions you have Owner role assigned to and any Datadog resource created in those subscriptions that is already linked to the same Datadog organization as the current resource.

If the subscription you want to monitor has a resource already linked to the same Datadog org, we recommended that you delete the Datadog resources to avoid shipping duplicate data, and incurring double the charges.

Select the subscriptions you want to monitor through the Datadog resource and select Add.

Screenshot showing subscriptions to add.

If the list doesn’t get updated automatically, select Refresh to view the subscriptions and their monitoring status. You might see an intermediate status of In Progress while a subscription gets added. When the subscription is successfully added, you see the status is updated to Active. If a subscription fails to get added, Monitoring Status shows as Failed.

Screenshot showing statuses of monitored subscriptions.

The set of tag rules for metrics and logs defined for the Datadog resource apply to all subscriptions that are added for monitoring. Setting separate tag rules for different subscriptions isn't supported. Diagnostics settings are automatically added to resources in the added subscriptions that match the tag rules defined for the Datadog resource. To reconfigure the rules, check Reconfigure rules for metrics and logs.

You can unlink subscriptions you don't want monitored through the Datadog resource by selecting Monitored Subscriptions from the Resource menu. Then, select any subscription you want to remove, and select Remove subscriptions. Select Refresh to view the updated list of subscriptions being monitored.

Screenshot showing subscriptions to remove.

API keys

To view the list of API keys for your Datadog resource, select the Keys in the left pane. You see information about the keys.

API keys for the Datadog organization.

The Azure portal provides a read-only view of the API keys. To manage the keys, select the Datadog portal link. After making changes in the Datadog portal, refresh the Azure portal view.

The Azure Datadog integration provides you with the ability to install Datadog agent on a virtual machine or app service. If a default key isn't selected, the Datadog agent installation fails.

Monitor virtual machines using the Datadog agent

You can install Datadog agents on virtual machines as an extension. Go to Virtual machine agent under the Datadog organization configurations in the Resource menu. This screen shows all the virtual machines across all subscriptions where you have the “Owner” role assigned to. All subscriptions are selected by default. You can select a subset of subscriptions to narrow down the list of virtual machines shown in the subscription.

For each virtual machine, the following data is displayed:

  • Resource Name – Virtual machine name
  • Resource Status – Whether the virtual machine is stopped or running. The Datadog agent can only be installed on virtual machines that are running. If the virtual machine is stopped, installing the Datadog agent is disabled.
  • Agent version – The Datadog agent version number.
  • Agent status – Whether the Datadog agent is running on the virtual machine.
  • Integrations enabled – The key metrics that are being collected by the Datadog agent.
  • Install Method – The specific tool used to install the Datadog agent. For example, Chef or Script.
  • Sending logs – Whether the Datadog agent is sending logs to Datadog.

Select the virtual machine to install the Datadog agent on. Select Install Agent.

The portal asks for confirmation that you want to install the agent with the default key. Select OK to begin installation. Azure shows the status as Installing until the agent is installed and provisioned.

After the Datadog agent is installed, the status changes to Installed.

To see that the Datadog agent has been installed, select the virtual machine and navigate to the Extensions window.

You can uninstall Datadog agents on a virtual machine by going to Virtual machine agent. Select the virtual machine and Uninstall agent.

Monitor App Services using the Datadog agent as an extension

You can install Datadog agents on app services as an extension. Go to App Service extension in left pane. This screen shows the list of all app services across all subscriptions you have “Owner” role assigned to. All subscriptions are selected by default. You can select a subset of subscriptions to narrow down the list of app services shown.

Screenshot showing App Service extension selected in the Resource menu.

For each app service, the following data elements are displayed:

  • Resource Name – Virtual machine name.
  • Resource Status – Whether the app service is stopped or running. The Datadog agent can only be installed on app services that are running. If the app service is stopped, installing the Datadog agent is disabled.
  • App service plan – The specific plan configured for the app service.
  • Agent version – The Datadog agent version number.

To install the Datadog agent, select the app service and Install Extension. The latest Datadog agent is installed on the app service as an extension.

The portal confirms that you want to install the Datadog agent. Also, the application settings for the specific app service are updated with the default key. The app service is restarted after the install of the Datadog agent completes.

Select OK to begin the installation process for the Datadog agent. The portal shows the status as Installing until the agent is installed. After the Datadog agent is installed, the status changes to Installed.

To uninstall Datadog agents on the app service, go to App Service Extension. Select the app service and Uninstall Extension

Reconfigure single sign-on

If you would like to reconfigure single sign-on, select Single sign-on in the left pane.

To establish single sign-on through Microsoft Entra ID, select Enable single sign-on through Microsoft Entra ID.

The portal retrieves the appropriate Datadog application from Microsoft Entra ID. The app comes from the enterprise app name you selected when setting up integration. Select the Datadog app name:

Reconfigure single sign-on application.

Change Plan

To change the Datadog billing plan, go to Overview and select Change Plan.

Select change Datadog billing plan.

The portal retrieves all the available Datadog plans for your tenant. Select the appropriate plan and select on Change Plan.

Select the Datadog billing plan to change. 

Disable or enable integration

You can stop sending logs and metrics from Azure to Datadog. You continue to be billed for other Datadog services that aren't related to monitoring metrics and logs.

To disable the Azure integration with Datadog, go to Overview. Select Disable and OK.

Disable Datadog resource.

To enable the Azure integration with Datadog, go to Overview. Select Enable and OK. Selecting Enable retrieves any previous configuration for metrics and logs. The configuration determines which Azure resources emit metrics and logs to Datadog. After you complete this step, metrics and logs are sent to Datadog.

Enable Datadog resource.

Delete Datadog resource

Go to Overview in left pane and select Delete. Confirm that you want to delete Datadog resource. Select Delete.

Delete Datadog resource

If only one Datadog resource is mapped to a Datadog organization, logs and metrics are no longer sent to Datadog. All billing stops for Datadog through Azure Marketplace.

If more than one Datadog resource is mapped to the Datadog organization, deleting the Datadog resource only stops sending logs and metrics for that Datadog resource. Because the Datadog organization is linked to other Azure resources, billing continues through the Azure Marketplace.

Explore the latest features for Datadog—An Azure Native ISV Service

 The service is easy to provision and manage, like any other Azure resource, using the Azure Portal, Azure Command-Line Interface (CLI), software development kits (SDKs), and more. You do not need any custom code or connectors to start viewing your logs and metrics on the Datadog portal.

The service has continued to grow and has been adopted well by our joint customers. This service is developed and managed by Microsoft and Datadog and based on your feedback, we continue to invest in deeper integrations to make the experience smoother for you. Here are some of the top features shipped recently that we would like to highlight:

Monitor multiple subscriptions with a single Datadog Resource

We are excited to announce a scalable multi-subscription monitoring capability that allows you to configure monitoring for all your subscriptions through a single Datadog resource. This simplifies the process of monitoring numerous subscriptions as you do not need to setup a separate Datadog resource in every single subscription that you wish to monitor.

To start monitoring multiple subscriptions through a single “Datadog—An Azure Native ISV Service” resource, click on the Monitored Subscriptions blade under the Datadog organizations configurations section.

graphical user interface, application, Teams

The subscription in which the Datadog resource is created is monitored by default. To include additional subscriptions, click on the “Add subscriptions” button and on the window that opens, select the subscriptions that you want to monitor using the same resource.

graphical user interface, text, application, email

We recommend deleting redundant Datadog resources linked to the same organization and consolidating multiple subscriptions into a single Datadog resource wherever possible. This would help avoid duplicate data flow and issues like throttling. For example, in the image shown below, there is a resource named DatadogLinkingTest linked to the same organization in one of the subscriptions. You should ideally delete the resource before proceeding to add the subscription.

graphical user interface, text, application, email

Click on Add to include the chosen subscriptions to the list of subscriptions being monitored through the Datadog resource.

graphical user interface, application

The set of tag rules for metrics and logs defined for the Datadog resource apply to all subscriptions that are added for monitoring. If you wish to reconfigure the tag rules at any point, check Reconfigure rules for metrics and logs.

And now you are done. Go to the “Monitored Resources” blade in your Datadog resource and filter the subscription of your choice to check the status of logs and metrics being sent to Datadog for the resources in that subscription.

table

Likewise, agent management experience for App Services and virtual machines (VMs) also spans multiple subscriptions now. 

table

Check out Monitor virtual machines using the Datadog agent and Monitor App Services using the Datadog agent as an extension.

If at any point you wish to stop monitoring resources in a subscription via the Datadog resource, you can remove the subscription from the Monitored subscriptions list. In the Monitored Subscriptions blade, choose the subscription you no longer wish to monitor and click on “Remove subscriptions”. The default subscription (the one in which the Datadog resource is created) can’t be removed.

graphical user interface, application, email, Teams

Log forwarder

The automatic log forwarding capability available out of the box with Datadog’s native integration on Azure eliminates time-consuming steps that require you to setup additional infrastructure and write custom code.

We are constantly working to support all resource categories on Azure Monitor to ship logs to Datadog. For customers who have setup monitoring tag rules in an Azure subscription, new resource types or categories get automatically enrolled for sending logs, without the need for customers to manually do any changes to enable new resource types. As of today, the native integration on Azure supports logs from 126 resource types to flow to Datadog.

Cloud Security Posture Management

In the Datadog Azure Native integration, enabling Cloud Security Posture Management (CSPM) for your Azure Resources is a straightforward operation in your Datadog resource. Navigate to the Cloud Security Posture Management blade, click on the checkbox to enable CSPM and click Save. The setting can be disabled at any point.

graphical user interface, text, application, email

You can learn more about Datadog’s CSPM product here

Mute monitor for expected virtual machine shutdowns

Imagine alerts being sent for expected VM shutdowns and waking you up in the middle of the night. Yikes! Now, with just the click of a checkbox, you can avoid scenarios where Datadog’s disaster prevention alert notifications get triggered during scheduled shutdowns. To mute the monitor for expected Azure Virtual Machine shutdowns, select the checkbox shown below in the Metrics and Logs blade.

graphical user interface, text, application, email

Get started with Datadog - An Azure Native ISV Service by creating new instance

 

 Get started with Datadog - An Azure Native ISV Service by creating new instance

Prerequisites

Before creating your first instance of Datadog in Azure, configure your environment. These steps must be completed before continuing with the next steps in this quickstart.

Find offer

Use the Azure portal to find Datadog.

  1. Go to the Azure portal and sign in.

  2. If you've visited the Marketplace in a recent session, select the icon from the available options. Otherwise, search for Marketplace.

    Screenshot of the Azure Marketplace icon.

  3. In the Marketplace, search for Datadog.

  4. In the plan overview screen, select Subscribe.

    Screenshot of the Datadog application in Azure Marketplace.

Create a Datadog resource in Azure

The portal displays a selection asking whether you would like to create a Datadog organization or link Azure subscription to an existing Datadog organization.

If you're creating a new Datadog organization, select Create under the Create a new Datadog organization

Screenshot of the create or link a Datadog organization.

The portal displays a form for creating the Datadog resource.

Screenshot of the create Datadog resource.

Provide the following values.

PropertyDescription
SubscriptionSelect the Azure subscription you want to use for creating the Datadog resource. You must have owner access.
Resource groupSpecify whether you want to create a new resource group or use an existing one. A resource group is a container that holds related resources for an Azure solution.
Resource nameSpecify a name for the Datadog resource. This name will be the name of the new Datadog organization, when creating a new Datadog organization.
LocationSelect West US 2. Currently, West US 2 is the only supported region.
Pricing planWhen creating a new organization, select from the list of available Datadog plans.
Billing TermMonthly.

Configure metrics and logs

Use Azure resource tags to configure which metrics and logs are sent to Datadog. You can include or exclude metrics and logs for specific resources.

Tag rules for sending metrics are:

  • By default, metrics are collected for all resources, except virtual machines, Virtual Machine Scale Sets, and App Service plans.
  • Virtual machines, Virtual Machine Scale Sets, and App Service plan with Include tags send metrics to Datadog.
  • Virtual machines, Virtual Machine Scale Sets, and App Service plan with Exclude tags don't send metrics to Datadog.
  • If there's a conflict between inclusion and exclusion rules, exclusion takes priority.

Tag rules for sending logs are:

  • By default, logs are collected for all resources.
  • Azure resources with Include tags send logs to Datadog.
  • Azure resources with Exclude tags don't send logs to Datadog.
  • If there's a conflict between inclusion and exclusion rules, exclusion takes priority.

For example, the following screenshot shows a tag rule where only those virtual machines, Virtual Machine Scale Sets, and App Service plan tagged as Datadog = True send metrics to Datadog.

Screenshot of how to configure metrics and logs in Azure for Datadog.

There are three types of logs that can be sent from Azure to Datadog.

  1. Subscription level logs - Provide insight into the operations on your resources at the control plane. Updates on service health events are also included. Use the activity log to determine the what, who, and when for any write operations (PUT, POST, DELETE). There's a single activity log for each Azure subscription.

  2. Azure resource logs - Provide insight into operations that were taken on an Azure resource at the data plane. For example, getting a secret from a Key Vault is a data plane operation. Or, making a request to a database is also a data plane operation. The content of resource logs varies by the Azure service and resource type.

  3. Microsoft Entra logs - As an IT administrator, you want to monitor your IT environment. The information about your system's health enables you to assess potential issues and decide how to respond.

The Microsoft Entra admin center gives you access to three activity logs:

  • Sign-in – Information about sign-ins and how your resources are used by your users.
  • Audit – Information about changes applied to your tenant such as users and group management or updates applied to your tenant's resources.
  • Provisioning – Activities performed by the provisioning service, such as the creation of a group in ServiceNow or a user imported from Workday.

To send subscription level logs to Datadog, select Send subscription activity logs. If this option is left unchecked, none of the subscription level logs are sent to Datadog.

To send Azure resource logs to Datadog, select Send Azure resource logs for all defined resources. The types of Azure resource logs are listed in Azure Monitor Resource Log categories. To filter the set of Azure resources sending logs to Datadog, use Azure resource tags.

You can request your IT Administrator to route Microsoft Entra logs to Datadog. For more information, see Microsoft Entra activity logs in Azure Monitor.

The logs sent to Datadog will be charged by Azure. For more information, see the pricing of platform logs sent to Azure Marketplace partners.

Once you have completed configuring metrics and logs, select Next: Single sign-on.

Configure single sign-on

If your organization uses Microsoft Entra ID as its identity provider, you can establish single sign-on from the Azure portal to Datadog. If your organization uses a different identity provider or you don't want to establish single sign-on at this time, you can skip this section.

To establish single sign-on through Microsoft Entra ID, select the checkbox for Enable single sign-on through Microsoft Entra ID.

The Azure portal retrieves the appropriate Datadog application from Microsoft Entra ID. The app matches the Enterprise app you provided in an earlier step.

Select the Datadog app name.

Screenshot of the enable Single sign-on to Datadog.

Select Next: Tags.

Add custom tags

You can specify custom tags for the new Datadog resource. Provide name and value pairs for the tags to apply to the Datadog resource.

Screenshot of the add custom tags for the Datadog resource.

When you've finished adding tags, select Next: Review+Create.

Review + Create Datadog resource

Review your selections and the terms of use. After validation completes, select Create.

Screenshot of Review and Create a Datadog resource.

Azure deploys the Datadog resource.

When the process completes, select Go to Resource to see the Datadog resource.

Screenshot of the Datadog resource deployment.