Azure Monitor Logs overview
Azure Monitor Logs is a centralized software as a service (SaaS) platform for collecting, analyzing, and acting on telemetry data generated by Azure and non-Azure resources and applications.
You can collect logs, manage data models and costs, and consume different types of data in one Log Analytics workspace, the primary Azure Monitor Logs resource. This means you never have to move data or manage other storage, and you can retain different data types for as long or as little as you need.
This article provides an overview of how Azure Monitor Logs works and explains how it addresses the needs and skills of different personas in an organization.
Log Analytics workspace
A Log Analytics workspace is a data store that holds tables into which you collect data.
To address the data storage and consumption needs of various personas who use a Log Analytics workspace, you can:
- Define table plans based on your data consumption and cost management needs.
- Manage low-cost long-term retention and interactive retention for each table.
- Manage access to the workspace and to specific tables.
- Use summary rules to aggregate critical data in summary tables. This lets you optimize data for ease of use and actionable insights, and store raw data in a table with a low-cost table plan for however long you need it.
- Create ready-to-run saved queries, visualizations, and alerts tailored to specific personas.
You can also configure network isolation, replicate your workspace across regions, and design a workspace architecture based on your business needs.
Kusto Query Language (KQL) and Log Analytics
You retrieve data from a Log Analytics workspace using a Kusto Query Language (KQL) query, which is a read-only request to process data and return results. KQL is a powerful tool that can analyze millions of records quickly. Use KQL to explore your logs, transform and aggregate data, discover patterns, identify anomalies and outliers, and more.
Log Analytics is a tool in the Azure portal for running log queries and analyzing their results. Log Analytics Simple mode lets any user, regardless of their knowledge of KQL, retrieve data from one or more tables with one click. A set of controls lets you explore and analyze the retrieved data using the most popular Azure Monitor Logs functionality in an intuitive, spreadsheet-like experience.
Users who are familiar with KQL can use Log Analytics KQL mode to edit and create queries, which they can then use in Azure Monitor features such as alerts and workbooks, or share with other users.
For a description of Log Analytics, see Overview of Log Analytics in Azure Monitor. For a walkthrough of using Log Analytics features to create a simple log query and analyze its results, see Log Analytics tutorial.
Built-in insights and custom dashboards, workbooks, and reports
Many of Azure Monitor's ready-to-use, curated Insights experiences store data in Azure Monitor Logs, and present this data in an intuitive way so you can monitor the performance and availability of your cloud and hybrid applications and their supporting components.
You can also create your own visualizations and reports using workbooks, dashboards, and Power BI.
Table plans
You can use one Log Analytics workspace to store any type of log required for any purpose. For example:
- High-volume, verbose data that requires cheap long-term storage for audit and compliance
- App and resource data for troubleshooting by developers
- Key event and performance data for scaling and alerting to ensure ongoing operational excellence and security
- Aggregated long-term data trends for advanced analytics and machine learning
Table plans let you manage data costs based on how often you use the data in a table and the type of analysis you need the data for.
The diagram and table below compare the Analytics, Basic, and Auxiliary table plans. For information about interactive and long-term retention, see Manage data retention in a Log Analytics workspace. For information about how to select or modify a table plan, see Select a table plan.
Features | Analytics | Basic | Auxiliary (Preview) |
---|---|---|---|
Best for | High-value data used for continuous monitoring, real-time detection, and performance analytics. | Medium-touch data needed for troubleshooting and incident response. | Low-touch data, such as verbose logs, and data required for auditing and compliance. |
Supported table types | All table types | Azure tables that support Basic logs and DCR-based custom tables | DCR-based custom tables |
Log queries | Full query capabilities. | Full Kusto Query Language (KQL) on a single table, which you can extend with data from an Analytics table using lookup. | Full KQL on a single table, which you can extend with data from an Analytics table using lookup. |
Query performance | Fast | Fast | Slower Good for auditing. Not optimized for real-time analysis. |
Alerts | ✅ | ❌ | ❌ |
Insights | ✅ | ❌ | ❌ |
Dashboards | ✅ | ✅ Cost per query for dashboard refreshes not included. | Possible, but slow to refresh, cost per query for dashboard refreshes not included. |
Data export | ✅ | ❌ | ❌ |
Microsoft Sentinel | ✅ | ✅ | ✅ |
Search jobs | ✅ | ✅ | ✅ |
Summary rules | ✅ | ✅ KQL limited to a single table | ✅ KQL limited to a single table |
Restore | ✅ | ✅ | ❌ |
Query price included | ✅ | ❌ | ❌ |
Ingestion cost | Standard | Reduced | Minimal |
Interactive retention | 30 days (90 days for Microsoft Sentinel and Application Insights). Can be extended to up to two years at a prorated monthly long-term retention charge. | 30 days | 30 days |
Total retention | Up to 12 years | Up to 12 years | Up to 12 years* *Public preview limitation: Auxiliary plan total retention is currently fixed at 365 days. |
Data collection
To collect data from a resource to your Log Analytics workspace:
- Set up the relevant data collection tool based on the table below.
- Decide which data you need to collect from the resource.
- Use transformations to remove sensitive data, enrich data or perform calculations, and filter out data you don't need, to reduce costs.
This table lists the tools Azure Monitor provides for collecting data from various resource types
Resource type | Data collection tool | Collected data |
---|---|---|
Azure | Diagnostic settings | Azure tenant - Microsoft Entra audit logs provide sign-in activity history and audit trail of changes made within a tenant. Azure resources - Logs and performance counters. Azure subscription - Service health records along with records on any configuration changes made to the resources in your Azure subscription. |
Application | Application insights | Application performance monitoring data. |
Container | Container insights | Container performance data. |
Virtual machine | Data collection rules | Monitoring data from the guest operating system of Azure and non-Azure virtual machines. |
Non-Azure source | Logs Ingestion API | File-based logs and any data you collect from a monitored resource. |
Working with Microsoft Sentinel and Microsoft Defender for Cloud
Microsoft Sentinel and Microsoft Defender for Cloud perform Security monitoring in Azure.
These services store their data in Azure Monitor Logs so that it can be analyzed with other log data collected by Azure Monitor.
No comments:
Post a Comment