Decipher how and when to utilize the Azure logging options
Logging is a big piece of the cloud management puzzle. Do you know which logs to track in Azure Monitor, and when to use them? Learn more about Azure log types and best practices.
Infrastructure managers and cloud developers don't need a reminder about the importance of log data to maintain smooth, reliable and secure operation of systems and applications. But Microsoft Azure users might not be familiar with all their options in the cloud.
Logs are a critical aspect of monitoring, and log data can help keep application performance in check, as well as protect corporate information from security intrusions. Microsoft Azure has many logging options to choose from, making it difficult to decide which ones to rely on to track workloads and performance demands. Learn the Azure logging types and categories, then pick up best practices for performance monitoring based on logs.
Azure logging
Azure's monitoring capabilities have significantly improved as the cloud platform has matured. Azure Monitor is a tool that monitors an enterprise's applications, infrastructure and network. Enterprises can use Azure Monitor to analyze, visualize, retrieve and export log data, as well as configure alerts. Azure logging options are broken into three main types:
- Control and management logs record events about Azure activity that is typically performed via the Azure Resource Manager.
- Data plane logs collect information about Azure service and resource usage and include diagnostic logs.
- Processed events include a record of alerts and other events triggered by particular resource configurations or parameters.
Each log has a defined purpose and scope, which means a complete Azure logging strategy requires each of these types. Azure customers don't need a separate tool or UI to access each kind of log; Azure Monitor gathers all types into one place. The following are Azure's eight log categories and what they are used for:
- Activity logs: Provide insights into the operations performed on resources in a subscription.
- Azure Resource logs: Deliver operational insights about individual resources.
- Azure Active Directory reporting: Report user sign-in activities and system activity around users and group management.
- Virtual machines and cloud services: Capture system data and logging data on the VMs and aggregate that data into a separate Azure storage service.
- Azure Storage Analytics: Provide insight into trace requests, analyze usage trends and diagnose issues with storage accounts.
- Network security group flow logs: Display information about ingress and egress IP traffic through a Network Security Group.
- Application insight: Offer an application performance monitoring (APM) service for web developers on multiple platforms.
- Process data/security alerts: Provide security information and alerts.
Azure Monitor features
One main feature of Azure Monitor is log data collection and analysis. It supports activity logs, metrics, diagnostics logs and alert rules, as well as quick links to advanced monitoring and analytics tool available in Azure.
Azure Monitor provides a central place in the Azure management portal to consume service metrics and logs, configure alerts, set triggers for automated actions and access logging APIs. Admins can use the dashboard for ad hoc troubleshooting or in-depth study of system performance with advanced analytics. They can extend Azure Monitor to connect with third-party services via its APIs and data export features.
Since Azure Monitor's debut in 2016, Microsoft has added many features. Some additions include:
- a feature for container monitoring, using the Prometheus open source event monitoring software, in preview at time of publication;
- more granular control over log access using role-based access control permissions;
- automated determination of alert thresholds using real-time data, along with composite alerts covering multiple resources; and
- the integration of Azure Log Analytics and Application Insights into the Azure Monitor UI.
Microsoft plans to extend Azure Monitor with the following features:
- distributed tracing and support for open source languages and dependencies using OpenTelemetry (an open source observability framework, OpenTelemetry offers APIs, libraries, agents and collector services to capture distributed traces and application metrics);
- application change analysis for compute and network resources;
- snapshot debugging without access to source code;
- health monitoring for VMs, containers and network resources;
- unified connectivity monitoring across on-premises and Azure network resources; and
- integrations with the Azure DevOps services for ChatOps, bots, project environments and workload pipelines.
Best practices with Azure logs
Best practices for Azure logging and monitoring are no different from those for any other IT infrastructure environment.
First, identify the resources, events and metrics that are most critical to the organization's strategies. For each resource, choose measurements for performance and reliability levels that must be maintained to achieve enterprise goals. Some of these key performance indicators are included in service-level agreements.
Enterprises should set targets and limits for critical parameters, such as memory usage or application response time, that trigger alerts and prompt automatic action whenever possible. For example, a company can indicate the conditions that should trigger Azure workload to automatically scale up. Additionally, don't confine monitoring to parameters around the cloud resources alone, such as storage and network performance. Broaden it to include end-to-end APM and the overall user experience. To do this, enterprises will likely have to combine multiple measures into a composite metric that is displayed on dashboards. Make sure that Azure admins can drill into the details, i.e., individual parameters and events, as well as underlying composite metrics, through the monitoring UI.
To ease management difficulties, consolidate monitoring tasks under a single interface. For Microsoft's cloud customers, Azure Monitor provides that interface, with a customizable, graphical dashboard that highlights critical monitoring areas.
If your enterprise has cloud environments on multiple services, it will require a third-party, cloud-agnostic monitoring platform. If appropriate, that platform should support monitoring of on-premises infrastructure, as well as cloud environments, and integrate with other types of monitoring software.
