What is LaaS and how does it work? Storing logs in the cloud can be a voluminous -- and expensive -- challenge, but the cloud offers a variety of enticing benefits.
Logs represent a data center's vital signs: They are a record of the systems, devices, users, and application events and metrics that indicate the health, state and anomalies of an IT environment.
Individual OSes have collected systems and service logs since the earliest mainframes, minicomputers and Unix workstations. But once networks connected multiple machines into a LAN, individual logs only told part of the story. With a LAN, most log events resulted from -- and correlated with -- activity on other systems.
The rise of VMs, cloud services and containers multiplied the sources of log data exponentially. It created an explosion of forensic data that is intractable to use and understand without software assistance.
Log aggregation and management software arose to satisfy the need for log-specific data collection, organization, archival and analysis tools. Like many IT functions, log management software has evolved into a category of managed cloud services -- colloquially called logging as a service (LaaS) -- to handle the configuration, operation and security of these complicated systems.
Log management vs. SIEM
Log management software is often confused or conflated with security information event management (SIEM) software. Both monitor and analyze system and application data, so vendors often blur the lines between the two categories, with many SIEM products including a log management module. Conversely, some log management vendors also have SIEM offerings that work with or supplement their logging products.
The primary distinction between log management and SIEM is focus. SIEM tools prioritize data and metrics relevant to security, not the totality of an environment's system, user and application log output. Log management software and services provide a scalable, holistic platform to collect, manage, archive and analyze all of an IT environment's log output -- on premises and in the cloud.
Figure 1. Overlap of log management and SIEM
As Figure 1 illustrates, most log management platforms and LaaSes include these features:
data collection from multiple sources;
data aggregation and collation;
policy-based data management and archiving;
storage scaling and management of hundreds of terabytes or more;
customized reports, dashboards and visualizations; and
data analysis, trending and anomaly detection.
Administrators use log management to aggregate event data and telemetry from all sources in an IT environment so they can trace related activity across multiple systems.
Logging as a service is merely the 'SaaSification' of log management.
The collection of information from disparate sources is straightforward because log data is inherently portable, typically written to text files. Those files come in one of a few structured formats, such as syslog, JSON, common event format and extended log format, including the W3C ELF and comma-separated values (CSV).
Logging as a service is merely the "SaaSification" of log management. LaaS can be achieved in several ways: First, as a comprehensive managed service. The vendor typically -- but not always -- rents out and manages the cloud infrastructure necessary to operate the service for the user. Some vendors offer both multi- and single-tenant infrastructure. The latter provides dedicated resources -- typically storage and databases -- with more stringent service-level agreements.
The second way is as a hybrid DIY service. In this LaaS approach, users piece together several cloud services, with one or more software packages from a cloud marketplace, to create a complete log management system.
LaaS benefits and tradeoffs
The benefits of SaaS products -- and therefore LaaS tools -- include:
operational efficiency through outsourced software management;
financial efficiency from the substitution of Opex subscriptions for Capex for servers and software;
faster deployment times;
adaptability to changing work and IT environments, such as remote work, container clusters, cloud infrastructure and SaaS services;
better service quality through automatic product updates and security hotfixes; and
better security, as vendors have the scale to hire dedicated security teams to configure and monitor their infrastructure and software.
For instance, after the SolarWinds breach, a security advisory from the U.S. National Security Agency recommended that affected users migrate to cloud services (identity and access management, in this case) to improve security.
The advisory noted that once attackers use the SolarWinds malware to breach an internal network, they often exploit weak Microsoft Active Directory installations to escalate privileges. The NSA suggested using Azure Active Directory as the authoritative identity provider instead. With Azure AD, Microsoft manages the federation of authentication, and extends its own protections, such as system hardening, configuration and monitoring, to the user.
A downside of SaaS -- also noted by the NSA -- is that managed services might not work with existing internal applications. And for organizations with technical expertise and atypical requirements, SaaS doesn't offer the flexibility to micromanage system configurations, infrastructure policies and system versions.
Still, LaaS is the best option for most organizations because the expertise and resources required to deploy and administer a log management system can be significant -- more so if an organization opts to assemble open source components like the Elasticsearch Stack into a holistic environment.
Furthermore, on-premises log management infrastructure is costly. While large organizations might save money on storage costs, the judicious use of cloud services for cold storage reduces costs for smaller organizations. Organizations can cut costs significantly through services such as Amazon S3 Glacier or S3 Glacier Deep Archive, Azure Archive Storage or Blob Storage, or Google Cloud Coldline or Archive Storage. And most LaaS products integrate with cloud object storage for data archival.
Like other enterprise software providers, log management vendors have embraced the SaaS deployment and pricing model, and most major products are available as a managed service. Some vendors, like Sumo Logic, were born in the cloud as LaaS. Other popular log management platforms include Amazon Elasticsearch Service, Datadog Log Management, LogDNA, New Relic, SolarWinds Loggly and Papertrail, Splunk Cloud, Sematext and Azure Monitor, as shown in Figure 2 (source: Azure Monitor documentation).
