AWS Lambda monitoring goes mainstream with New Relic beta
New Relic users and prospects will investigate AWS Lambda monitoring features released in private beta, as serverless usage filters in to enterprise IT shops.
A familiar DevOps monitoring vendor will burnish its AWS Lambda monitoring for enterprise early adopters of serverless deployment.
Many DevOps monitoring vendors, including New Relic Inc., Datadog and AppDynamics, already collect Amazon CloudWatch logs for AWS Lambda monitoring, but more sophisticated visibility into the relationships between AWS Lambda functions and advanced metrics such as request rates and high-percentile network latencies are less widely available. New Relic plans to add advanced AWS Lambda monitoring to its New Relic Infrastructure and APM tools, including distributed tracing features that track code execution through chains of Lambda functions, and automated anomaly detection to identify performance outliers. The features were released in private beta this week and are scheduled for general availability in the first half of 2019.
"This has been a pain point with Lambda," said Shannon Lietz, leader and director of DevSecOps for a financial software maker in San Francisco, which uses New Relic's DevOps monitoring suite. "Anyone can write a Lambda function, but the question is what the software looks like at the enterprise level and how microservices come together to create an app."
Lietz's DevSecOps team created its own AWS Lambda monitoring utility, but the internal tool primarily addresses security for Lambda functions rather than performance monitoring.
"We tend to look into AWS-native features first, but they generally deal with the masses," Lietz said. "New Relic's specialization in this area [with distributed tracing] could be helpful."
Before this week's launch of a private beta for New Relic's AWS Lambda monitoring, Datadog was the only major DevOps monitoring firm to offer serverless support beyond CloudWatch integration. Datadog supports custom serverless monitoring metrics through an instrumented code approach, which injects a small library of monitoring code into functions. That library then sends monitoring data back to a repository for analysis when functions are invoked. Aside from Datadog, specialist startup vendors such as IOPipe, Dashbird and Espagon proffer advanced AWS Lambda monitoring.
We tend to look into AWS-native features first, but they generally deal with the masses. ... New Relic's specialization in this area [with distributed tracing] could be helpful.
Shannon Lietzleader and director of DevSecOps, San Francisco-based financial software provider
As AWS Lambda monitoring features become available in more widely used tools, such as New Relic, IT pros with serverless deployments face a time-honored choice: the convenience of a vendor that's a one-stop shop for multiple IT initiatives or the cutting-edge features of a specialized vendor.
"We're very happy with IOPipe, but serverless is a small portion of the environment. Any monitoring tool we choose also has to fit all of our older stuff, too," said Dave Townsend, principal software engineer at Matson, a cargo shipping company in Honolulu.
Townsend chose IOPipe's AWS Lambda monitoring technology to manage a small-scope serverless project in 2018. Matson's IT staff also began an evaluation of New Relic, Dynatrace and Datadog in 2018, and will consider New Relic's AWS Lambda monitoring features in that evaluation in early 2019, Townsend said.
Jury's still out on enterprise serverless adoption
New Relic officials say they've seen an increase in customer demand for AWS Lambda monitoring features in the last six months, but some industry watchers doubt serverless computing will see serious production use among traditional enterprises.
"People still haven't figured out a great use case for serverless beyond batch jobs and cron jobs, and they haven't figured out the right cost model," said Rich Lane, analyst at Forrester Research. "Some users still don't size serverless functions' memory consumption correctly, and then their spin-up and spin-down time takes longer and costs more than if the app ran on other infrastructure."
More-sophisticated monitoring tools could help solve that under-provisioning problem, but not without expensive trial and error to optimize serverless performance and costs, Lane predicted.
"Everyone seems to be realizing that the overall cloud computing dream, that it will save you a kajillion dollars, might not be fully realized," he said. "I don't know if [vendors] build it, users will come in terms of Lambda adoption."
Other analyst firms say serverless functions might not take over the world, but that they will have a place alongside containers and traditional VMs within enterprises, and that this will bolster the appeal of AWS Lambda monitoring features in established tools such as New Relic.
Serverless functions usually combine with other types of workloads to provide an application service, wrote Nancy Gohring, analyst at 451 Research, in an April 2018 report about serverless monitoring tools. Thus, enterprises will seek serverless monitoring features from IT vendors they already use, but those vendors may acquire specialized startups to quickly meet the demand for serverless monitoring features, Gohring's report stated.
