Which APM tool works for devs, CIOs? New Relic vs. Stackify
Stackify Retrace and New Relic tools typify a divide between developers and CIOs on application monitoring. The products share many capabilities, but those features aren't equally granular or accessible.
Modern applications include a tangled web of components linked through network-connected workflows and hosted on layers of technology, which makes them challenging to manage.
Despite this complexity, applications still must meet the quality of experience (QoE) goals set for them in the business plan, as well as hit performance expectations, from testing through deployment.
Application performance monitoring (APM) tools have evolved as applications change. APM vendors, as represented in this New Relic vs. Stackify showdown, often tailor products to focus on a critical opportunity in the market, such as the shift from system log analysis as the exclusive source of performance insights to the integration of problem tracking at the code level. As APM gravitates toward becoming a part of application lifecycle management overall, vendors find development teams reluctant to accept a shift to higher-level, overall app health metrics -- known as integrated APM. And, so far, CIOs don't want to push matters.
"Developers keep coming up with reasons [the integrated APM] approach won't work," said one CIO in a CIMI Corporation survey.
With a complex combination of features in each product, APM tool selection is difficult. It's hard to understand what will be required to adopt a tool, what benefits it will provide and which barriers might wait behind this maze of features. So, the best APM tool is either one that can produce the most useful results or one that produces good results with minimal effort.
Integrated APM tool showdown
Stackify Retrace and New Relic APM both fit the integrated APM model. These multifaceted products are poster children for that best-versus-easiest tradeoff. Both products enable teams to gain insight into server operation via log data analysis, and both extend performance and fault visibility right into the code. Additionally, each one provides dashboards that display performance, application and component data, including some insights into infrastructure behavior. The two tools also support synthetic APM, a form of test data injection monitoring that uses simulated web activity to create specific and controllable interactions that can be measured and validated.
Organizations can struggle to figure out exactly what they need and how much it will cost in an APM tool comparison. New Relic vs. Stackify doesn't compare just two sets of features: These tools' sheer scope of action, with accompanying variability in cost and learning curve, tends to frustrate users.
IT teams need different UI options to monitor overall application health, as well as other metrics, and both products aim to accommodate these diverse needs. Demand for overall application health and QoE measurement is on the rise. The Application Performance Index (Apdex), an open standard that measures user satisfaction with application performance, helps some APM users, while others find Apdex numbers difficult to interpret and want lower-level performance metrics to inform app health decisions.
As with every APM tool, logging informs the trends and conclusions of both Retrace and New Relic APM. Each APM vendor customizes its product to collect code-level log events at important points. That functionality expands visibility into the application code, which enables teams to use APM as a testing tool, not just to monitor live application performance. Without code-level data, these products would rely only on system logs and thereby limit their information about problems and performance.
New Relic vs. Stackify
Stackify Retrace targets developers and facilitates integration of within-the-code custom logging to provide more detailed information. New Relic APM doesn't offer as deep a set of developer tools, though users say that it's easier to adopt. New Relic APM includes stronger support for applications that mix programming languages and for apps based largely on third-party software.
User response to the New Relic vs. Stackify debate varies depending on perspective. Adopters say that they like New Relic APM for the higher-level, less granular assessment, as it provides comprehensible knowledge of the state of the application based on standardized metrics. Users can monitor almost any mobile, web, third-party or custom application with New Relic APM. However, users prefer Stackify Retrace for digging into application performance, testing and infrastructure states, which combine to define the user experience.
New Relic APM is the incumbent APM tool, the older product with the larger market share. This established market position gives challengers a chance to assess its weaknesses before they launch competing products. In that vein, Retrace focuses on a different constituency within the enterprise market for APM tools.
CIMI Corporation surveys revealed that enterprise CIOs favor New Relic APM by a ratio of nearly 3-to-1, while developers favor Stackify Retrace by a nearly 4-to-1 ratio. Obviously, the ideal APM product would be one that can manage to synthesize and satisfy the needs of both groups.
CIOs say New Relic APM gives them insight into their critical applications rapidly, with easily understood metrics and full use of system and application logs. They also like the ability to add logging and code points into applications to expand their performance visibility. However, CIOs admit that developers push back strongly on these points.
Stackify Retrace can satisfy CIOs. It's much more difficult to make New Relic APM appeal to developers, which makes Stackify Retrace the winner in this face-off.
CIOs' objections to Retrace center on difficulty understanding its basic displays. While Retrace includes the custom code profiling that developers like, CIOs argue it is devalued by limited integration and correlation of this data with the baseline system logs. Additionally, Retrace imposes a per-server charge that can quickly add up in a big data center.
Developers see the New Relic vs. Stackify debate through an entirely different lens. Retrace enables teams to take performance logging beyond anything New Relic APM achieves, including diagnosis of software bugs. Developers agree it takes time to understand, customize and contextualize the tool's logging points in code. But they think that the result is much better than with New Relic APM and worth the effort.
Developers think the benefits of New Relic's high-level integrated APM metrics are overstated. They believe that the Apdex information and other key performance indicators and alerts that New Relic APM offers are too difficult to correlate with code conditions that cause actual problems. Retrace's App Dashboard is busier than New Relic APM and takes some adjustment, but it conveys what developers really need to know.
Crown the champion
There's a cultural divide in the APM space, epitomized by the clash of New Relic vs. Stackify. It's difficult for any product to bridge such a divide. But, in the end, a combination of both perspectives reveals a critical truth.
With proper management and attention to the integration of the logging points added to the code, Stackify Retrace can satisfy CIOs. It's much more difficult to make New Relic APM appeal to developers, which makes Stackify Retrace the winner in this face-off.
While New Relic APM supports Apdex, that number doesn't translate as easily into detailed performance drilldowns as Stackify Retrace's App Score grades. Furthermore, Retrace also offers Apdex results for user satisfaction, and it's fairly easy to correlate these with the App Score grades by application or technology. It seems likely that CIO opinions on the high-level displays of both products are influenced by website material and early reviews, rather than detailed assessments.
When you dig down to the details, Stackify Retrace is equally capable of presenting information to senior management as New Relic APM, and the integration of code profiling doesn't excessively complicate the product's capabilities. CIOs must simply communicate monitoring metrics needs with their teams to get on the same page with the Retrace tool. Retrace is more complex than New Relic APM because it can do more; that means it's harder to pick out the product's strengths at a glance, but those strengths are real.
That said, for organizations that outsource development, only run third-party packages or focus only on the cloud front end of applications, New Relic APM is an equally feasible option. Here, the APM tool comparison would be a more even match, and the product choice could come down to familiarity or personal taste.
Ultimately, APM isn't getting easier, as app dev teams build increasingly more complicated applications and deploy them on diverse hosting options. Even CIOs recognize the need to integrate code logging and profiling with traditional APM. In this case, developers are right in thinking that Retrace provides more of what they need.
Complexity in applications and their deployment options mandates richness and complexity in an APM tool. Thus, it's development and IT operations teams who lead the charge on tool selection. Management attitudes might lag behind the curve during the period of significant change, but CIOs should understand that dichotomy and work to bridge the gap.
