Monitoring Modern Apps is a Problem Only Analytics Can Solve
Monitoring is an analytics problem, especially in modern distributed architectures that use microservices and other tools, such as container technologies. That’s evident when talking to people such as Karthik Rau, CEO and founder of SignalFx, the service for monitoring modern applications.
The roots of the SignalFx technology come from the work Co-Founder Phillip Liu did as a Facebook software engineer, working on a team that built systems management tools for the rapidly growing social network. In those days, much of Facebook’s experience came from developing their own tools to manage the ever growing scope of its distributed architecture. The problems they encountered are similar to the problems many companies face today.
SignalFx uses a streaming analytics approach to identify trends. In a demo, Rau showed the SignaFx interactive analytics pipeline to get a real-time view of aggregated data from multiple data sources that are instrumented and pulled in through standard APIs. Metrics from collectd, StatsD, Metrics or anything else can be used, or the customer can write directly to the SignalFx API. SignalFx also offers a streamlined configuration of collectd and Java client libraries to get immediate visibility. It also integrates with AWS to pull in CloudWatch metrics and AWS metadata. There are also integrations with PagerDuty, HipChat, Slack and webhooks that provide outbound alerts and notifications.
The pipeline gives the ability to look at issues as if, for example, a trend occurring at 6 a.m. is normal or should be explored. Customers send data that allows for filtering to make comparisons, such as latency across different regions or how latency compares to transaction requests. Most customers instrument as much as possible to get a complete picture.
SignalFx categorizes metrics into infrastructure, application or business metrics:
Infrastructure
- CPU percent of a server.
- Number of bytes of free disk space.
- Number of bytes transmitted on a network interface.
Applications
- Response time in milliseconds of an API call.
- Cache hit ratio of an internal cache used in a service.
- Percent of time spent on garbage collection by a JVM instance.
Business
- Number of unique users who logged in over the last day.
- Number of new customers who signed up in the last hour.
- Week over week percentage growth in revenue.
SignalFx processes metrics data and related events (like code pushes) that customers use to annotate or overlay onto charts or dashboards.
A particular instance type can be analyzed and re-rendered on the fly. It gives an aggregate view that allows you to send alerts on aggregate metrics, for example on growth rates.
The number of apps that customers deploy and manage means a greater interest in microservices. Cusstomers that use multiple microservices can monitor upstream and downstream dependencies with SignalFx. Each service might have dozens of components, requiring an aggregate view, as opposed to examining one component at a time.
SignalFx is a sponsor of The New Stack.
