The rise of AI has influenced almost every domain, including DevOps and SysOps. When AI is infused into tools that are used for systems management, they become more efficient and intelligent.
AIOps
Like other machine learning-based systems, AIOps relies on massive amounts of data. The metrics, logs and events captured from tens of thousands of machines help data scientists and ML engineers derive interesting insights through correlation.
Amazon is equipped with everything it takes to design an effective AIOps strategy for its infrastructure, operations, and management services. It has an army of researchers and engineers working on machine learning and artificial intelligence domains. The massive data center footprint that AWS has generates large datasets extremely useful for training models and implementing AIOps.
Here are five AWS services that exploit machine learning to deliver AIOps capabilities to customers:
Amazon Macie
Amazon Macie is one of the first AI-enabled services that help customers discover sensitive data stored in Amazon S3. It uses machine learning and pattern matching to automatically detect sensitive data types, including personally identifiable information (PII) such as names, addresses, and credit card numbers.
Apart from discovering standard data types such as SSN and credit card numbers, customers can use Macie to detect custom-defined data types using regular expressions such as employee ID, department codes, and other custom data types.
Amazon Macie is a one-click solution to gain visibility into the security and privacy of data stored in Amazon S3.
Predictive Scaling for Amazon EC2
Introduced in 2018, the predictive scaling feature can “automagically” shrink and expand an EC2 fleet’s size without DevOps’ manual intervention. This feature provides a hands-free approach to scaling, which is very different from manual or scheduled scaling.
AWS has added this feature as a simple checkbox to existing scaling options. Customers can optimize the predictions by fine-tuning a few parameters.
EC2 predictive scaling is extremely valuable for customers running workloads that experience regular spikes in usage. It fits in between the reactive scaling, which is triggered based on resource utilization, and proactive scaling, where customers manually schedule the scaling operation.
Amazon CloudWatch Anomaly Detection
Launched in 2009, Amazon CloudWatch is one of the oldest services from AWS. Last year, it finally got anomaly detection as an in-built feature. Powered by machine learning and over a decade of experience of Amazon engineers, CloudWatch Anomaly Detection has its roots in over 12,000 internal models from Amazon.
CloudWatch Anomaly Detection analyzes the historical values of a specific metric and looks for predictable patterns that repeat hourly, daily, or weekly. It then creates a best-fit model that will help customers better predict the future and cleanly differentiate normal and problematic behavior.
Customers can create CloudWatch Alarms that are automatically triggered when an anomaly is observed.
Amazon GuardDuty
Amazon GuardDuty is an intelligent threat detection service based on sophisticated machine learning algorithms. The service continuously monitors and protects AWS accounts, workloads, and data stored in Amazon S3.
According to AWS, GuardDuty analyzes tens of billions of events across multiple AWS data sources, such as AWS CloudTrail event logs, Amazon VPC Flow Logs, and DNS logs. With a few clicks in the AWS Management Console, GuardDuty can be enabled with no software or hardware to deploy or maintain.
Amazon GuardDuty protects customers’ AWS accounts, workloads, and data by identifying threats such as attacker reconnaissance, instance compromise, account compromise, and bucket compromise.
Amazon Detective
Amazon Detective is a service to investigate and identify the root cause of potential security issues or suspicious activities. It automatically collects log data from various AWS resources and uses machine learning, statistical analysis, and graph theory to help customers investigate security issues.
The service collects logs from VPC, CloudTrail and GuardDuty and organizes them into a graph model that is continuously updated. Since Amazon Detective maintains up to a year of aggregated data, it becomes easy to understand how it has changed over time.
Amazon Detective helps customers quickly perform root cause analysis in the event of a security breach or access violations.
