AWS Log Blog

AWS Log Blog

AWS Log Blog

AWS Log Blog

Log Blog

As companies migrate to cloud providers, protecting valuable assets is of utmost importance. The Shared Responsibility Model says that organizations are responsible for monitoring access to sensitive data in the cloud, as well as what actions are being performed in their environment.

Logging, one of the Three Pillars of Observability, is one of the most powerful tools to provide the visibility to monitor a cloud environment. Traditional forensics cannot be performed on a Lambda, because the data and resources used for those activities are not available to the end user. It is impossible to access and analyze the memory data on an instance that only existed for a few minutes and is now terminated. Given this ephemeral nature, it is best to have robust logging enabled allowing for detection, automated response, and isolation.

With that in mind, what are some common sources for logs in AWS and when should they be enabled? In this blog, we will dive into some of the logging sources that can be leveraged for Detection in the Well-Architected Security Pillar [PDF]. Included for each is the type of log each source provides as well as how to centrally manage and store the files to meet compliance requirements.


CloudTrail Logs

CloudTrail should always be enabled in your environment. It is an AWS Service which provides audit logs for most services on the platform. These logs show what actions an entity performed, who that entity was, when the action was invoked, and to what resources. These actions are known as Management Events. CloudTrail can also be configured to log Data Events, which are S3 bucket and object focused, and Lambda function executions.

By default, 90 days of CloudTrail logs are searchable on the console. In order to retain the logs and import them into another system, create and configure a Trail to send to a S3 bucket and/or CloudWatch. Data Events (API calls against S3 Objects and Lambda invocations) are not searchable in the CloudTrail Console, so in order to view them, they must be sent to one of these places.

It is a best practice to enable an Organizational Trail which applies to all AWS accounts in the Organization. An organizational trail will ensure that CloudTrail is always enabled on Organizational accounts, as well as centralize the logs and allow scaling AWS accounts horizontally.

Not every AWS service sends events to CloudTrail, and not every action is logged in CloudTrail. This means there are some actions which cannot be accounted for in CloudTrail logs. Check here for each service’s CloudTrail support.

CloudTrail Logs
Log Type Audit
Organizations Support Yes
Long Term Storage Direct to S3, CloudWatch

CloudWatch Logs

CloudWatch is the catch all destination for AWS services’ logs. Lambda, RDS, API Gateway, Route53, and more are all stored in CloudWatch log groups, though not always by default. Sending logs to CloudWatch is usually a feature that must be enabled on the service.

By default, logs sent to CloudWatch are stored there indefinitely, but custom retention schedules can be configured on a per-log group basis. While it is cheaper to store the logs in S3, the CloudWatch-specific analysis tool CloudWatch Log Insights would not be able to index and search the full set of historical data. Each environment is different and there is likely a happy medium to be found between the storage tiers.

CloudWatch logs are portable. They can be subscribed to Kinesis streams, Kinesis Firehose, and streamed directly to ElasticSearch for analysis and action.

CloudWatch Logs
Log Type Audit / Metrics / Tracing
Organizations Support No - regional
Long Term Storage S3 via Kinesis Firehose, Cloudwatch

Config Logs

Ever wanted to track resources configuration changes that happen in an account? When enabled, Config will publish logs that list a resource’s current state and what has just changed. Use the logs to look into past configuration settings for resources.

Slightly more advanced usage of Config, called Config Rules, can be leveraged to create and enforce resource configuration baselines. For example, to ensure all EC2 instances have a required tag, Config Rules can monitor those resources. If the tag is not found, Config can remediate the instance. Config Rule capabilities are also available for Organizations so that rules can be deployed and managed to enforce centrally managed governance.

Config Logs
Log Type Audit / Configuration
Organizations Support Yes
Long Term Storage Direct to S3

VPC Flow Logs

To watch for suspicious network traffic and find points of latency in a VPC, enable VPC Flow Logs.

VPC Flow logs are not enabled by default, but when configured can encompass an entire VPC, a specific subnet, or an individual network interface. When enabling Flow logs, the fields included as well as the order in which they should appear are customizable.

Flow logs can be published to CloudWatch or directly to an S3 bucket for Athena querying or ingestion into a SIEM tool.

When analyzing VPC Flow logs to track the traffic to and from resources with IP addresses, for resources that reside within the VPC, the IPv4 address in the flow log will always be presented as the internal RFC1918 address even if the Elastic Network Interface has a public IP.

VPC Flow Logs
Log Type Audit / Metrics / Tracing
Organizations Support No
Long Term Storage Direct to S3

GuardDuty Logs

Leveraging VPC Flow logs, CloudTrail, and DNS logs, GuardDuty is threat detection as a service. VPC Flow logs do not need to be explicitly enabled for GuardDuty to leverage them. GuardDuty retrieves these logs directly from the VPC Flow log service. GuardDuty also has a backend connection to the internal AWS DNS resolvers, the logs of which are not exposed to AWS account owners.

GuardDuty generates findings and ranks them on a scale from 0.1 to 8.9, with higher values ranking the characteristics of the finding based upon security risk. These findings can also be used with Cloudwatch Event triggers to initiate incident response with Lambdas or a Security Orchestration, Automation, and Response (SOAR) tool.

See our blog on GuardDuty for a deep dive on its features.

GuardDuty Logs
Log Type Threat Intelligence
Organizations Support Yes
Long Term Storage CloudWatch Events to S3

EC2 Instance and Container Logs

While not an AWS-specific log type, logs from EC2 instances and containers are an important data source to enable. An ephemeral instance that performs a single task and then is destroyed cannot be isolated for memory dumps or disk images, so logs are of utmost importance.

When configured on a default AMI with Imagebuilder or through Systems Manager, Cloudwatch agents can send OS and application logs/metrics directly to a CloudWatch log group. Sending to a CloudWatch log group is supported with the CloudWatch agent, as well as EKS with Fluentd.

Common configurations for sending directly to S3 typically use tools like Fluentd or Logstash. If the main SIEM tool is Splunk, the Splunk Universal forwarder can also be configured to send directly to Splunk.

EC2 Instance Logs
Log Type Audit / Metrics / Tracing
Organizations Support No
Long Term Storage S3 via forwarder

S3 Access Logs

S3 Access logging provides insight on requests to access S3 objects. This is similar to CloudTrail Data Events, but S3 Access logs have a few failings in comparison.

There is slight overlap between the two types of logs on object and bucket actions, however S3 Access logs will provide additional fields.

If availability and timely notification is essential, it should be noted that S3 Access logs are best effort delivery and may experience delivery delays of a few hours. CloudTrail Data events are delivered every 5 minutes.

Choosing to enable and utilize one log type or both will be specific to the environment and the needs of the application. However for security events, CloudTrail will present the most timely and reliable source.

S3 Access Logs are sent directly to a specific logging bucket. Ensure Access logging is not enabled upon the Access logging bucket itself, or it will recursively log its own actions, which only ends once the AWS bill is noticed.

S3 Access Logs
Log Type Metrics / Tracing
Organizations Support No
Long Term Storage Direct to S3

AWS Web Application Firewall (WAF) Logs

AWS WAF defends Application Load Balancers, API Gateway RESTful APIs, and Cloudfront distributions against Cross Site Scripting, SQL injection, and more. With WAF Classic, logging was managed on an individual account basis. The latest version of AWS WAF, also referred to as WAFV2, introduces a new logging method that leverages Kinesis Firehose to stream to AWS Firewall Manager.

With Firewall Manager, WAFV2 can be centrally logged and managed. If WAF Classic resources exist in an environment, they should be migrated to WAFV2 in order to leverage the new features.

CloudFront and Elastic Load Balancing Logs
Log Type Audit / Metrics
Organizations Support Yes, with AWS Firewall Manager
Long Term Storage S3 via Kinesis Firehose

What about … ?

While this guide is not an inclusive list of log types in AWS, it has touched upon some of the major logging types on the platform. Most services in AWS expose logs in some form or another, and deciding which to enable and what use cases to address is an organizational, risk-based decision. Some of these log types can generate an extremely large amount of data, which costs money to store. The cost/benefit ratio on storing logs that are not used to find security issues is relatively low. Turning on logging sources should be a methodical process of log enablement, auto-remediation when possible, and response playbooks.

About ScaleSec

ScaleSec is a service-disabled, veteran-owned small business (SDVOSB) for cloud security and compliance that helps innovators meet the requirements of their most scrutinizing customers. We specialize in cloud security engineering and cloud compliance. Our team of experts guides customers through complex cloud security challenges, from foundations to implementation, audit preparation and beyond.

We are hiring!

Why Your YubiKey Won’t Work With AWS CLI (and the fix)

Set up a YubiKey as two-factor authentication in AWS that will work with the Console, CLI, and API - at the same time.

Next article

Here for you

Have questions? Leverage our expertise to help you meet your business goals with a strong security posture.

Join us

ScaleSec is a well-connected, fully remote team. We thrive in the great undocumented beyond. We’re hiring in most US metros.

Get in touch

Considering cloud? Want to optimize and transform your existing digital portfolio?
Reach out to us.

Gap Assessment

Get perspective. Address security comprehensively.

Prepare for compliance.

ScaleSec
San Diego, CA 92120, United States

619-SCALE15

© 2020 ScaleSec. All rights reserved. | Privacy Policy