Our Modernizing Security: AWS series has discussed a number of ways to improve security posture in an AWS environment. Building off some of these concepts, this article takes the perspective of an attacker and demonstrates one method of establishing a foothold in an account.
While impractical in a well instrumented environment, without proper monitoring and IAM segmentation this may well go undiagnosed for long enough to cause more than a headache. To be clear, this is not a vulnerability, per se, in the underlying technology but more so an abuse of existing functionality. Securing access keys and tokens is of the utmost importance and would mitigate this method from the outset. To start, we discuss the attack surface then the background and motivation for this method. A barebones demonstration is made using Terraform and we close with notes on key indicators and strategies for mitigation.
A Cornucopia of Shell
Assuming you have disabled user access to the console, one could still gain remote access to a server or resource through:
- SSH/RDP via unsecured/unmonitored network ingress.
- Remote execution via Simple Systems Manager.
- Server side code exploitation and privilege escalation to abuse loosely implemented machine roles.
- Direct API calls or CLI invocation.
These options all assume some direct or persistent pathway through the perimeter that could raise alarms in an environment with common rudimentary monitoring. Host based logging and agents could catch server exploits and basic Cloudtrail aggregation would at least make note of rogue API calls. What If there was a path to trigger arbitrary execution, a Lambda function or SSM automation for instance, that didn’t traverse the network perimeter and could otherwise be overlooked as normal intra-environmental machine-to-machine traffic? Well, we will get to that in a minute. First, a quick history lesson.
The Old Way
Before the likes of SDN and WAF, when SSH was a luxury, one was lucky if they had a basic firewall running and configured on each server.
Back when servers were more pets than cattle and firewalls were little smarter than electrified fences, admins went to great lengths to obfuscate and protect their remote access to individual servers. An obfuscation technique, port knocking, was developed earlier in this century that allowed a remote admin to authenticate themselves using a combination of specially crafted packets sent to a daemon that would “unlock” the server by adjusting firewall rules or starting a remote shell service.
The New Way
Could cloud native services be used to replicate the type of obfuscated authentication achieved with legacy port knocking? More critically, could a nefarious actor use such a pattern to conceal their command-and-control signals? Yes! First, we need a way to receive an arbitrary outside signal. Route53 provides a “highly available and scalable cloud Domain Name System DNS web service”, it should work just fine. Then that signal will need to be captured and routed into the environment. Route53 public record queries can be monitored through CloudWatch Logs and linked to an alarm with a metric filter. Action can be taken directly on this alarm through an SQS topic but the relatively new EventBridge service provides even greater configurability through more target options and robust event pattern matching. From here, the sky, or more specifically the credential, is the limit.
- Activate a lambda function
- Fire off an SSM Automation
- CloudFormation template update.
- Launch an EC2 or ECS task
- Trigger a CodePipeline, Step Function or Batch Job
######################### | |
## | |
## variables | |
## | |
######################### | |
variable zone_id {} | |
variable region { | |
default = "us-east-1" | |
} | |
# Use SMS for quick'n'dirty notification testing | |
variable notification_target {} | |
######################### | |
## | |
## terraform setup | |
## | |
######################### | |
terraform { | |
required_version = ">= 0.12" | |
backend "local" {} | |
} | |
provider "aws" { | |
alias = "us-east-1" | |
region = var.region | |
} | |
data "aws_caller_identity" "current" {} | |
######################### | |
## | |
## log groups, query, requisite policy | |
## | |
######################### | |
resource "aws_cloudwatch_log_group" "aws_route53_logrp" { | |
name = "/aws/route53/${var.zone_id}" | |
retention_in_days = 1 | |
} | |
# Example CloudWatch log resource policy to allow Route53 to write logs | |
# to any log group under /aws/route53/* | |
data "aws_iam_policy_document" "route53-query-logging-policy" { | |
statement { | |
actions = [ | |
"logs:CreateLogStream", | |
"logs:PutLogEvents", | |
] | |
resources = ["arn:aws:logs:${var.region}:${data.aws_caller_identity.current.account_id}:log-group:/aws/route53/*"] | |
principals { | |
identifiers = ["route53.amazonaws.com"] | |
type = "Service" | |
} | |
} | |
} | |
resource "aws_cloudwatch_log_resource_policy" "route53-query-logging-policy" { | |
provider = aws.us-east-1 | |
policy_document = data.aws_iam_policy_document.route53-query-logging-policy.json | |
policy_name = "route53-query-logging-policy-${var.zone_id}" | |
} | |
resource "aws_route53_query_log" "query_log" { | |
depends_on = [aws_cloudwatch_log_resource_policy.route53-query-logging-policy] | |
cloudwatch_log_group_arn = aws_cloudwatch_log_group.aws_route53_logrp.arn | |
zone_id = var.zone_id | |
} | |
######################### | |
## | |
## metric filter and alarm | |
## | |
######################### | |
resource "aws_cloudwatch_log_metric_filter" "cloudknocking-metric-filter" { | |
name = "cloudknocking-metric-filter" | |
pattern = "cloudknocking" | |
log_group_name = aws_cloudwatch_log_group.aws_route53_logrp.name | |
metric_transformation { | |
name = "KnockCount" | |
namespace = "CloudKnockingNamespace" | |
value = "1" | |
default_value = "0" | |
} | |
} | |
resource "aws_cloudwatch_metric_alarm" "cloudkncocking-alarm" { | |
alarm_name = "cloudknocking-alarm" | |
comparison_operator = "GreaterThanThreshold" | |
evaluation_periods = "1" | |
metric_name = "KnockCount" | |
namespace = "CloudKnockingNamespace" | |
period = "10" | |
statistic = "Sum" | |
threshold = "0" | |
alarm_description = "r53 query trip alarm" | |
## nonBreaching means green on the dashboard. | |
treat_missing_data = "notBreaching" | |
} | |
######################### | |
## | |
## event rule and target spec | |
## | |
######################### | |
resource "aws_cloudwatch_event_rule" "cloudknocking-rule" { | |
name = "CloudKnockingNamespace" | |
## not many detailed examples of event patterns out in the wild. | |
## for complex situations, refer to the Cloudwatch Event spec for a service | |
event_pattern = <<PATTERN | |
{ | |
"source": [ | |
"aws.cloudwatch" | |
], | |
"detail-type": [ | |
"CloudWatch Alarm State Change" | |
], | |
"resources": ["arn:aws:cloudwatch:${var.region}:${data.aws_caller_identity.current.account_id}:alarm:${aws_cloudwatch_metric_alarm.cloudkncocking-alarm.alarm_name}"], | |
"detail": { | |
"state": { | |
"value": ["ALARM"] | |
} | |
} | |
} | |
PATTERN | |
} | |
resource "aws_cloudwatch_event_target" "event_target" { | |
rule = aws_cloudwatch_event_rule.cloudknocking-rule.name | |
target_id = "SendToSNS" | |
arn = aws_sns_topic.notification_topic.arn | |
} | |
resource "aws_sns_topic" "notification_topic" { | |
name = "ck_topic" | |
} | |
resource "aws_sns_topic_policy" "default" { | |
arn = aws_sns_topic.notification_topic.arn | |
policy = data.aws_iam_policy_document.sns_topic_policy.json | |
} | |
data "aws_iam_policy_document" "sns_topic_policy" { | |
statement { | |
effect = "Allow" | |
actions = ["SNS:Publish"] | |
principals { | |
type = "Service" | |
identifiers = ["events.amazonaws.com"] | |
} | |
resources = [aws_sns_topic.notification_topic.arn] | |
} | |
} | |
resource "aws_sns_topic_subscription" "sms_subscription" { | |
topic_arn = aws_sns_topic.notification_topic.arn | |
protocol = "sms" | |
endpoint = var.notification_target | |
} |
As a simple demonstration, I’ve written some basic Terraform to wire everything together. For proof of concept, simply provide an existing Route53 hosted zone ID and valid number for SMS notification delivery. This code is provided without warranty or guarantee and is for demonstration purposes only.
Defending from the Threat Within
A good defensive posture always starts with an inventory of services in use or that you intend to use. If a service is not expected to be used in a particular account or organizational unit, use Service Control Policies to disable it completely. For services that cannot be disabled, anomaly detection through GuardDuty may be useful but regular audit of environments is a must. This should include right-sizing of IAM roles and review of deprecated or abandoned resources for potential inclusion in organization level deny lists.
Reactive control can only scale so far and at some point a proactive approach must be implemented. Consider creating a unified deployment pipeline for your wider environment including automated checks for unapproved services and overly permissive IAM role. It is much easier to lock down and monitor a cluster of integration/deployment servers that you trust to deploy a limited scope of hardened resources than it is to harden a wider set of bespoke infrastructure hand crafted by multiple teams with different end goals. Think of it as a “single path of permit” rather than a “single point of failure”. Infrastructure as code is your friend!
Connect with ScaleSec for AWS business
ScaleSec is an APN Consulting Partner guiding AWS customers through security and compliance challenges. ScaleSec’s compliance advisory and implementation services help teams leverage cloud-native services on AWS. We are hiring!
Connect with ScaleSec for AWS business.