Getting Started with TDD in AWS

Getting Started with TDD in AWS

As a best practice, infrastructure in Amazon Web Services (AWS) should be provisioned utilizing infrastructure as code (IaC). In the past, ScaleSec has investigated the use of IaC in a number of blog posts, and have touted its upsides for provisioning cloud infrastructure. In this article, we will highlight the benefits of writing tests for infrastructure as code at a high level, and set the stage for future blog posts which examine this very content-rich topic.

Testing software helps find bugs before they are deployed to production, and can be done in a similar fashion for cloud infrastructure. In addition, writing tests before code can help organizations achieve their compliance requirements. Automated testing of IaC includes unit testing, integration testing, and functional testing - all which should be implemented in different quantities due to their nature.

What is TDD?

Test-Driven Development (TDD) is a software development approach where tests are written before writing the code that’s necessary for the test to pass. Code will go through a process called refactoring to pass the test, with the process being repeated for each description of functionality provided by the tests. This approach is commonly summarized as “red, green, refactor” - think about what to develop (red), think about how to make tests pass (green), and then think about how to improve these tests (refactor).

Infrastructure Context

TDD has traditionally been applied to software, and as infrastructure is now defined as software (e.g. IaC) we can begin to take this methodology into account when crafting code that defines infrastructure. When developing, we can first write tests based on what we want the infrastructure to do. There are a wealth of options, for example, let’s say that we want to ensure an autoscaling group has a minimum set to two EC2 instances to assure high availability. Next, we will write code to describe how that infrastructure will be provisioned. This is relatively straightforward, since for most use cases this is where the majority of IaC practitioners start. In Terraform:

resource "aws_autoscaling_group" "bar" {
  availability_zones = ["us-east-1a"]
  desired_capacity   = 2
  max_size           = 2
  min_size           = 2
}

A third and final (ongoing) step is to refactor the code (in this case Terraform) to improve this test. In Terraform, this is typically done by: finding more efficient ways to define resources, abstracting reusable code into modules, or simplifying configurations. Refactoring Terraform code is outside of the scope of this article, so it will be left as an exercise for the reader as to how this IaC can be improved upon.

Security Context

The question is then posed: how does TDD help organizations achieve security goals? For seasoned AWS security practitioners, it may already be apparent that security is all about developing secure infrastructure and processes around an AWS environment. From a risk assessment perspective, the total amount of risk exposure is decreased if the probability of an unfortunate event is decreased.

A test suite is a collection of test cases that are used to validate the behaviour of software. By adhering to TDD, a suite of tests will be created to validate the syntax, interactions, and functionality of infrastructure. By constantly running test suites and having them pass, an organization can validate that the necessary controls are in place to reduce risk.

Writing tests for infrastructure also helps by creating a “translation layer” between compliance requirements and the actual implementation of such requirements. One of the hardest challenges for organizations of varying sizes is bridging the gap between engineering and governance, risk, and compliance (GRC) teams. Tests provide a simple way to do this with the following steps:

1. Compliance requirements are mandated by the business.

   ex) NIST, CIS, FedRAMP, PCI, and so on.

2. Engineers building infrastructure on AWS strive to demonstrate adherence to benchmarks and compliance in these areas. As such, they can take these requirements and translate them into controls that should be followed for various types of resources.

   ex) AWS CIS Benchmark v1.2.0, 4.1: Ensure no security groups allow ingress from 0.0.0.0/0 to port 22.

3. Engineers can follow the “red, green, refactor” principles to practice TDD. First, write tests to demonstrate controls are in place. Secondly write the code to make these tests pass. Finally, iterate and enhance code as necessary. This entire process can run in an automated CI/CD pipeline with security checks that run the test suite.

   ex) Defining unit tests using a framework such as awspec to run against Terraform resources that define security groups and ensure they don’t have ingress rules allowing ingress from 0.0.0.0/0 to port 22.

As explored above, TDD doesn’t just help from a security engineering perspective, it helps from a GRC perspective as well as a risk management tool. TDD can help to ensure a secure AWS environment is in fact being described and applied in code and is continuously evaluated for compliance.

How to TDD?

With TDD, organizations can unlock additional benefits that treating infrastructure as software provides from both an engineering and security perspective (in other words, a DevSecOps perspective). Now it is time to explore how to effectively practice TDD, examine best practices, and identify pitfalls to watch out for.

Testing Pyramid

There are many different types of infrastructure testing, but in the interest of brevity we will investigate them in this article as unit tests, integration tests, and functional tests. When exploring these concepts, it’s useful to describe infrastructure as components, so the terminology will be interchanged throughout. Thinking of infrastructure as components under test can help to understand the intentions behind testing. To visualize how tests should look in a cloud environment, we will leverage the testing pyramid to display the differences between the three types of testing we will explore.

Unit tests validate the configuration or syntax of a single component in an AWS environment. Since these tests don’t typically (and shouldn’t) set up infrastructure to test, and instead examine the code or plan, these tests do not take a lot of time to make and run much faster. Instead of provisioning infrastructure, components are typically mocked - that is to emulate infrastructure behavior to focus on code functionality. Intuitively, this leads to the conclusion that there should be more unit tests than any other test. This leads to unit tests being the foundation of a good TDD strategy and the base of the unit testing pyramid.

Integration tests check that the output of one component matches the expected input to another one. Infrastructure usually needs to be provisioned in order to run this test, but it is typically only a few components that are under test to verify the contract (checking that output of one service matches the expected support to another service) between them. Since integration testing requires only small amounts of infrastructure to be live and are typically automated they comprise the middle part of the test pyramid. In other words, they should not be as plentiful as unit tests, yet should be more utilized than functional tests.

Functional tests verify the functionality of running infrastructure and ensure that expected results are returned without examining the code (also known as black-box testing). Many times, they take the form of API calls that would be sent to running infrastructure which might have databases with test data. Since the data flow from beginning to end is tested live, these types of tests are typically called end to end (E2E) tests. E2E tests can be automated, but are commonly manual as well (e.g. smoke tests). Since these tests are elaborate, requiring running infrastructure, test data, and the occasional manual test, they should be limited in nature, and are thus on the tip of the testing pyramid.

How to Get Started with TDD in AWS

There are many different testing frameworks that can be used to validate infrastructure in AWS. TerraTest uses the Go language to perform unit/integration/functional tests of infrastructure using helper libraries and patterns for common infrastructure types. AWSSpec is an implementation of RSpec (Ruby testing framework) testing specifications that can be used for integration testing of infrastructure. Chef InSpec is along a similar vein by using Ruby syntax in its testing framework to validate AWS infrastructure. For E2E tests, using a tool like Postman to collect API endpoints to call and verify expected results would be highly recommended.

It was noted earlier in the article that unit tests typically shouldn’t provision infrastructure and instead should be mocked. A great mocking framework for AWS is localstack, which is a Python library used to help assist in testing infrastructure and serverless applications.

Conclusion

Overall, security of infrastructure can be improved using methodologies described in this article. In future articles, there will be further exploration of unit tests, integration tests, and functional tests for cloud infrastructure. A shift in cloud security comes with a shift in mindset for organizations, and emphasizing TDD within an organization can help to enhance AWS security posture.