Implementing Secure Code in the Cloud

As customers develop and migrate their workloads to the cloud, including refactoring to use serverless technologies, application layer security is more critical than ever. Access, encryption, and web application firewalls (WAF) are the usual controls mentioned for many cloud solutions. However, none of those are focused on code specific protections. Let’s explore additional application security layers to implement.

Insecure Code Example

To demonstrate some of the examples of secure coding practices, an intentionally insecure AWS Lambda using the Python 3.8 runtime will be utilized. The source code provided is below and is a simple handler that will take in user input to calculate mathematical expressions from API Gateway on the “queryStringParameters” portion of the JSON for the parameter, “calc”:

def lambda_handler(event, context):
  #pretend we have a valid API with a parameter of calc
  calcexpr = str(event["queryStringParameters"]["calc"])
  print("*****")
  print("This is the input from user: " +calcexpr)
  print("*****")

  mathchars = ['*', '+', '/', '**', '-', '(', ')']
  subsearch = any(i in calcexpr for i in mathchars)
  if len(calcexpr) == 0:
    print("Error, you need to enter something")
    raise("Null input")
  elif subsearch == False:
    print("Error, you didn't enter a valid math expression")
    raise AssertionError("Invalid Expression")
  elif subsearch == True:
    #dangerous eval without.
    result = str(eval(calcexpr))

return result

Notice in the above the use of a simple character pass list which validates that basic mathematical characters are part of the expression. The result is calculated using the eval function and then returned back to the caller. As mentioned, a simple API gateway sample event is used with the key value pair parameter added, calc and our expression. In this case, it’s “2+2” to get our return value:

Below is the expected result of 4 returned and our runtime cycles. Take note of the duration in milliseconds for a standard operation:

Command Injection

In this section the sample vulnerable code can be easily injected due to the use of the eval function without any additional input validation or security instrumentation. Instead of the standard math expression, the following payload will be injected into the parameter:

exec('import subprocess') or subprocess.run(['ls', '-l'], capture_output=True)

The exec function is also built into Python which will import the necessary libraries for us. The “OR” logic works because importing a library does not return a boolean “True”. This causes the imported library to run its subroutine normally. Simulating the injected payload produces the threat actor’s results:

In the above image the command is successfully executed and the results of existing working directory is returned to standard out. Another behavior to examine is that the runtime duration is now significantly greater than the regular use case.

Security Layer - Code Scanning

A method of preventing the vulnerable lambda code to begin with is to leverage Static Analysis Security Tools (SAST) which can scan for known vulnerable libraries, syntax, and patterns. In the particular case of Python, a tool such as Bandit can be used within the CI/CD pipeline to fail builds that don’t pass specific security policies.

Setting up Bandit as part of a scanning stage is trivial for scanning Python scripts as a one line statement:

mkdir bandit && cd bandit && python3 -m ‘venv’ ./ && source ./bin/activate && pip install bandit && bandit </path/to/code/>

A screenshot of Bandit running against the example code is shown below:

Note that while the confidence is high, the severity is only medium. Many SAST platforms will have policies that allow a return of ‘0’ that can be tuned to meet organizational needs. For example; the policy may be acceptable to pass a build that only has no high severity findings, but can have X number of medium and low findings. Had this been the policy at an organization for the example code, the build would have passed. Additional layers of protection should be used beyond the CI/CD pipeline.

Security Layer - Libraries and Scoping

As part of proper input validation in code, developers should also leverage the use of known secure libraries and scope their functions to only the permitted actions or calls necessary. Let’s modify the code to leverage Python’s scoping capabilities for the eval function:

elif subsearch == True:
  #dangerous eval without.
  #result = str(eval(calcexpr))

  #more secure using module filters
  result = str(eval(calcexpr,{"__builtins__":None}))

The code above will prevent the existing code injection test payload by removing unnecessary functionality:

In the screenshot above the injection causes the run to error out. However, there is a problem. Scanning the code through Bandit or another SAST tool may still fail the build because of the use of the less secure eval function as seen below:

The recommendation from the scanner is directing the developer to consider the use of ast.literal_eval which can be more secure. However, the secure library the example code will work best with is the library seval which still allows for mathematical expressions to be evaluated unlike Python 3.7+ runtimes of ast.literal_eval.

The simple implementation of seval is shown below for a complete picture:

result = seval.safe_eval(calcexpr)

Simulating the injected payload again gives the expected error status but something interesting also happens, unlike the simple builtin function scoping filter; the runtime billed is only 5 milliseconds compared to the 200+ milliseconds for a failed and successful command injection; shown below:

Using Bandit to scan the code manually or through a pipeline with the new secure eval function results in a pass:

Based on what is known so far, incorporating a SAST in the pipeline along with secure libraries can help mitigate issues between releases and potentially increase performance of the lambda, resulting in cost savings.

Security Layer - Instrumentation and RASP

Another layer to consider is the use of instrumentation and Runtime Application Self-Protection (RASP) technologies. Instrumentation is very powerful as it can log different metrics, errors, and trace functions during runtime. Developers may already use instrumentation for the purpose of performance testing and monitoring within their solution.

Below is an example of modifying a Flask based application monitoring for inbound requests using the AWS X-Ray service. The full documentation on the Python SDK can be found here.

from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware

app = Flask(__name__)

xray_recorder.configure(service='My application')
XRayMiddleware(app, xray_recorder)

How does instrumentation relate to security? Recall in the previous sections that each execution of the lambda had different duration times depending on the payload used. The injected payload ran much longer than the harmless payload. In AWS X-Ray for Lambda, the following trace metrics occur when executing both types of payload using the secure eval function:

Notice the difference in trace status. The trace with the error of “4XX” was when the secure eval function blocked the injection. The trace with “OK” was the harmless payload. The trace provides more details than the simple “HTTP 200 OK” response code. Over time and if metrics were set for acceptable thresholds, security and developer teams can investigate anomalies at runtime using inference. This is similar to how network and security analysts infer data using netflow or IPFIX traffic.

In the same manner of modifying code, RASP can also be leveraged to help detect or block threat activity at runtime. In Python, “instrumenting” a RASP tool involves importing a library, and then using a property decorator so that the RASP can call the function “as-is” without further modification.

A syntax example of implementing a RASP library in the example lambda code is provided below:

import my_rasp
from my_rasp import protect_this_handler
@protect_this_handler
def lambda_handler(event, context):

There is a caveat to keep in mind when using RASP. Many RASP solutions offered are expert systems just like many WAFs and anti-malware solutions. There are known signatures and there could be some heuristics built in. While the technology itself is another layer, customers should always have their solutions tested using qualified penetration testers.

At the time of this writing running injected payload against the example code did not yield detection results with one vendor using default tuning:

More tuning was required which included setting additional detection timeout thresholds in the lambda environment variables for adequate analysis to be performed and adding 4x the memory into lambda:

When evaluating a RASP, ensure that there is some form of Machine Learning (ML) functionality to help predict and evaluate normal and abnormal conditions outside of signature parameters. RASP is a very useful security tool to protect applications, but like every other control, should be part of multiple layers.

Summary

Building security in layers should not be limited to infrastructure and operational controls alone. A robust program also includes adding security to developer operations (DevOps) practices which can include code scanning, secure libraries, scoping, instrumentation, and RASP solutions. Leveraging multiple application security specific layers is the ideal approach in implementing defense-in-depth design.