Access GCP from AWS using Workload Identity Federation

No more GCP Service Account Keys!

GCP service account keys are a security risk. They can be hardcoded in source code, pushed to public repositories like GitHub, shared between users, and are generally long-lived without expiration dates. For an application to access GCP resources or services from outside of the platform, you need a downloaded service account key. For example, if you’re running a multi-cloud environment or are a SaaS provider that runs infrastructure in AWS, you didn’t have any options other than to use those unsafe service account keys… until now. Google has released a new service called Workload identity federation with the aim to remove the service account key burden and provide ephemeral, short-lived credentials to access GCP services and resources from outside of GCP.

Currently in Beta, this service provides support for AWS, Azure, and OIDC identity providers. This blog will focus on the AWS functionality and aims to provide clear and concise directions on how to configure your AWS applications to authenticate safely to GCP with no keys necessary! We have written a Python module to federate access with two lines of code.

Workload Identity Federation Overview

Before we dive into the AWS specifics of authenticating to GCP using workload identity federation, we first need to understand how the service operates. If you are already familiar with workload identity federation, feel free to skip to section Accessing GCP from AWS.

Workload identity federation contains two components: workload identity pools and workload identity providers. Workload identity pools, as the name suggests, is a logical container of external identities (AWS roles, Azure managed identities, etc.). Workload identity providers are the entities that contain the relative metadata about the relationship between the external identity provider (AWS, Azure. etc.) and GCP. For example, providers can contain information like AWS account IDs, IAM role ARNs, etc.

In addition to the above components, there is also the concept of Attribute mappings. Attributes are metadata attached to the external identity token (more on the tokens below) that supply information to GCP via attribute mappings. Attributes can be combined with conditions to secure your tokens so that they can only be used by approved external identities during the Workload identity federation process. Examples of attributes include name, email, or user ID. Complex attribute conditions are outside of the scope of this blog but we do provide an example to get you started.

Accessing GCP from AWS

This section provides an overview of how you can assign credentials and authenticate to GCP from Amazon Web Services (AWS) without the use of service account keys. For our example we’ll be using a fictional application called “GCP CopyCat” that is running on an EC2 instance in AWS that copies files from a GCS bucket for analysis.

Before we can dive into our token exchange process we need to configure our AWS and GCP resources. We will be using the gcloud CLI for the next sections.

Step 1: GCP Service Account Creation

The GCP service account will be what our AWS application will authenticate with in order to access GCS objects. For the below command substitute your own values where it has a $.

gcloud iam service-accounts create $SERVICE_ACCOUNT_ID \
     --description="$DESCRIPTION" \
     --display-name="$DISPLAY_NAME"

For example:

gcloud iam service-accounts create gcp-copycat \
     --description="AWS application that copies GCS objects." \
     --display-name="gcp-copycat"

For GCP CopyCat we have added the Storage Object Viewer role to our GCP bucket for the newly created service account.

Step 2: AWS IAM Role Creation

Now we need to create an AWS IAM role that the GCP CopyCat application will assume while running on an EC2 instance. This role will be what we use to exchange tokens with GCP and access our GCS bucket.

For the below command substitute your own values where it has a $.

aws iam create-role --role-name $ROLE-NAME  \
     --assume-role-policy-document $FILE-LOCATION  \
     --description "$DESCRIPTION"

For example:

aws iam create-role --role-name "gcp-copycat"  \
     --assume-role-policy-document "/Users/copycat/assume.json"  \
     --description "Allows EC2 instances to copy GCS objects"

A successful response will look like:

{
  "Role": {
    "AssumeRolePolicyDocument": "<URL-encoded-JSON>",
    "RoleId": "AKIAIOSFODNN7EXAMPLE",
    "CreateDate": "2021-02-08T20:43:32.821Z",
    "RoleName": "gcp-copycat",
    "Path": "/",
    "Arn": "arn:aws:iam::123456789012:role/gcp-copycat"
  }
}

Note: The /Users/copycat/assume.json file above must allow our EC2 instance to assume the role.

All that’s left is to attach an IAM policy to our newly created IAM role. Same process as before:

aws iam put-role-policy --role-name $ROLE-NAME  \
     --policy-name $EXAMPLE-POLICY  \
     --policy-document $FILE-LOCATION

For example:

aws iam put-role-policy --role-name "gcp-copycat"  \
     --policy-name "gcp-copycat-policy"  \
     --policy-document "/Users/copycat/role.json"

An example role.json could be:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CopyObjectBucket",
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::mybucket/*",
 "arn:aws:s3:::mybucket"]
    }
  ]
}

Note: We did not provide steps here on creating the EC2 instance profile. For further details please refer to AWS documentation.

Step 3: Workload Identity Pool Creation

Next is our workload identity pool creation. As explained earlier, workload identity pools are logical containers for external identities, like AWS roles.

Note: ScaleSec recommends you to isolate your pools by AWS environment. For example, one GCP workload identity pool for your AWS development environment and a separate isolated pool for your production environment.

For the below command substitute your own values where it has a $.

gcloud beta iam workload-identity-pools create $POOL-ID \
     --location="global" \
     --description="$DESCRIPTION" \
     --display-name="DISPLAY-NAME"

For example:

gcloud beta iam workload-identity-pools create gcpcopycatpool \
     --location="global" \
     --description="Workload identity pool for GCP copycat." \
     --display-name="GCP CopyCat Pool"

Step 4: Workload Identity Provider Creation

Now that we have our workload identity pool, GCP service account, and AWS IAM role, we need to create our AWS identity provider in GCP. The below example contains the flag --attribute-condition and is not required to create an identity pool provider. We have included this additional security configuration as an example of one way to restrict GCP token generation.

gcloud beta iam workload-identity-pools  \
     providers create-aws $PROVIDER_NAME  \
     --location="global"  \
     --workload-identity-pool="$VALUE_FROM_STEP_3"  \
     --display-name="$DISPLAY_NAME"  --description="$DESCRIPTION"  \
     --attribute-condition="$EXAMPLE_CONDITION"  \
     --account-id=$AWS_ACCOUNT_ID

For example:

gcloud beta iam workload-identity-pools  \
     providers create-aws gcpcopycatprovide  \
     --location="global"  \
     --workload-identity-pool="gcpcopycatpool"  \
     --display-name="GCP CopyCat AWS Provider"  \
     --description="The Identity Provider for GCP CopyCat"  \
     --attribute-condition="'arn:aws:sts::123456789012:assumed-
role/gcp-copycat' == attribute.aws_role"  \
     --account-id=123456789012

Step 5: GCP Service Account Impersonation Binding

The next step in our configuration is to provide the AWS role the ability to impersonate our GCP service account. We do this by binding the IAM role roles/iam.workloadIdentityUser to the GCP service account. Keep in mind that the below gcloud command expects the GCP Project NUMBER which is different from your PROJECT ID.

gcloud iam service-accounts add-iam-policy-binding
$workload_sa_email \
   --role roles/iam.workloadIdentityUser \
   --member 
"principalSet://iam.googleapis.com/projects/$gcp_project_number/l
ocations/global/workloadIdentityPools/$workload_id/attribute.aws_
role/arn:aws:sts::${aws_account_id}:assumed-role/$role_name"

For example:

gcloud iam service-accounts add-iam-policy-binding gcp-
copycat@example-project.iam.gserviceaccount.com  \
     --role "roles/iam.workloadIdentityUser"  \
     --member 
"principalSet://iam.googleapis.com/projects/123456789012/location
s/global/workloadIdentityPools/gcpcopycatpool/attribute.aws_role/
arn:aws:sts::123456789012:assumed-role/gcp-copycat"

Step 6: Enable Required GCP Services

Finally, we need to enable the Google Security Token Service and the IAM credentials service in our GCP project.

gcloud services enable sts.googleapis.com && gcloud services 
enable iamcredentials.googleapis.com

That’s it for our configuration steps. In the next section we will execute our token exchange and download our GCS objects.

Workload Identity Federation Token Flow

For example request/response bodies and documentation visit the official GCP docs.

Note: Typically you do not need to explicitly generate temporary credentials. The AWS SDK, CLI, etc. automatically get the credentials for you from the instance metadata. This is only required in this instance to use Workload identity federation to authenticate to GCP.

2. To begin the token exchange process you first need to generate temporary security credentials for AWS. This can be done with the AssumeRole or GetSessionToken APIs depending on your use case. We are using an AWS role for our example so we need to use the AssumeRole API.

3. Using the credentials returned by AssumeRole, we can create a GetCallerIdentity token. This token is similar to what we send in a request with the STS GetCallerIdentity() method. You do not need to call GetCallerIdentity() from your code because we are sending the token to GCP’s Security token service in the next step.

4. Now that we have our GetCallerIdentity token, we can exchange this for a GCP federated access token. This is done via a HTTP POST call to the STS url: https://sts.googleapis.com/v1beta/token

5. The GCP STS returns a federated access token which can only be used for a limited number of GCP services for authorization. In order to interact with all of the GCP services and API endpoints, we need a service account access token.

6. The service account access token is an OAuth 2.0 credential that can be used for GCP authorization in our GCP CopyCat application. We exchange our federated access token via a HTTP POST to the Cloud IAM URL: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$SA-NAME@$PROJECT-ID.iam.gserviceaccount.com:generateAccessToken

7. Now that our _GCP CopyCa_t application has an ephemeral service account access token, we can make calls to the GCS API without the need of a static downloaded service account key! GCP CopyCat only needs to include the newly acquired access token in our authorization header for the API requests in order to copy files from GCS.
   curl -H "Authorization: Bearer $OAUTH_TOKEN" "https://storage.googleapis.com/storage/v1/b/$BUCKETNAME/o"

Note: The target service account in GCP must have the proper GCS permissions in order to copy files. For more examples, visit the examples folder in our GitHub repository.

Workload Identity Federation Module

We have created a Python module to take care of the steps outlined above with just two lines of code!

First, create a virtualenv:

Python3 -m venv .venv
source .venv/bin/activate

Next, install the package:

pip install scalesec-gcp-workload-identity

Set up your cloud credentials:

export AWS_PROFILE=xyz
gcloud auth login

Now you are ready to import the module and get a token:

from scalesec_gcp_workload_identity.main import TokenService

token_service = TokenService(args...)
sa_token, expiry_date = token_service.get_token()

Conclusion

Workload identity federation is a boon for multi-cloud organizations in AWS and GCP. Using GCP service account keys involves risk and effort when rotating and/or sharing keys, something that is endemic to all static credentials. By leveraging workload identity federation with this module, organizations can optimize, manage, and secure their communications between AWS and GCP.

Special thanks to Aidan Steele (@__steele) for volunteering his time and technical contributions.