Impersonate service account

At this day and age it should be the default that one does not operate in cloud environment with Owner or Editorrole. You should always use the least amount of privileges possible and better yet, you should always be using the privileges your application is going to be running. It might feel burdensome at the beginning but you avoid many time consuming problems. First, you avoid “It worked when I run it” situations where things work great with your personal user but fail when running for example with service account under GKE with workload identity. Second you avoid security problems and accidental deletions or similar operator errors.

When working with GCP there is awesome feature called “Impersonate service account” which allows you to work as if you are the service account in question. Many of the GCP services utilize service accounts to provide credentials to workloads running inside Google Cloud. Perhaps the most obvious example is Compute Engine. By default Compute Engine uses service account created when you enable Compute Engine API. However you can provide dedicated service account - and you actually should do that - when implementing your workload. This way you can precisely control what your application can and cannot do. Service account impersonation allows you to act like that service account and know exactly how your workload is going to behave when running inside GCP even though you are running it on your own laptop.

Let’s first go through couple things you need to understand!

Google Application Default Credentials

Application Default Credentials or ADC in short is mechanism used by GCP SDK’s.

Consider the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
package main

import (
	"context"
	"flag"
	"fmt"
	"log"

	"cloud.google.com/go/storage"
	"google.golang.org/api/iterator"
)

func main() {
	projectID := flag.String("project", "", "Google Cloud project ID")
	flag.Parse()
	if *projectID == "" {
		log.Fatalf("Project ID is mandatory. Use the --project flag.")
	}
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		log.Fatalf("Failed to create google cloud client: %v", err)
	}
	defer client.Close()

	it := client.Buckets(ctx, *projectID)
	fmt.Println("Buckets:", *projectID)
	for {
		bucketAttrs, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatalf("Failed to list buckets: %v", err)
		}
		fmt.Println(bucketAttrs.Name)
	}
}

In this example highlighted lines interact with GCP and as we know all API calls must be authenticated and this is the place where ADC steps in. When you have ADC defined the credentials associated as ADC will be used in the API calls. If you use gcloud auth login and complete the login in browser window that opens the credentials you are acting will be your own and gsutil ls CLI command will act like it is actually you who issues the command. ADC is different.

By default the above code would fail due to lack of credentials. We must have ADC configured so SDK knows what credentials should be used when invoking the GCP API’s. We can authenticate for ADC with the following: gcloud auth application-default login but I wouldn’t recommend using this unless you’ve taken further actions discussed later in this post. This would associate your own credentials to be used when authenticating API calls. However the better way of handling ADC is to use impersonate the service account we are going to use when actually running our workload.

Why should we impersonate service accounts?

We are going to avoid one the very nasty caveats that plague most examples out there and that is the Organization policy called iam.disableServiceAccountKeyCreation. This policy prevents the creation of the JSON-formatted service account keys. Enforcing this policy is security best practice but it tends to cause endless amount of confusion among developers who are not that very well versed in the inner working of GCP.

First of all we have to understand that issuing gcloud auth application-default login we write credentials received to ~/.config/gcloud/credentials/application_default_credentials.json and you should take a look at it to really grasp what it includes:

1
2
3
4
5
6
7
8
9
{
  "account": "",
  "client_id": "000000000000000000000000000000000000000000000.apps.googleusercontent.com",
  "client_secret": "d-0000000000000000000000",
  "quota_project_id": "example-project",
  "refresh_token": "1//0c2aaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbcccccccccccccccccccccc",
  "type": "authorized_user",
  "universe_domain": "googleapis.com"
}

As you can guess this is very sensitive information. Having this key when you’ve logged in with your own credentials allows anyone act like it was you. Leaking this key is disastrous. The same goes for service account JSON keys. Leaking them is disastrous and that is the reason organizations prevent the key creation all together. You cannot leak something you do not have. Since our topic is securing GCP service accounts we won’t go into how you should secure your work laptop. It is enough to know that application_default_credentials.json should be handled with extreme care.

Since we need somekind of initial stepping stone there isn’t much we can do about ~/.config/gcloud/credentials/application_default_credentials.json. However we can affect how much power those credentials hold and how we use them. Ideally we would only use ADC credentials to impersonate the service accounts we tend to use. We want to avoid using our own credentials, we do not want to execute Terraform or run applications with them operating as us.

Impersonating with CLI

There are multiple ways of impersating service account. First, when using gcloud CLI you can add --impersonate-service-account=SERVICE_ACCOUNT_EMAIL where SERVICE_ACCOUNT_EMAIL is the email of your service account in the form of: <SERVICE_ACCOUNT_NAME>@<PROJECT_ID>.iam.gserviceaccount.com. However after about three commands it starts to become extremely irritating to always having to apped that in every command. Easier option is to use:

1
gcloud config set auth/impersonate_service_account SERVICE_ACCOUNT_EMAIL

so it is always assumed that you want to use impersonation.

The good thing with gcloud CLI is that you do not need to have ADC credentials. Invoking

1
gcloud iam service-accounts list --project <PROJECT_ID> --impersonate-service-account <SERVICE_ACCOUNT_NAME>@<PROJECT_ID>.iam.gserviceaccount.com

doesn’t need ADC. It will use your own credentials to first impersonate and then invoke API calls as impersonated service account.

Example:

1
2
3
4
5
> gcloud iam service-accounts list --project example-project-id --impersonate-service-account [email protected]
WARNING: This command is using service account impersonation. All API calls will be executed as [[email protected]].
WARNING: This command is using service account impersonation. All API calls will be executed as [[email protected]].
DISPLAY NAME                            EMAIL                                                         DISABLED
example-sa-00001                        [email protected]   False

you get warnings to highlight that it is actually the service account which is used to make the API calls.

Impersonating programmatically

furthermore the impersonation can also be done programmatically. So let’s revisit our bucket listing program and make some changes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
package main

import (
	"context"
	"flag"
	"fmt"
	"log"

	"cloud.google.com/go/storage"
	"golang.org/x/oauth2"
	"google.golang.org/api/iamcredentials/v1"
	"google.golang.org/api/option"
)

func main() {
	projectID := flag.String("project", "", "Google Cloud Project ID")
	serviceAccount := flag.String("impersonate", "", "Service Account to impersonate")
	flag.Parse()

	if *projectID == "" || *serviceAccount == "" {
		log.Fatal("Both --project and --impersonate flags are required.")
	}

	ctx := context.Background()
	token, err := getImpersonatedToken(ctx, *serviceAccount)
	if err != nil {
		log.Fatalf("Failed to get impersonated token: %v", err)
	}

	client, err := storage.NewClient(ctx, option.WithTokenSource(token))
	if err != nil {
		log.Fatalf("Failed to create storage client: %v", err)
	}
	defer client.Close()

	fmt.Printf("Bucket: %s:\n", *projectID)
	it := client.Buckets(ctx, *projectID)
	for {
		bucketAttrs, err := it.Next()
		if err != nil {
			if err.Error() == "iterator.Done" {
				break
			}
			log.Fatalf("Failed to list buckets: %v", err)
		}
		fmt.Println(bucketAttrs.Name)
	}
}

func getImpersonatedToken(ctx context.Context, targetServiceAccount string) (oauth2.TokenSource, error) {
	iamService, err := iamcredentials.NewService(ctx)
	if err != nil {
		return nil, fmt.Errorf("failed to create IAM client: %w", err)
	}

	req := &iamcredentials.GenerateAccessTokenRequest{
		Scope: []string{"https://www.googleapis.com/auth/cloud-platform"},
	}
	name := fmt.Sprintf("projects/-/serviceAccounts/%s", targetServiceAccount)

	resp, err := iamService.Projects.ServiceAccounts.GenerateAccessToken(name, req).Do()
	if err != nil {
		return nil, fmt.Errorf("failed to generate access token: %w", err)
	}

	return oauth2.StaticTokenSource(&oauth2.Token{
		AccessToken: resp.AccessToken,
	}), nil
}

Whoa! A lot of code but fortunately it is pretty simple. We’ve added a function that uses ADC to retrieve access-token for service account we specified via command line flag. The beauty of this is that it doesn’t matter whether the ADC is your own credentials or not. As long as ADC credentials are allowed to retrieve access-tokens for the service account we are good. We do not even need to know what is going on. This solution however doesn’t depend on the service account keys. This makes security teams happy.

IAM and service account impersonation

In order to be able to impersonate service accounts there are some IAM permissions that you need to have. The permission is iam.serviceAccounts.getAccessToken and it is included in the role roles/iam.serviceAccountTokenCreator. But! There is again small caveat! Do not go and assign this role on folder or project level! Doing that allows users to impersonate any service account under the project or folder (or sub-folder/-projects) and that is probably not what you actually want. What you should do is to assign roles/iam.serviceAccountTokenCreator on the service account itself.

1
2
3
4
gcloud iam service-accounts add-iam-policy-binding \
    SERVICE_ACCOUNT_EMAIL \
    --member="user:USER_EMAIL" \
    --role="roles/iam.serviceAccountTokenCreator"