AWS_PROFILE is ignored if AWS_WEB_IDENTITY_TOKEN_FILE is set

[deniojunior-hotmart]

Actual Behavior

Running sops inside an GitHub self-hosted action it broke with the message below:

Group 0: FAILED
  arn:aws:kms:us-east-1:888888888888:alias/my-lambda: FAILED
    - | Error decrypting key: AccessDeniedException: The ciphertext
      | refers to a customer master key that does not exist, does
      | not exist in this region, or you are not allowed to access.
      | 	status code: 400, request id:
      | 2ec5457d-3ab5-4320-a258-31f67916a847

(fake account and lambda names)

I didn't set aws-profile when encrypting the file, aws_profile is empty ("aws_profile": "").

I'm using AssumeRoleWithWebIdentity and assuming a role called github-actions-runner-base-module-lambda-role containing the permissions to decrypt the file.

Locally, I was able to assume that role and perform encrypt and decrypt, so there's no problem with the role/permission.

Inside my github actions action I setup the configuration inside "$HOME"/.aws/config setting the profile target, such as below:

printf "[profile origin]\nrole_arn = arn:aws:iam::%s:role/%s\nweb_identity_token_file = %s" "$ACCOUNT_ID" "${ORIGIN_ROLE_NAME}" "$AWS_WEB_IDENTITY_TOKEN_FILE" > "$HOME"/.aws/config
printf "\n[profile target]\nrole_arn = arn:aws:iam::%s:role/%s\nsource_profile = origin\n%s" "${ACCOUNT_ID}" 

I logged inside the github actions pod and confirmed that the configuration was set properly.

Inside my action I run an aws cli command before running sops decrypt using the same profile (target) and it works smoothly:

AWS_PROFILE=target aws s3 cp s3://$BUCKET/cxsast_config.yaml .

It download the file from S3 correctly.

Right after running aws cli I run sops decrypt:

AWS_SDK_LOAD_CONFIG=1 AWS_PROFILE=target sops -d -i .env-staging

And sops command fails.

Workaround

After some time investigating, I have found that the issue was that sops was using AWS_WEB_IDENTITY_TOKEN instead of the profile set AWS_PROFILE=target.

To validate it I unset the AWS_WEB_IDENTITY_TOKEN_FILE env var right before executing the the sops decrypt command:

unset AWS_WEB_IDENTITY_TOKEN_FILE
AWS_SDK_LOAD_CONFIG=1 AWS_DEFAULT_PROFILE=target AWS_PROFILE=target sops -d -i .env-staging
export AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token

And it worked like a charm 😃

Possible Solution

Currently, sops load the credentials this way: https://github.com/mozilla/sops/blob/master/kms/keysource.go#L163-L219

I think it could be a good idea to considering using LoadDefaultConfig() from aws go sdk instead of creating a session. That's suggested by the AWS Go SDK documentation

The session package and associated functionality are replaced with a simplified configuration system provided by the config package. The config package is a separate Go module, and can be included in your applications dependencies by with go get.

The session.New, session.NewSession, NewSessionWithOptions, and session.Must must be migrated to config.LoadDefaultConfig.

It would be something close to this:

	if c.Profile != "" {
		cfg, err = config.LoadDefaultConfig(context.TODO(),
			config.WithRegion(c.Region), 
			config.WithSharedConfigProfile(c.Profile),
		)
	} else {
		cfg, err = config.LoadDefaultConfig(context.TODO(),
			config.WithRegion(c.Region), 
		)
	}

	if err != nil {
		level.Error(l).Log("err", err)
	}

	client := kms.NewFromConfig(cfg)

Sops Version

sops --version
sops 3.7.1

Tested with latest version (3.7.3 (latest)) and still facing the same issue.

References

May be related to some of the opened issues below:

• Enable sops with aws roles anywhere temporary credentials #1099 (comment)
• AWS Profiles not properly resolving the .aws/config file #679
• aws_profile doesn't work with profiles stored in ~/.aws/config #686

[deniojunior-hotmart]

This PR seems to apply the change suggested: #1179