Figure out a scalable way to reference backend artefact IDs into custom lambda code

[bogris]

Environment information

System:
  OS: macOS 15.0
  CPU: (8) arm64 Apple M1 Pro
  Memory: 141.11 MB / 16.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node
  Yarn: undefined - undefined
  npm: 10.2.3 - ~/.nvm/versions/node/v20.10.0/bin/npm
  pnpm: 8.9.0 - ~/.npm-packages/bin/pnpm
NPM Packages:
  @aws-amplify/auth-construct: Not Found
  @aws-amplify/backend: 1.0.4
  @aws-amplify/backend-auth: Not Found
  @aws-amplify/backend-cli: 1.2.2
  @aws-amplify/backend-data: Not Found
  @aws-amplify/backend-deployer: Not Found
  @aws-amplify/backend-function: Not Found
  @aws-amplify/backend-output-schemas: Not Found
  @aws-amplify/backend-output-storage: Not Found
  @aws-amplify/backend-secret: Not Found
  @aws-amplify/backend-storage: Not Found
  @aws-amplify/cli-core: Not Found
  @aws-amplify/client-config: Not Found
  @aws-amplify/deployed-backend-client: Not Found
  @aws-amplify/form-generator: Not Found
  @aws-amplify/model-generator: Not Found
  @aws-amplify/platform-core: Not Found
  @aws-amplify/plugin-types: Not Found
  @aws-amplify/sandbox: Not Found
  @aws-amplify/schema-generator: Not Found
  aws-amplify: 6.4.4
  aws-cdk: 2.151.0
  aws-cdk-lib: 2.151.0
  typescript: 5.5.3
AWS environment variables:
  AWS_APP_ID = awsamplifygen2
  AWS_BRANCH = sorin-sandbox-d1434647ce
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables
npm notice
npm notice New minor version of npm available! 10.2.3 -> 10.8.3
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.8.3>
npm notice Run `npm install -g npm@10.8.3` to update!
npm notice
npm notice 
npm notice New minor version of npm available! 10.2.3 -> 10.8.3
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.3
npm notice Run npm install -g npm@10.8.3 to update!
npm notice

Describe the feature

get in code amplify generated resource ids like:

• user pool ID
• model source Dynamo db tablename
• etc

Use case

When writing custom lambda code, a lot of time we need direct access to resources in order to accomplish advanced tasks.

In my current iteration, I managed to encapsulate this logic in a way that it works in multiple branches, including sandboxes even in the same account.

I took the ideea behind the backend "secret" logic.

I defiend a custom type to give me the published "short names" for the ids.

import { Schema } from "./data/resource";

export const awsAppId = process.env.AWS_APP_ID ?? "undefined";
export const awsBranch = process.env.AWS_BRANCH ?? "undefined";

const ssmPrefix = `/amplify/custom/${awsAppId}/${awsBranch}`;
export const runtimeSsmPrefix = `/amplify/custom/${process.env.AWS_APP_ID}/${process.env.AWS_BRANCH}`;

export const getSsmTableName = (tableName: string) => {
  return `${ssmPrefix}/DYNAMO_TABLE_${tableName}`;
};

export type AmplifyResources =
  | "AMPLIFY_AUTH_USERPOOL_ID"
  | "AMPLIFY_STORAGE_BUCKET_NAME";
export const getSsmAmplifyResourceName = (resourceName: AmplifyResources) => {
  return `${ssmPrefix}/${resourceName}`;
};

export type DataModels = {
  [K in keyof Schema]: Schema[K]["__entityType"] extends "model" ? K : never;
}[keyof Schema];

type ModelsParamNames = `DYNAMO_TABLE_${DataModels}`;

export type AllCustomSsmPArams = ModelsParamNames | AmplifyResources;

i create the ssm params:

Object.entries(backend.data.resources.tables).map(([shortName, table]) => {
  const ssmParamName = getSsmTableName(shortName);
  new ssm.StringParameter(customParamsStack, ssmParamName, {
    parameterName: ssmParamName,
    stringValue: table.tableName,
  });
});

const userPoolIdSsmName = getSsmAmplifyResourceName("AMPLIFY_AUTH_USERPOOL_ID");
new ssm.StringParameter(customParamsStack, userPoolIdSsmName, {
  parameterName: userPoolIdSsmName,
  stringValue: userPoolId,
});
//storage bucket name
const storageBucketName = backend.storage.resources.bucket.bucketName;
const storageBucketNameSsmName = getSsmAmplifyResourceName(
  "AMPLIFY_STORAGE_BUCKET_NAME"
);
new ssm.StringParameter(customParamsStack, storageBucketNameSsmName, {
  parameterName: storageBucketNameSsmName,
  stringValue: storageBucketName,
});

and in the code, I have a helper function that queries ssm:

const region = process.env.AWS_REGION || "eu-central-1";

export const getAmplifyParam = async (param: AllCustomSsmPArams) => {
  const ssmClient = new SSMClient({ region });

  const Name = `${runtimeSsmPrefix}/${param}`;
  const command = new GetParameterCommand({ Name });
  const response = await ssmClient.send(command);
  return response.Parameter?.Value;
};

in order for the function to know what to get, i add 2 env vars to the handler:

export const externalRestApi = defineFunction({
  name: "externalRestApi",
  entry: "./handler.ts",
  environment: {
    AWS_APP_ID: awsAppId,
    AWS_BRANCH: awsBranch,
  },
});

and now in the source code i get a types function that returns my info:

 const DYNAMO_TABLE_Organization = await getAmplifyParam(
    "DYNAMO_TABLE_Organization"
  );

I tried to use the amplify JSON in the function Env param to define the list of available params, but I don;t have acces to that without a circular dependency.

This is where you guys can complete the loop in my mind.

I think this can be an extension of the way you publish GRAPQL params for the functions that are auth in the schema to query it.

to make this work, I also published 2 env params before starting my sandbox. This can also be formalised so that sandboxes can behave more like branches to adhere to a formalised naming convention for ssm params.

this is my package.json where I start the sandbox:

  "scripts": {
     ...
    "sandbox": "export AWS_APP_ID=awsamplifygen2 && export AWS_BRANCH=$(whoami)-sandbox && npx ampx sandbox --stream-function-logs"
  },

I did not put in here all the imports and authorizations to get params etc. I tried to stick to the core code.

[ykethan]

Hey @bogris, thank you for taking the time in filing this issue with the details. Marking this as feature request to access resource context. Additionally, this may help elevate some the circular dependency issues being tracked on #1850