-
Notifications
You must be signed in to change notification settings - Fork 13
feat: formation #550
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
thantos
wants to merge
26
commits into
main
Choose a base branch
from
sussman/node_cfn
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
feat: formation #550
Changes from 9 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
c3aa9a0
deployments work except for asset upload
thantos 84c235e
Merge remote-tracking branch 'upstream/main' into sussman/node_cfn
thantos 3c8c86c
stuff
thantos 1e3dc13
remove cloud control api
thantos 62f5643
add timing
thantos d9e930e
support flag to swtich between CFN and NODE_CFN
thantos 507c68d
Merge branch 'main' into sussman/node_cfn
thantos a4b266f
formation
thantos 7596931
upgrade lerna
thantos 414e497
delete me
thantos 1ef17f0
delete me
thantos 53aa025
condition graph
thantos 8281844
more work on formation plan
thantos 77ebdc3
progress on printing the plan
thantos 3eff83e
refactor things
thantos 47e22d9
template resolver and skip update logic
thantos a5ed5f7
abstract out resource operation
thantos 5470e7c
separate skip update output and refactor graph to use template resolver
thantos be65a68
topo sort filtering
thantos 4db37c9
compute maybe updates
thantos 64cae8b
move plan display
thantos 75c6f98
extract plan executor
thantos e9be09d
add simple detailed mode for plan
thantos fe09d7b
waiter bug
thantos 3d7291a
plan as table
thantos 446d5c2
stuff
thantos File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,391 @@ | ||
| import * as fs from "fs"; | ||
| import { join } from "path"; | ||
| import { typescript, TextFile, Project } from "projen"; | ||
| import { GithubCredentials } from "projen/lib/github"; | ||
| import { Job, JobPermission } from "projen/lib/github/workflows-model"; | ||
|
|
||
| /** | ||
| * Adds githooks into the .git/hooks folder during projen synth. | ||
| * | ||
| * @see https://git-scm.com/docs/githooks | ||
| */ | ||
| class GitHooksPreCommitComponent extends TextFile { | ||
| constructor(project: Project) { | ||
| super(project, ".git/hooks/pre-commit", { | ||
| lines: ["#!/bin/sh", "yarn run typecheck", "npx -y lint-staged"], | ||
| }); | ||
| } | ||
|
|
||
| public postSynthesize() { | ||
| fs.chmodSync(this.path, "755"); | ||
| } | ||
| } | ||
|
|
||
| const MIN_CDK_VERSION = "2.43.1"; | ||
|
|
||
| /** | ||
| * Projen does not currently support a way to set `*` for deerDependency versions. | ||
| * https://github.com/projen/projen/issues/1802 | ||
| * | ||
| * Why do we need `*` for peer dependencies? | ||
| * | ||
| * @aws-cdk 2.0 uses pre-release version tags (ex: 2.17.0-alpha.0) for all experimental features. | ||
| * NPM/Semver does not allow version ranges for versions with pre-release tags (ex: 2.17.0-alpha.0) | ||
| * | ||
| * This means we cannot specify a peer version range for @aws-cdk/aws-appsync-alpha, pinning consumers to one CDK version | ||
| * or ignoring/overriding npm/yarn errors and warnings. | ||
| * | ||
| * TODO: Remove this hack once https://github.com/projen/projen/issues/1802 is resolved. | ||
| */ | ||
| class CustomTypescriptProject extends typescript.TypeScriptProject { | ||
| constructor(opts: typescript.TypeScriptProjectOptions) { | ||
| super(opts); | ||
|
|
||
| new GitHooksPreCommitComponent(this); | ||
|
|
||
| this.postSynthesize = this.postSynthesize.bind(this); | ||
| } | ||
|
|
||
| public postSynthesize() { | ||
| super.postSynthesize(); | ||
|
|
||
| /** | ||
| * Hack to fix peer dep issue | ||
| */ | ||
|
|
||
| const outdir = this.outdir; | ||
| const rootPackageJson = join(outdir, "package.json"); | ||
|
|
||
| const packageJson = JSON.parse( | ||
| fs.readFileSync(rootPackageJson).toString("utf8") | ||
| ); | ||
|
|
||
| const updated = { | ||
| ...packageJson, | ||
| peerDependencies: { | ||
| ...packageJson.peerDependencies, | ||
| "@aws-cdk/aws-appsync-alpha": "*", | ||
| }, | ||
| }; | ||
|
|
||
| fs.writeFileSync(rootPackageJson, `${JSON.stringify(updated, null, 2)}\n`); | ||
| } | ||
| } | ||
|
|
||
| const assumeRoleStep = { | ||
| name: "Configure AWS Credentials", | ||
| uses: "aws-actions/configure-aws-credentials@v1", | ||
| with: { | ||
| "role-to-assume": | ||
| "arn:aws:iam::593491530938:role/githubActionStack-githubactionroleA106E4DC-14SHKLVA61IN4", | ||
| "aws-region": "us-east-1", | ||
| "role-duration-seconds": 60 * 60, | ||
| }, | ||
| if: `contains(fromJson('["release", "build", "close"]'), github.workflow)`, | ||
| }; | ||
|
|
||
| const project = new CustomTypescriptProject({ | ||
| defaultReleaseBranch: "main", | ||
| name: "functionless", | ||
| description: | ||
| "Functionless, a TypeScript plugin and Construct library for the AWS CDK", | ||
| bin: { | ||
| functionless: "./bin/functionless.js", | ||
| }, | ||
| projenrcTs: true, | ||
| deps: [ | ||
| "@types/aws-lambda", | ||
| "fs-extra", | ||
| "minimatch", | ||
| "@functionless/nodejs-closure-serializer", | ||
| "@functionless/ast-reflection@^0.3.1", | ||
| "@swc/cli", | ||
| "@swc/[email protected]", | ||
| "@swc/register", | ||
| "source-map", | ||
| ], | ||
| devDeps: [ | ||
| `@aws-cdk/aws-appsync-alpha@${MIN_CDK_VERSION}-alpha.0`, | ||
| "@types/fs-extra", | ||
| "@types/minimatch", | ||
| "@types/uuid", | ||
| "amplify-appsync-simulator", | ||
| "axios", | ||
| "eslint-plugin-no-only-tests", | ||
| "graphql-request", | ||
| "ts-node", | ||
| "ts-patch", | ||
| "flatted", | ||
| /** | ||
| * For CDK Local Stack tests | ||
| */ | ||
| "@aws-sdk/client-dynamodb", | ||
| `@aws-cdk/cloud-assembly-schema@${MIN_CDK_VERSION}`, | ||
| `@aws-cdk/cloudformation-diff@${MIN_CDK_VERSION}`, | ||
| `@aws-cdk/cx-api@${MIN_CDK_VERSION}`, | ||
| "aws-sdk", | ||
| `aws-cdk@${MIN_CDK_VERSION}`, | ||
| `cdk-assets@${MIN_CDK_VERSION}`, | ||
| "promptly", | ||
| "proxy-agent", | ||
| /** | ||
| * End Local | ||
| */ | ||
| // for serializer testing | ||
| "uuid", | ||
| "@swc/jest", | ||
| "node-cfn", | ||
| "@aws-sdk/client-cloudcontrol", | ||
| "@aws-sdk/client-ssm", | ||
| "@aws-sdk/client-eventbridge", | ||
| "[email protected]", | ||
| "cross-fetch", | ||
| "graphql-tag", | ||
| "graphql", | ||
| "cdk-assets", | ||
| ], | ||
| jestOptions: { | ||
| jestConfig: { | ||
| collectCoverage: false, | ||
| coveragePathIgnorePatterns: ["/test/", "/node_modules/", "/lib"], | ||
| moduleNameMapper: { | ||
| "^@fnls$": "<rootDir>/lib/index", | ||
| }, | ||
| transform: { | ||
| "^.+\\.(t|j)sx?$": ["./jest.js", {}], | ||
| }, | ||
| }, | ||
| extraCliOptions: ["--no-cache"], | ||
| }, | ||
| scripts: { | ||
| localstack: "./scripts/localstack", | ||
| "build:website": "npx tsc && cd ./website && yarn && yarn build", | ||
| }, | ||
| peerDeps: [ | ||
| `aws-cdk-lib@^${MIN_CDK_VERSION}`, | ||
| "aws-sdk", | ||
| "constructs@^10.0.0", | ||
| "esbuild", | ||
| "typesafe-dynamodb@^0.1.5", | ||
| "typescript@^4.8.2", | ||
| ], | ||
| eslintOptions: { | ||
| dirs: ["src"], | ||
| ignorePatterns: [ | ||
| "scripts/**", | ||
| "register.js", | ||
| "jest.js", | ||
| "swc-config.js", | ||
| "website/**", | ||
| ], | ||
| lintProjenRc: false, | ||
| }, | ||
| tsconfig: { | ||
| compilerOptions: { | ||
| // @ts-ignore | ||
| declarationMap: true, | ||
| lib: ["dom", "ES2022"], | ||
| noUncheckedIndexedAccess: true, | ||
| resolveJsonModule: true, | ||
| skipLibCheck: true, | ||
| }, | ||
| }, | ||
| tsconfigDev: { | ||
| compilerOptions: { | ||
| paths: { | ||
| "@fnls": ["lib/index"], | ||
| }, | ||
| baseUrl: ".", | ||
| skipLibCheck: true, | ||
| }, | ||
| }, | ||
| gitignore: [".DS_Store", ".dccache", ".swc"], | ||
| releaseToNpm: true, | ||
| depsUpgradeOptions: { | ||
| workflowOptions: { | ||
| projenCredentials: GithubCredentials.fromApp(), | ||
| }, | ||
| }, | ||
| prettier: true, | ||
| workflowBootstrapSteps: [assumeRoleStep], | ||
| }); | ||
| // projen assumes ts-jest | ||
| delete project.jest!.config.globals; | ||
| delete project.jest!.config.preset; | ||
|
|
||
| const packageJson = project.tryFindObjectFile("package.json")!; | ||
|
|
||
| packageJson.addOverride("lint-staged", { | ||
| "*.{tsx,jsx,ts,js,json,md,css}": ["eslint --fix"], | ||
| }); | ||
|
|
||
| const closeWorkflow = project.github?.addWorkflow("close"); | ||
| closeWorkflow?.on({ | ||
| pullRequest: { | ||
| types: ["closed"], | ||
| }, | ||
| }); | ||
| const cleanJob: Job = { | ||
| permissions: { contents: JobPermission.WRITE, idToken: JobPermission.WRITE }, | ||
| runsOn: ["ubuntu-latest"], | ||
| env: { | ||
| CI: "true", | ||
| }, | ||
| steps: [ | ||
| assumeRoleStep, | ||
| { | ||
| uses: "marvinpinto/action-inject-ssm-secrets@latest", | ||
| with: { | ||
| ssm_parameter: | ||
| // on merge, github.ref no longer returns the same format. github.event.pull_request.number returns the PR number so we can re-construct the ref. | ||
| // https://github.com/actions/runner/issues/256/ | ||
| "/functionlessTestDeleter/FunctionlessTest-refs/pull/${{ github.event.pull_request.number }}/merge/deleteUrl", | ||
| env_variable_name: "FL_DELETE_URL", | ||
| }, | ||
| }, | ||
| { | ||
| uses: "fjogeleit/http-request-action@v1", | ||
| with: { | ||
| url: "${{ env.FL_DELETE_URL }}", | ||
| method: "GET", | ||
| }, | ||
| }, | ||
| ], | ||
| }; | ||
| closeWorkflow?.addJob("cleanUp", cleanJob); | ||
|
|
||
| project.compileTask.prependExec( | ||
| "yarn link && cd ./test-app && yarn link functionless" | ||
| ); | ||
| project.compileTask.env("NODE_OPTIONS", "--max-old-space-size=6144"); | ||
| project.compileTask.env("TEST_DEPLOY_TARGET", "AWS"); | ||
|
|
||
| project.compileTask.prependExec("ts-node ./scripts/sdk-gen.ts"); | ||
|
|
||
| // start over... | ||
| project.testTask.reset(); | ||
|
|
||
| // To run tests on github using localstack instead of AWS, uncomment the below and comment out TEST_DEPLOY_TARGET. | ||
| // project.testTask.prependExec("./scripts/localstack"); | ||
| // project.testTask.exec("localstack stop"); | ||
| // project.testTask.env("DEFAULT_REGION", "ap-northeast-1"); | ||
| // project.testTask.env("AWS_ACCOUNT_ID", "000000000000"); | ||
| // project.testTask.env("AWS_ACCESS_KEY_ID", "test"); | ||
| // project.testTask.env("AWS_SECRET_ACCESS_KEY", "test"); | ||
| project.testTask.env("TEST_DEPLOY_TARGET", "AWS"); | ||
| project.testTask.env("NODE_OPTIONS", "--max-old-space-size=6144"); | ||
|
|
||
| const typeCheck = project.addTask("typecheck", { | ||
| exec: "tsc -p ./tsconfig.dev.json --noEmit", | ||
| }); | ||
|
|
||
| const testFast = project.addTask("test:fast", { | ||
| exec: "jest --passWithNoTests --all --updateSnapshot --testPathIgnorePatterns '(localstack|runtime)'", | ||
| }); | ||
|
|
||
| const testRuntime = project.addTask("test:runtime", { | ||
| exec: "jest --passWithNoTests --all --updateSnapshot --testPathPattern '(localstack|runtime)' --no-cache", | ||
| }); | ||
|
|
||
| const testApp = project.addTask("test:app", { | ||
| exec: "cd ./test-app && yarn && yarn build && yarn synth --quiet", | ||
| }); | ||
|
|
||
| project.testTask.spawn(typeCheck); | ||
| project.testTask.spawn(testFast); | ||
| project.testTask.spawn(testApp); | ||
| project.testTask.spawn(testRuntime); | ||
| project.testTask.spawn(project.tasks.tryFind("eslint")!); | ||
|
|
||
| project.addPackageIgnore("/test-app"); | ||
| project.addPackageIgnore("/website"); | ||
|
|
||
| // id-token is required for aws-actions/configure-aws-credentials@v1 with OIDC | ||
| // https://github.com/aws-actions/configure-aws-credentials/issues/271#issuecomment-1012450577 | ||
| // @ts-ignore | ||
| project.buildWorkflow.workflow.jobs.build = { | ||
| // @ts-ignore | ||
| ...project.buildWorkflow.workflow.jobs.build, | ||
| // deploy the clean up stack during tests to be available for the cleanup pull_request closed job | ||
| // only do this for build workflow as the release workflow deletes immediately | ||
| env: { | ||
| // @ts-ignore | ||
| ...project.buildWorkflow.workflow.jobs.build.env, | ||
| CLEAN_UP_STACK: "1", | ||
| }, | ||
| permissions: { | ||
| // @ts-ignore | ||
| ...project.buildWorkflow.workflow.jobs.build.permissions, | ||
| "id-token": "write", | ||
| contents: "write", | ||
| }, | ||
| }; | ||
|
|
||
| // id-token is required for aws-actions/configure-aws-credentials@v1 with OIDC | ||
| // https://github.com/aws-actions/configure-aws-credentials/issues/271#issuecomment-1012450577 | ||
| // @ts-ignore | ||
| project.release.defaultBranch.workflow.jobs.release = { | ||
| // @ts-ignore | ||
| ...project.release.defaultBranch.workflow.jobs.release, | ||
| env: { | ||
| // @ts-ignore | ||
| ...project.release.defaultBranch.workflow.jobs.release.env, | ||
| // on release, do not maintain the stacks, delete them right away | ||
| TEST_STACK_RETENTION_POLICY: "DELETE", | ||
| }, | ||
| permissions: { | ||
| // @ts-ignore | ||
| ...project.release.defaultBranch.workflow.jobs.release.permissions, | ||
| "id-token": "write", | ||
| contents: "write", | ||
| }, | ||
| }; | ||
|
|
||
| project.eslint!.addRules({ | ||
| quotes: "off", | ||
| "comma-dangle": "off", | ||
| "quote-props": "off", | ||
| "@typescript-eslint/indent": "off", | ||
| "@typescript-eslint/no-shadow": "off", | ||
| "@typescript-eslint/member-ordering": "off", | ||
| "brace-style": "off", | ||
| "@typescript-eslint/explicit-member-accessibility": "off", | ||
| "no-debugger": "error", | ||
| }); | ||
|
|
||
| project.eslint!.addIgnorePattern("test-app/hook.js"); | ||
|
|
||
| project.eslint!.addOverride({ | ||
| files: ["*.ts", "*.mts", "*.cts", "*.tsx"], | ||
| // @ts-ignore | ||
| plugins: ["no-only-tests"], | ||
| parserOptions: { | ||
| project: [ | ||
| "./tsconfig.dev.json", | ||
| "./test-app/tsconfig.json", | ||
| "./website/tsconfig.json", | ||
| "./test/tsconfig.json", | ||
| ], | ||
| }, | ||
| rules: { | ||
| "@typescript-eslint/explicit-member-accessibility": [ | ||
| "error", | ||
| { | ||
| accessibility: "explicit", | ||
| overrides: { | ||
| accessors: "explicit", | ||
| constructors: "no-public", | ||
| methods: "explicit", | ||
| properties: "off", | ||
| parameterProperties: "off", | ||
| }, | ||
| }, | ||
| ], | ||
| "no-only-tests/no-only-tests": ["error", { fix: true, block: ["test."] }], | ||
| }, | ||
| }); | ||
|
|
||
| project.prettier!.addIgnorePattern("coverage"); | ||
| project.prettier!.addIgnorePattern("lib"); | ||
|
|
||
| project.synth(); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delete this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, must have come from the merge? deleted