CI/CD with CDK pipelines - Workflows for increasing development speed

[alex5207]

We've adopted the CDK pipelines construct library for maintaining a ci/cd of our cdk app with our git repo as a source. It works quite well and we're fairly happy with it. But since this pipeline's construct wraps the stack that contains our application we can no longer use cdk deploy. This effectively leaves us in a place where (as far as I can see) can only deploy to our staging environment by pushing to the repo which starts the pipeline. This presents us with two challenges:

(1) A very crowded and non-meaningful commit history
(2) Long feedback cycles when developing. Our pipeline takes 10-15 min including builds, unit and integ tests.

For previous projects I've used aws sam which had great support for fast iterations using sam sync which I found really effective.

What we've tried to solve it:
We've tried to let developers use a script to modify our app.py to replace the Pipeline construct with the the Application stack which then can be cdk deploy'ed. But challenges us since some uniquely named resources already exists on the account from the wrapped stack. We're using best practices to let CDK define most resource names (dynamodb tables, lambda etc.) but still have resources such as SES email identities etc. which we have to name explicitly.
What is an effective way of managing these challenges while getting the benefits of cdk pipelines?

Thanks a lot.

Alex

[peterwoodworth]

Thanks for posting this question @alex5207,

There's some things about your use case I'd like to understand more.

I'm curious what exactly is causing a "very crowded and non-meaningful commit history", could you go into this a bit?

When would you want to run cdk deploy if you haven't made a change?

[alex5207]

Hi Peter,

Thanks for your reply. Let me try and clarify how we're using it. It might reveal a misunderstanding of the pipelines concept on my side.

Assume that we're developing some feature (e.g a lambda) that we'd like to test. Our project structure is such that app.py contains our PipelineStack which then contains an AppStage with the stack that actually contains the resources in our application (e.g the lambda that we're developing).

Now, I'd like to test some change I made to this lambda:

(1) Without CDK pipelines: I would just run cdk deploy which then would deploy my stack containing the application
(2) With CDK pipelines: This is where the "very crowded ... commit history comes"; We're comitting to our git dev branch to allow the lambda change to be pulled by the codepipeline because my understanding is that cdk deploy would not affect the underlying application as it is being managed by the pipeline that just pulls from it's source (e.g github).

Hope that clarifies a bit - Looking forward to hear your thoughts.

Alex

[peterwoodworth]

You can bypass the need to commit and have the pipeline run if you would like to make changes to your stack directly. The pipeline takes the stages that are synthesized to the cdk.out directory and deploys those, there's nothing stopping you from doing that in the CLI yourself 🙂

I have a PipelineStack which contains a TestStage that contains a TestStack that will deploy a Lambda Function. When I run cdk list, here is the output:

PipelineStack
PipelineStack/TestStage/TestStack

I deployed the PipelineStack and committed to a repo setup as the Pipeline source. So, the pipeline ran and deployed TestStack. I ran the Lambda function, and it returned Hello World.

Then, I made a change to the Lambda function in TestStack, and deployed locally instead of committing to git. I did this by running cdk deploy PipelineStack/TestStage/TestStack.

I ran the Lambda function again, and it returned Hello World 2.0.

---

An alternate solution to your problem could be setting up a Pipeline on a testing branch, where you won't care what the commit history looks like.

Hope this helps 🙂

[BwL1289]

@alex5207 Im very late to this thread, but another idea that we employ is, on a feature branch, make your meaningless commits to that feature branch and manually release the change on codepipeline by specifying the commit in the feature branch.

Then before merging to dev squash all commits.

One caveat that can get frustrating is if you have self mutation on and the pipeline needs to self mutate, it will pull the latest commit from the default branch for that pipeline (dev/staging/prod) and you’ll end up unexpectedly deploying the “wrong” stack.

[BwL1289]

Also linking this discussion as CDK pipelines, while powerful, can be painfully slow