helm dependency build failing with empty index.yaml file #48
Comments
|
Hi @piljaechae! Unfortunately, I haven't encountered that issue before. Are you running these 31 applications in a "real" ArgoCD cluster? If so, you might want to check the ArgoCD configuration you're using in your "real" cluster and replicate it in the |
|
I think the timeout isn't the core problem here. The problem I'm facing isn't really deterministic, but I can verify that the problem is closely related to the stage of fetching the chart from the chartmuseum. |
|
Yes and indeed, we have about 50 argocd applications in a single cluster. Running diff for 31 of them (only ones related to our backend) is having problems, whereas running diff for only 3~7 applications (for frontend, infra, ml, .. - we group these) works without any problems. |
|
btw, I'm curious if the speed can be improved. Spinning up the local cluster, setting up argocd, and running syncs for the applications seems to be quite slow. I like the idea, but having to wait a few minutes for the diff to show up may not be optimal I believe. |
Interesting! In this case, If you think this may be a temporary error that shouldn't cause an immediate exit, you can try removing that line from the source code and see if it resolves the issue :) I see you've already forked the repo 🚀
This is the main downside of this solution. It’s slow because you have to start a new cluster each time and install ArgoCD on it. I’ve been exploring whether it’s possible to pre-bake the ArgoCD instance into the Dockerized kind cluster so that when the local kind cluster starts up, ArgoCD would already be set up and ready to go. However, I haven’t been able to make this work yet, and I’m still looking for ways to speed up the process. Any suggestions are highly appreciated 🙌🏻 |
|
Have a couple ideas regarding speed.
|
Automatically detecting which applications need to be rendered is very challenging, as there are many edge cases to handle. argocd-diff-preview already supports a few methods for selecting which applications to render [docs]. However, I am working on another feature that will allow users to define a regex pattern to match only the changed files in the PR, ensuring that only the relevant apps are rendered (as long as you manually add the annotation to all your apps). apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
namespace: argocd
annotations:
argocd-diff-preview/watch-pattern: "some/path/.*" # regex - listens for all changes in <root>/some/path/**
spec:
... |
|
@piljaechae, Are you still experiencing this issue, or can I close the issue? 👼🏻 |
|
Sorry @dag-andersen !! I'll take a look at it within a week :) |
|
We use ApplicationSet and git generator. Seeing the same problem with argocd-diff-preview v0.0.30: Unfortunately we have more than 50 applications from one ApplicationSet. The filtering seems only filter ApplicationSet but not the nested applications. Not sure if it something we can improve in argocd-diff-preview. But thie error Update: |
|
Hi @HotThoughts, Could you elaborate on this: "Unfortunately we have more than 50 applications from one ApplicationSet. The filtering seems only filter ApplicationSet but not the nested applications."? It would be helpful if you could share the ApplicationSet that’s generating these applications 🤞🏻 What are you experiencing, and what changes would you like to see? :) Let me know — I might be able to make it happen! |
|
Hi @dag-andersen, Apologies for the confusion in my previous message. We have 19 ApplicationSet resources, and each generates more than 50 Application resources. Due to the large number of applications, we might encounter Helm build update concurrency issues. To address this, I attempted to use the annotation apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: app-dev
namespace: argocd
annotations:
argocd-diff-preview/watch-pattern: apps/*/.* 👈🏻 here we watch all under directory appsIn the GitHub Actions workflow, we use --filters to limit changes to the relevant applications: - name: Get changed directories
id: changed-files
uses: tj-actions/changed-files@v45
with:
separator: ","
- name: Generate Diff
run: |
docker run \
--network=host \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(pwd)/main:/base-branch \
-v $(pwd)/pull-request:/target-branch \
-v $(pwd)/output:/output \
-v $(pwd)/secrets:/secrets \
-e TARGET_BRANCH=${{ github.head_ref }} \
-e REPO=${{ github.repository }} \
-e FILES_CHANGED=${{ steps.changed-files.outputs.all_changed_files }} \ 👈 here we pass changed files
dagandersen/argocd-diff-preview:v0.0.30Despite using the watch-pattern annotation and passing only changed files, the process still attempts to render all applications defined by the ApplicationSet. Seems it only filters ApplicationSet. Would be nice if we can go further and only generate changed Application in ArgoCD. In the workflow logs, I can see: |
|
Thank you for the detailed description! :) Can you verify that the tools writes It seems like you’re doing everything right, so there might be a bug. You should see something like this: |
|
However, after checking again... It does say: ⬇️ It "correctly" only renders the apps based on the So, I guess the problem is that it doesn’t apply the filtering to the Applications generated by the ApplicationSet.
Looks like you’re right! :) I think i can fix that 🚀 |
|
Hi @HotThoughts, I’ve created a new version of the tool that performs a second filtering after generating apps based on AppSets. Would you mind trying You will need to add the annotation to both the ApplicationSets and the Applications they are creating. |
|
Hi @dag-andersen, ❤️ Thanks. I tested v0.0.31 and added annotations to Applications, so now: apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: app-dev
namespace: argocd
annotations:
argocd-diff-preview/watch-pattern: apps/*/.* 👈🏻 here we watch all under directory apps
spec:
...
generators:
- git:
...
template:
metadata:
annotaitons:
argocd-diff-preview/watch-pattern: '{{ default (printf "apps/%s/.*" (index .path.segments 1) .valuesPath }}/.*' 👈🏻 here we watch all files under one appStill the same error - argocd trying to render 98 applications while only one changed 😞 |
|
@HotThoughts Essentially, I'm asking you to check if this log appears before the line: |
|
@HotThoughts |
|
It works now! Argocd-diff-preview filters Applications in ApplicationSet! I found out the root problem at my end. Thank you for the help. ❤️ Love the action! |
Hi @dag-andersen, I came across your nice project. I succeed when I run docker container for few applications (I first tested with 4). However, if I increase the application numbers (31), I'm constantly getting error below.
I keep failing at the last application, but the application that's causing the ComparisonError is actually random. So I'm guessing that the chart itself would not be the problem.
Maybe the issue is related with helm itself but none of the solutions seemed to work.
helm dependency buildin argocd-repo-server argoproj/argo-cd#12902I was wondering if you've encountered this issue?
The text was updated successfully, but these errors were encountered: