Using CRDs with multi-clusters

[arthurcogo]

I'm trying to migrate from using annotations to the new CRDs with v1 🙌 However, I'm not sure of the best way to configure things for our setup.

We have two clusters. ArgoCD runs on cluster A, where all of our Applications are defined. Some of those Applications are deployed on cluster B. Image updater runs on cluster A only.

With annotations this was straightforard, because I just added the annotations to the Applications on cluster A and it was away.

When attempting to migrate, I removed the annotations and used the new ImageUpdater CRDs for each application instead. What I didn't foresee was that the application attempted to deploy the ImageUpdater resources to the target cluster alongside the other resources, but that doesn't work on cluster B because it doesn't have image updater installed and therefore doesn't have the CRDs.

It seems the expected solution is to deploy the ImageUpdater resources in cluster A... but this means they're decoupled from the application they are updating (as there is no way to deploy one application on two clusters).

Am I missing something? Does anyone else have a good solution for this?

For context, this is what I'm trying to migrate from annotation-based to CRD-based:

• Production application on cluster A using a semver update strategy
• Staging application on cluster B using a digest strategy on the edge tag
• Preview application set on cluster B using a digest strategy on the pr-<number> tag

[dkarpele]

Thanks for asking, your assumptions are correct.
You can use Image Updater with multi-cluster setup with 1 restriction:

• Image Updater resources (including controller) and all Image Updater CRs must be installed on the same cluster where your Argo CD Application resources are managed. You can install these resources to any namespace though.

You don't need to install Image Updater controller to the cluster B to create Image Updater CRs there, you need to install only CRD. But it makes no sense. Controller just doesn't "see" CRs on the cluster B.

So the CR can look like this:

apiVersion: argocd-image-updater.argoproj.io/v1alpha1
kind: ImageUpdater
metadata:
  name: iu-cr
  namespace: dev # any namespace on cluster A
spec:
  namespace: argocd # namespace where Application is defined, not deployed. ArgoCD in any namespace is supported.
  applicationRefs:
    - namePattern: "cluster-b-*"
      images:
        - alias: "image-name"
...

[dkarpele]

I extended documentation with information about this use case #1363

[kristeey]

Not directly related to the problem described but directly related to v1 CRD based image update + multicluster + appsets.

With applicationSets we were able to patch image tags,repos etc of the argocd image updater annotations making it quite easy to separate dev and platform concerns. Devs could only care about a single file in a repo, and the appsets would create applications and configure image update automation based on the values they configure. Now with V1 we cannot use annotations on ArgoCD Applications anymore removing the possibility for that type of split between dev concerns and platform concerns in a centralized ArgoCD multicluster scenario. Since we only have argo installed in a single cluster (mgmt cluster) we cannot create ImageUpdater CRs in the spoke clusters (as described in the scenario over). Hence we need to create it in the mgmt cluster, but since the image repo and tagging is a developer concern, it makes it a lot harder to solve now that we cannot use annotations and patching within ApplicationSets. It breaks with our current setup...

I don't say that i really enjoy ApplicationSets as they are not always intuitive when glancing at them, but they are nice in regards to not writing duplicate code, and making the code more static, and not relying that much on kustomization and difficult overlay structures.

• Since we have dev/platform split concerns it does not make sense to have a single ImageUpdater per application in the sense that the ImageUpdater is a platform concern and the image and tagging is a dev concern that might change -> breaking the image update.
• Since we have do not have a special Application naming convention we cannot use ImageUpdater applicationRefs with namepatterns since this would break as the environment promotion tags will not fit with this pattern (for us).
• We are left with label selectors and a wildcard name pattern. But this requires that we change our Application label configuration to support this, and it do not actually solve our problem since we still need to set the image to use in the ImageUpdater. If we could have been able to fetch the image from elsewhere we would be able to recreate the way our current setup with Appsets work. But since we need to specify the image we cannot do that.

[Lappihuan]

It seems like the ImageUpdater CRD was not designed with ApplicationSets in mind.
perhaps ApplicationSets could be extended to also generate ImageUpdater CRDs?