Sync Helm charts from Artifact Registry

This page shows you how to sync Helm charts from Artifact Registry by creating and pushing a Helm chart to a repository in Artifact Registry. It also contains a sample configuration to sync a chart from your Helm repository.

You can configure Config Sync to sync from Helm repositories. You can store Helm charts in Artifact Registry, which is the recommended Helm repository for Google Cloud. To use this feature, you must enable the RootSync and RepoSync APIs. Config Sync renders Helm charts by using helm template and therefore does not support full Helm lifecycle management.

Bundled Helm and Kustomize versions lists the Kustomize and Helm versions bundled with the corresponding version of Config Sync.

Before you begin

Limitations

  • You can't change any immutable field in a config by just changing the value in the source of truth. If you need to update an immutable field, first make the change in the source of truth, then manually delete the object in the cluster. Config Sync can then re-create the object with the new field value.

  • The following Helm charts include Jobs and aren't recommended for deployment by Config Sync:

    To learn more about why Jobs aren't recommended for use with Config Sync, see Avoid managing Jobs with Config Sync.

Create an Artifact Registry repository

In this section, you create an Artifact Registry repository. To learn more about creating Artifact Registry repositories, see Create repositories.

  1. Enable the Artifact Registry API:

    gcloud services enable artifactregistry.googleapis.com --project=PROJECT_ID
    
  2. Create an Artifact Registry repository:

    gcloud artifacts repositories create AR_REPO_NAME \
       --repository-format=docker \
       --location=AR_REGION \
       --description="Config Sync Helm repo" \
       --project=PROJECT_ID
    

Replace the following:

  • PROJECT_ID: the organization's project ID.
  • AR_REPO_NAME: the ID of the repository.
  • AR_REGION: the regional or multi-regional location of the repository.

Variables used in the following sections:

  • FLEET_HOST_PROJECT_ID: if you're using GKE Workload Identity Federation for GKE, this is the same as PROJECT_ID. If you're using fleet Workload Identity Federation for GKE, this is the project ID of the fleet that your cluster is registered to.
  • GSA_NAME: the name of the custom Google service account that you want to use to connect to Artifact Registry.
  • KSA_NAME: the Kubernetes service account for the reconciler.
    • For root repositories, if the RootSync name is root-sync, add root-reconciler. Otherwise, add root-reconciler-ROOT_SYNC_NAME.
    • For namespace repositories, if the RepoSync name is repo-sync, add ns-reconciler-NAMESPACE. Otherwise, add ns-reconciler-NAMESPACE-REPO_SYNC_NAME-REPO_SYNC_NAME_LENGTH where REPO_SYNC_NAME_LENGTH is the number of characters in REPO_SYNC_NAME.

Grant reader permission

If the Config Sync version is 1.17.2 or later on your cluster, you can use the Kubernetes service account to authenticate to Artifact Registry. Otherwise, use the Google service account for authentication.

Using Kubernetes service account

Grant the Artifact Registry Reader (roles/artifactregistry.reader) IAM role to the Kubernetes service account with the Workload Identity Federation for GKE pool:

gcloud artifacts repositories add-iam-policy-binding AR_REPO_NAME \
    --location=AR_REGION \
    --member="serviceAccount:FLEET_HOST_PROJECT_ID.svc.id.goog[config-management-system/KSA_NAME]" \
    --role=roles/artifactregistry.reader \
    --project=PROJECT_ID

Using Google service account

  1. Grant the Artifact Registry Reader (roles/artifactregistry.reader) IAM role to the Google service account:

    gcloud artifacts repositories add-iam-policy-binding AR_REPO_NAME \
       --location=AR_REGION \
       --member=serviceAccount:GSA_NAME@PROJECT_ID.iam.gserviceaccount.com \
       --role=roles/artifactregistry.reader \
       --project=PROJECT_ID
    
  2. Create an IAM policy binding between the Kubernetes service account and Google service account:

    gcloud iam service-accounts add-iam-policy-binding \
       --role roles/iam.workloadIdentityUser \
       --member "serviceAccount:FLEET_HOST_PROJECT_ID.svc.id.goog[config-management-system/KSA_NAME]" \
       GSA_NAME@PROJECT_ID.iam.gserviceaccount.com \
       --project=PROJECT_ID
    

Push a Helm chart to the Artifact Registry repository

In this section, you download a public Helm chart and push it to Artifact Registry.

  1. Retrieve the mysql-9.3.1.tgz package from the public Helm repository and download it locally:

    helm pull mysql --repo https://charts.bitnami.com/bitnami --version 9.3.1
    
  2. Authenticate with an access token:

    Linux / macOS

    gcloud auth print-access-token | helm registry login -u oauth2accesstoken \
    --password-stdin https://AR_REGION-docker.pkg.dev
    

    Windows

    gcloud auth print-access-token
    ya29.8QEQIfY_...
    
    helm registry login -u oauth2accesstoken -p "ya29.8QEQIfY_..." \
    https://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME
    

    In this command, oauth2accesstoken is the username to use when authenticating with an access token and gcloud auth print-access-token is the command to obtain the access token. Your access token is the password for authentication. Authentication with an access token is the safest authentication method.

  3. Push the Helm chart to Artifact Registry:

    helm push mysql-9.3.1.tgz oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME
    

Configure Config Sync to sync from your Helm chart

In this section, you create a RootSync object and configure Config Sync to sync from the Helm chart.

If you want to override the Helm chart's default values, you can do this by either specifying values in the spec.helm.values field or by adding a reference to a ConfigMap by using the spec.helm.valuesFileRefs field. To learn more about the optional fields, see Configuration for the Helm repository.

values

  1. Create a RootSync object with a unique name:

    cat <<EOF>> ROOT_SYNC_NAME.yaml
    apiVersion: configsync.gke.io/v1beta1
    kind: RootSync
    metadata:
      name: ROOT_SYNC_NAME
      namespace: config-management-system
    spec:
      sourceFormat: unstructured
      sourceType: helm
      helm:
        repo: oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME
        chart: mysql
        version: 9.3.1
        releaseName: my-mysql
        namespace: test
        # The k8sserviceaccount auth type is available in version 1.17.2 and
        # later. Use "gcpserviceaccount" if using an older version.
        # auth: gcpserviceaccount
        # gcpServiceAccountEmail: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
        auth: k8sserviceaccount
        # Use the optional field spec.helm.values to override default values.
        # You can use the same format as the default values file to override
        # default values.
        values:
          image:
            pullPolicy: Always
          primary:
            resources:
              limits:
                cpu: 250m
                memory: 256Mi
              requests:
                cpu: 250m
                memory: 256Mi
    EOF
    

    Replace ROOT_SYNC_NAME with the name of your RootSync object. The name should be unique in the cluster and have no more than 26 characters. If you installed Config Sync using the Google Cloud console or Google Cloud CLI, choose a name other than root-sync.

    In this example, the Helm chart is deployed in the test namespace because its resources contain namespace: {{ .Release.Namespace }} in its templates.

    You can use helm.values to override the default values. To learn about the optional fields, see Configuration for the Helm repository.

  2. Apply the RootSync object:

    kubectl apply -f ROOT_SYNC_NAME.yaml
    
  3. Verify that Config Sync is syncing from the image:

    nomos status --contexts=$(kubectl config current-context)
    

    The output is similar to the following:

    Connecting to clusters...
    
    *cluster-name
      --------------------
      <root>:root-sync   oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME/mysql:9.3.1
      SYNCED             9.3.1
      Managed resources:
          NAMESPACE  NAME                       STATUS    SOURCEHASH
          default    configmap/my-mysql         Current   9.3.1
          default    secret/my-mysql            Current   9.3.1
          default    service/my-mysql           Current   9.3.1
          default    service/my-mysql-headless  Current   9.3.1
          default    serviceaccount/my-mysql    Current   9.3.1
          default    statefulset.apps/my-mysql  Current   9.3.1
    

    You have now successfully synced the Helm chart to your cluster.

valuesFileRefs

  1. Create a RootSync object with a unique name:

    cat <<EOF>> ROOT_SYNC_NAME.yaml
    apiVersion: configsync.gke.io/v1beta1
    kind: RootSync
    metadata:
      name: ROOT_SYNC_NAME
      namespace: config-management-system
    spec:
      sourceFormat: unstructured
      sourceType: helm
      helm:
        repo: oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME
        chart: mysql
        version: 9.3.1
        releaseName: my-mysql
        # The k8sserviceaccount auth type is available in version 1.17.2 and
        # later. Use "gcpserviceaccount" if using an older version.
        # auth: gcpserviceaccount
        # gcpServiceAccountEmail: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
        auth: k8sserviceaccount
        # use the optional field spec.helm.valuesFilesRefs to override default values
        # by referencing a ConfigMap
        valuesFileRefs:
        - name: CONFIGMAP_NAME
          dataKey: DATA_KEY
    
    EOF
    

    Replace the following:

    • ROOT_SYNC_NAME: the name of your RootSync object. The name should be unique in the cluster and have no more than 26 characters. If you installed Config Sync using the Google Cloud consoleor Google Cloud CLI, choose a name other than root-sync.
    • CONFIGMAP_NAME: the name of your ConfigMap. This can be any valid ConfigMap name accepted by Kubernetes that is unique in your cluster.
    • (optional) DATA_KEY: the data key in your ConfigMap that you would like to read the values from. The default is values.yaml.
  2. Create the ConfigMap object with your values:

    cat <<EOF>> CONFIGMAP_NAME.yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: CONFIGMAP_NAME
      namespace: config-management-system
    immutable: true
    # You can use the same format as the default values file to override
    # default values.
    data:
      DATA_KEY: |-
        image:
          pullPolicy: Always
        primary:
          resources:
            limits:
              cpu: 250m
              memory: 256Mi
            requests:
              cpu: 250m
              memory: 256Mi
    
    EOF
    

    If you didn't specify a value for DATA_KEY in the RootSync, it should be the default values.yaml.

  3. Apply the ConfigMap object:

    kubectl apply -f CONFIGMAP_NAME.yaml
    
  4. Apply the RootSync object:

    kubectl apply -f ROOT_SYNC_NAME.yaml
    
  5. Verify that Config Sync is syncing from the image:

    nomos status --contexts=$(kubectl config current-context)
    

    The output is similar to the following:

    Connecting to clusters...
    
    *cluster-name
      --------------------
      <root>:root-sync   oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME/mysql:9.3.1
      SYNCED             9.3.1
      Managed resources:
          NAMESPACE  NAME                       STATUS    SOURCEHASH
          default    configmap/my-mysql         Current   9.3.1
          default    secret/my-mysql            Current   9.3.1
          default    service/my-mysql           Current   9.3.1
          default    service/my-mysql-headless  Current   9.3.1
          default    serviceaccount/my-mysql    Current   9.3.1
          default    statefulset.apps/my-mysql  Current   9.3.1
    

    You have now successfully synced the Helm chart to your cluster.

    You can also look at the imagePullPolicy in one of the synced resources in the cluster to verify that the values from the ConfigMap were used to render the chart:

    kubectl get statefulset -n test my-mysql -o yaml | grep imagePullPolicy
    
  6. Because the ConfigMap is immutable, to change the values, you must create a new ConfigMap and update spec.helm.valuesFileRefs in the RootSync or RepoSync spec to point to the new ConfigMap. Creating a new ConfigMap ensures changes to values cause the Helm chart to re-render, which is useful when multiple ConfigMaps referenced in spec.helm.valuesFileRefs need to be updated at the same time when re-rendering. To change the values that are used to render your chart, create a new ConfigMap with a different name:

    cat <<EOF>> CONFIGMAP_NAME-2.yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: CONFIGMAP_NAME-2
      namespace: config-management-system
    immutable: true
    # You can use the same format as the default values file to override
    # default values.
    data:
      DATA_KEY: |-
        image:
          pullPolicy: Never
        primary:
          resources:
            limits:
              cpu: 100m
              memory: 256Mi
            requests:
              cpu: 250m
              memory: 200Mi
    
    EOF
    
  7. Update your RootSync object to reference the new ConfigMap:

    cat <<EOF>> ROOT_SYNC_NAME.yaml
    apiVersion: configsync.gke.io/v1beta1
    kind: RootSync
    metadata:
      name: ROOT_SYNC_NAME
      namespace: config-management-system
    spec:
      sourceFormat: unstructured
      sourceType: helm
      helm:
        repo: oci://AR_REGION-docker.pkg.dev/PROJECT_ID/AR_REPO_NAME
        chart: mysql
        version: 9.3.1
        releaseName: my-mysql
        namespace: test
        # The k8sserviceaccount auth type is available in version 1.17.2 and
        # later. Use "gcpserviceaccount" if using an older version.
        # auth: gcpserviceaccount
        # gcpServiceAccountEmail: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
        auth: k8sserviceaccount
        # use the optional field spec.helm.valuesFilesRefs to override default values
        # by referencing a ConfigMap
        valuesFileRefs:
        - name: CONFIGMAP_NAME-2
          dataKey: DATA_KEY
    
    EOF
    
  8. Apply the ConfigMap object:

    kubectl apply -f CONFIGMAP_NAME-2.yaml
    
  9. Apply the RootSync object:

    kubectl apply -f ROOT_SYNC_NAME.yaml
    
  10. Verify that Config Sync is syncing from the image:

    nomos status --contexts=$(kubectl config current-context)
    

    You can also look at the imagePullPolicy in one of the synced resources in the cluster to verify that the new values from the updated ConfigMap were used to render the chart:

    kubectl get statefulset -n test my-mysql -o yaml | grep imagePullPolicy
    

What's next