Skip to content

creating crds with the helm chart fails with configmap too long #1911

Closed

Description

@amithalle

When trying to apply the helm chart, we are getting the following error:
Failed sync attempt to ####: one or more objects failed to apply, reason: ConfigMap "altinity-clickhouse-operator-crds" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes (retried 5 times).

helm chart version: 0.25.6

The issue was resolved the issue locally by splitting the configmap into 2 parts:

Values.crdHook.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-part1
  namespace: {{ include "altinity-clickhouse-operator.namespace" . }}
  labels:
    {{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
    app.kubernetes.io/component: crd-install-hook
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "-7"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  clickhouseinstallationtemplates.yaml: |
{{ .Files.Get "crds/CustomResourceDefinition-clickhouseinstallationtemplates.clickhouse.altinity.com.yaml" | indent 4 }}
  clickhousekeeperinstallations.yaml: |
{{ .Files.Get "crds/CustomResourceDefinition-clickhousekeeperinstallations.clickhouse-keeper.altinity.com.yaml" | indent 4 }}
  clickhouseoperatorconfigurations.yaml: |
{{ .Files.Get "crds/CustomResourceDefinition-clickhouseoperatorconfigurations.clickhouse.altinity.com.yaml" | indent 4 }}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-part2
  namespace: {{ include "altinity-clickhouse-operator.namespace" . }}
  labels:
    {{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
    app.kubernetes.io/component: crd-install-hook
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "-7"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
data:
  clickhouseinstallations.yaml: |
{{ .Files.Get "crds/CustomResourceDefinition-clickhouseinstallations.clickhouse.altinity.com.yaml" | indent 4 }}

{{- end }}

then fix in the job's pod spec:

    spec:
      serviceAccountName: {{ include "altinity-clickhouse-operator.fullname" . }}-crd-install
      restartPolicy: OnFailure
      {{- with .Values.crdHook.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.crdHook.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.crdHook.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      containers:
      - name: crd-install
        image: "{{ ( tpl $.Values.crdHook.image.repository $ ) }}:{{ ( tpl $.Values.crdHook.image.tag $ ) }}"
        imagePullPolicy: {{ .Values.crdHook.image.pullPolicy | default "IfNotPresent" }}
        command:
        - /bin/sh
        - -c
        - |
          set -e
          echo "Installing/Updating ClickHouse Operator CRDs..."
          for crd_file in /crds/*/*.yaml; do
            echo "Applying $(basename $crd_file)..."
            kubectl apply --server-side=true --force-conflicts -f "$crd_file"
          done
          echo "CRD installation completed successfully"
        volumeMounts:
        - name: crds-part1
          mountPath: /crds/part1
          readOnly: true
        - name: crds-part2
          mountPath: /crds/part2
          readOnly: true
        {{- with .Values.crdHook.resources }}
        resources:
          {{- toYaml . | nindent 10 }}
        {{- end }}
      volumes:
      - name: crds-part1
        configMap:
          name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-part1
      - name: crds-part2
        configMap:
          name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-part2
        mountPath: /crds
        readOnly: true

Activity

  1. Slach commented on Jan 28, 2026

    @Slach
    Collaborator

    Thanks for reporting!
    What is your kubernetes installation? Which kubernetes version?

    Which cloud priovider EKS, GKE, AKS?

    looks like your metadata.annotations insideconfigmap contains something else before applying helm chart

  2. andrei-dascalu commented on Jan 28, 2026

    @andrei-dascalu

    @amithalle I believe this could be solved by using server side apply (though I believe this involves rendering the chart and then passing it to Kubectl with server side apply flag).

  3. madrisan commented on Mar 25, 2026

    @madrisan
    Contributor

    Same issue with version 0.26.1 on a self-hosted Kubernetes cluster (v1.34.3).

    Thanks to @amithalle for the fix, which has been rewritten here as a patch for greater clarity.

    diff --git a/clickhouse/operator/templates/hooks/crd-install-configmap.yaml b/clickhouse/operator/templates/hooks/crd-install-configmap.yaml
    index e31c127..479fdb2 100644
    --- a/clickhouse/operator/templates/hooks/crd-install-configmap.yaml
    +++ b/clickhouse/operator/templates/hooks/crd-install-configmap.yaml
    @@ -2,7 +2,7 @@
     apiVersion: v1
     kind: ConfigMap
     metadata:
    -  name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds
    +  name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-1
       namespace: {{ include "altinity-clickhouse-operator.namespace" . }}
       labels:
         {{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
    @@ -21,6 +21,23 @@ data:
     {{ .Files.Get "crds/CustomResourceDefinition-clickhouseinstallationtemplates.clickhouse.altinity.com.yaml" | indent 4 }}
       clickhousekeeperinstallations.yaml: |
     {{ .Files.Get "crds/CustomResourceDefinition-clickhousekeeperinstallations.clickhouse-keeper.altinity.com.yaml" | indent 4 }}
    +---
    +apiVersion: v1
    +kind: ConfigMap
    +metadata:
    +  name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-2
    +  namespace: {{ include "altinity-clickhouse-operator.namespace" . }}
    +  labels:
    +    {{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
    +    app.kubernetes.io/component: crd-install-hook
    +  annotations:
    +    "helm.sh/hook": pre-install,pre-upgrade
    +    "helm.sh/hook-weight": "-7"
    +    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
    +    {{- with .Values.crdHook.annotations }}
    +    {{- toYaml . | nindent 4 }}
    +    {{- end }}
    +data:
       clickhouseoperatorconfigurations.yaml: |
     {{ .Files.Get "crds/CustomResourceDefinition-clickhouseoperatorconfigurations.clickhouse.altinity.com.yaml" | indent 4 }}
     {{- end }}
    diff --git a/clickhouse/operator/templates/hooks/crd-install-job.yaml b/clickhouse/operator/templates/hooks/crd-install-job.yaml
    index 16e1d22..4be42ea 100644
    --- a/clickhouse/operator/templates/hooks/crd-install-job.yaml
    +++ b/clickhouse/operator/templates/hooks/crd-install-job.yaml
    @@ -50,21 +50,27 @@ spec:
             - |
               set -e
               echo "Installing/Updating ClickHouse Operator CRDs..."
    -          for crd_file in /crds/*.yaml; do
    +          for crd_file in /crds/*/*.yaml; do
                 echo "Applying $(basename $crd_file)..."
                 kubectl apply --server-side=true --force-conflicts -f "$crd_file"
               done
               echo "CRD installation completed successfully"
             volumeMounts:
    -        - name: crds
    -          mountPath: /crds
    +        - name: crds-1
    +          mountPath: /crds/1
    +          readOnly: true
    +        - name: crds-2
    +          mountPath: /crds/2
               readOnly: true
             {{- with .Values.crdHook.resources }}
             resources:
               {{- toYaml . | nindent 10 }}
             {{- end }}
           volumes:
    -      - name: crds
    +      - name: crds-1
    +        configMap:
    +          name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-1
    +      - name: crds-2
             configMap:
    -          name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds
    +          name: {{ include "altinity-clickhouse-operator.fullname" . }}-crds-2
     {{- end }}
    
  4. Slach commented on Mar 26, 2026

    @Slach
    Collaborator

    @amithalle @madrisan could you create pull request for it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions