Skip to content

add S3 storage support for ClickHouse #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
def merged 1 commit into from
Mar 6, 2026
Merged

add S3 storage support for ClickHouse #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions api/v1/coroot_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
networkingv1 "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

Expand Down Expand Up @@ -149,6 +150,34 @@ type ClickhouseSpec struct {
LogLevel string `json:"logLevel,omitempty"`

Keeper ClickhouseKeeperSpec `json:"keeper,omitempty"`

// S3 storage configuration for ClickHouse (optional).
S3 *ClickhouseS3Spec `json:"s3,omitempty"`
}

type ClickhouseS3Spec struct {
// S3 endpoint URL. E.g., https://s3.amazonaws.com/my-bucket/clickhouse/
Endpoint string `json:"endpoint"`
// S3 region (optional).
Region string `json:"region,omitempty"`
// S3 credentials (optional — omit for IAM/IRSA/workload identity).
Credentials *S3Credentials `json:"credentials,omitempty"`
// Local cache size for S3 reads (default: 10Gi). Must be less than clickhouse storage size.
CacheSize resource.Quantity `json:"cacheSize,omitempty"`
// Storage mode: "tiered" keeps recent data on local disk and moves older data to S3;
// "s3only" stores all data on S3 using local disk only for cache.
// +kubebuilder:validation:Enum="tiered";"s3only"
// +kubebuilder:default="tiered"
Mode string `json:"mode,omitempty"`
// Fraction of local disk free space that triggers moving data to S3 (default: 0.1). Only used in tiered mode.
MoveFactor string `json:"moveFactor,omitempty"`
}

type S3Credentials struct {
// Secret reference for the access key ID.
AccessKeyID *corev1.SecretKeySelector `json:"accessKeyId,omitempty"`
// Secret reference for the secret access key.
SecretAccessKey *corev1.SecretKeySelector `json:"secretAccessKey,omitempty"`
}

type ClickhouseKeeperSpec struct {
Expand Down
51 changes: 51 additions & 0 deletions api/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

86 changes: 86 additions & 0 deletions config/crd/coroot.com_coroots.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3328,6 +3328,92 @@ spec:
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
type: object
s3:
description: S3 storage configuration for ClickHouse (optional).
properties:
cacheSize:
anyOf:
- type: integer
- type: string
description: 'Local cache size for S3 reads (default: 10Gi).
Must be less than clickhouse storage size.'
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
credentials:
description: S3 credentials (optional — omit for IAM/IRSA/workload
identity).
properties:
accessKeyId:
description: Secret reference for the access key ID.
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
secretAccessKey:
description: Secret reference for the secret access key.
properties:
key:
description: The key of the secret to select from. Must
be a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key
must be defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
type: object
endpoint:
description: S3 endpoint URL. E.g., https://s3.amazonaws.com/my-bucket/clickhouse/
type: string
mode:
default: tiered
description: |-
Storage mode: "tiered" keeps recent data on local disk and moves older data to S3;
"s3only" stores all data on S3 using local disk only for cache.
enum:
- tiered
- s3only
type: string
moveFactor:
description: 'Fraction of local disk free space that triggers
moving data to S3 (default: 0.1). Only used in tiered mode.'
type: string
region:
description: S3 region (optional).
type: string
required:
- endpoint
type: object
shards:
type: integer
storage:
Expand Down
77 changes: 77 additions & 0 deletions config/crd/coroot.com_coroots_legacy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2999,6 +2999,83 @@ spec:
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
type: object
s3:
description: S3 storage configuration for ClickHouse (optional).
properties:
cacheSize:
anyOf:
- type: integer
- type: string
description: 'Local cache size for S3 reads (default: 10Gi). Must be less than clickhouse storage size.'
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
type: string
credentials:
description: S3 credentials (optional — omit for IAM/IRSA/workload identity).
properties:
accessKeyId:
description: Secret reference for the access key ID.
properties:
key:
description: The key of the secret to select from. Must be a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key must be defined
type: boolean
required:
- key
type: object
secretAccessKey:
description: Secret reference for the secret access key.
properties:
key:
description: The key of the secret to select from. Must be a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key must be defined
type: boolean
required:
- key
type: object
type: object
endpoint:
description: S3 endpoint URL. E.g., https://s3.amazonaws.com/my-bucket/clickhouse/
type: string
mode:
default: tiered
description: |-
Storage mode: "tiered" keeps recent data on local disk and moves older data to S3;
"s3only" stores all data on S3 using local disk only for cache.
enum:
- tiered
- s3only
type: string
moveFactor:
description: 'Fraction of local disk free space that triggers moving data to S3 (default: 0.1). Only used in tiered mode.'
type: string
region:
description: S3 region (optional).
type: string
required:
- endpoint
type: object
shards:
type: integer
storage:
Expand Down
Loading