diff --git a/content/integrate/redis-data-integration/data-pipelines/deploy.md b/content/integrate/redis-data-integration/data-pipelines/deploy.md index ff33b5b12e..1abe4d4307 100644 --- a/content/integrate/redis-data-integration/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/data-pipelines/deploy.md @@ -16,241 +16,11 @@ type: integration weight: 50 --- -The sections below explain how to deploy a pipeline after you have created the required -[configuration]({{< relref "/integrate/redis-data-integration/data-pipelines" >}}). - -## Set secrets - -Before you deploy your pipeline, you must set the authentication secrets for the -source and target databases. Each secret has a name that you can pass to the -[`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) -command (VM deployment) or the `rdi-secret.sh` script (K8s deployment) to set the secret value. -You can then refer to these secrets in the `config.yaml` file using the syntax "`${SECRET_NAME}`" -(the sample -[config.yaml file]({{< relref "/integrate/redis-data-integration/data-pipelines/pipeline-config#example" >}}) -shows these secrets in use). - -The table below lists all valid secret names. Note that the -username and password are required for the source and target, but the other -secrets are only relevant for TLS/mTLS connections. - -| Secret name | Description | -| :-- | :-- | -| `SOURCE_DB_USERNAME` | Username for the source database | -| `SOURCE_DB_PASSWORD` | Password for the source database | -| `SOURCE_DB_CACERT` | (For TLS only) Source database CA certificate | -| `SOURCE_DB_CERT` | (For mTLS only) Source database client certificate | -| `SOURCE_DB_KEY` | (For mTLS only) Source database private key | -| `SOURCE_DB_KEY_PASSWORD` | (For mTLS only) Source database private key password | -| `TARGET_DB_USERNAME` | Username for the target database | -| `TARGET_DB_PASSWORD` | Password for the target database | -| `TARGET_DB_CACERT` | (For TLS only) Target database CA certificate | -| `TARGET_DB_CERT` | (For mTLS only) Target database client certificate | -| `TARGET_DB_KEY` | (For mTLS only) Target database private key | -| `TARGET_DB_KEY_PASSWORD` | (For mTLS only) Target database private key password | - -{{< note >}} -{{< embed-md "rdi-tls-secrets.md" >}} -{{< /note >}} - -### Set secrets for VM deployment - -Use [`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) -to set secrets for a VM deployment. - -The specific command lines for source secrets are as follows: - -```bash -# For username and password -redis-di set-secret SOURCE_DB_USERNAME yourUsername -redis-di set-secret SOURCE_DB_PASSWORD yourPassword - -# With source TLS, in addition to the above -redis-di set-secret SOURCE_DB_CACERT /path/to/myca.crt - -# With source mTLS, in addition to the above -redis-di set-secret SOURCE_DB_CERT /path/to/myclient.crt -redis-di set-secret SOURCE_DB_KEY /path/to/myclient.key -# Use this only if SOURCE_DB_KEY is password-protected -redis-di set-secret SOURCE_DB_KEY_PASSWORD yourKeyPassword -``` - -The corresponding command lines for target secrets are: - -```bash -# For username and password -redis-di set-secret TARGET_DB_USERNAME yourUsername -redis-di set-secret TARGET_DB_PASSWORD yourPassword - -# With target TLS, in addition to the above -redis-di set-secret TARGET_DB_CACERT /path/to/myca.crt - -# With target mTLS, in addition to the above -redis-di set-secret TARGET_DB_CERT /path/to/myclient.crt -redis-di set-secret TARGET_DB_KEY /path/to/myclient.key -# Use this only if TARGET_DB_KEY is password-protected -redis-di set-secret TARGET_DB_KEY_PASSWORD yourKeyPassword -``` - -### Set secrets for K8s/Helm deployment using the rdi-secret.sh script - -Use the `rdi-secret.sh` script to set secrets for a K8s/Helm deployment. To use this script, unzip the archive that contains the RDI Helm chart and navigate to the resulting folder. The `rdi-secret.sh` script is located in the `scripts` subfolder. The general pattern for using this script is: - -```bash -scripts/rdi-secret.sh set -``` - -The script also lets you retrieve a specific secret or list all the secrets that have been set: - -```bash -# Get specific secret -scripts/rdi-secret.sh get - -# List all secrets -scripts/rdi-secret.sh list -``` - -The specific command lines for source secrets are as follows: - -```bash -# For username and password -scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername -scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword - -# With source TLS, in addition to the above -scripts/rdi-secret.sh set SOURCE_DB_CACERT /path/to/myca.crt - -# With source mTLS, in addition to the above -scripts/rdi-secret.sh set SOURCE_DB_CERT /path/to/myclient.crt -scripts/rdi-secret.sh set SOURCE_DB_KEY /path/to/myclient.key -# Use this only if SOURCE_DB_KEY is password-protected -scripts/rdi-secret.sh set SOURCE_DB_KEY_PASSWORD yourKeyPassword -``` - -The corresponding command lines for target secrets are: - -```bash -# For username and password -scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername -scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword - -# With target TLS, in addition to the above -scripts/rdi-secret.sh set TARGET_DB_CACERT /path/to/myca.crt - -# With target mTLS, in addition to the above -scripts/rdi-secret.sh set TARGET_DB_CERT /path/to/myclient.crt -scripts/rdi-secret.sh set TARGET_DB_KEY /path/to/myclient.key -# Use this only if TARGET_DB_KEY is password-protected -scripts/rdi-secret.sh set TARGET_DB_KEY_PASSWORD yourKeyPassword -``` - -### Set secrets for K8s/Helm deployment using Kubectl command - -In some scenarios, you may prefer to use [`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) -to set secrets for a K8s/Helm deployment. The general pattern of the commands is: - -```bash -kubectl create secret generic \ ---namespace=rdi \ ---from-literal== -``` - -Where `` is either `source-db` for source secrets or `target-db` for target secrets. - -If you use TLS or mTLS for either the source or target databases, you also need to create the `source-db-ssl` and/or `target-db-ssl` K8s secrets that contain the certificates used to establish secure connections. The general pattern of the commands is: - -```bash -kubectl create secret generic -ssl \ ---namespace=rdi \ ---from-file== -``` - -The specific command lines for source secrets are as follows: - -```bash -# Without source TLS -# Create or update source-db secret -kubectl create secret generic source-db --namespace=rdi \ ---from-literal=SOURCE_DB_USERNAME=yourUsername \ ---from-literal=SOURCE_DB_PASSWORD=yourPassword \ ---save-config --dry-run=client -o yaml | kubectl apply -f - - -# With source TLS -# Create of update source-db secret -kubectl create secret generic source-db --namespace=rdi \ ---from-literal=SOURCE_DB_USERNAME=yourUsername \ ---from-literal=SOURCE_DB_PASSWORD=yourPassword \ ---from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \ ---save-config --dry-run=client -o yaml | kubectl apply -f - -# Create or update source-db-ssl secret -kubectl create secret generic source-db-ssl --namespace=rdi \ ---from-file=ca.crt=/path/to/myca.crt \ ---save-config --dry-run=client -o yaml | kubectl apply -f - - -# With source mTLS -# Create or update source-db secret -kubectl create secret generic source-db --namespace=rdi \ ---from-literal=SOURCE_DB_USERNAME=yourUsername \ ---from-literal=SOURCE_DB_PASSWORD=yourPassword \ ---from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \ ---from-literal=SOURCE_DB_CERT=/etc/certificates/source_db/client.crt \ ---from-literal=SOURCE_DB_KEY=/etc/certificates/source_db/client.key \ ---from-literal=SOURCE_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if SOURCE_DB_KEY is password-protected ---save-config --dry-run=client -o yaml | kubectl apply -f - -# Create or update source-db-ssl secret -kubectl create secret generic source-db-ssl --namespace=rdi \ ---from-file=ca.crt=/path/to/myca.crt \ ---from-file=client.crt=/path/to/myclient.crt \ ---from-file=client.key=/path/to/myclient.key \ ---save-config --dry-run=client -o yaml | kubectl apply -f - -``` - -The corresponding command lines for target secrets are: - -```bash -# Without target TLS -# Create or update target-db secret -kubectl create secret generic target-db --namespace=rdi \ ---from-literal=TARGET_DB_USERNAME=yourUsername \ ---from-literal=TARGET_DB_PASSWORD=yourPassword \ ---save-config --dry-run=client -o yaml | kubectl apply -f - - -# With target TLS -# Create of update target-db secret -kubectl create secret generic target-db --namespace=rdi \ ---from-literal=TARGET_DB_USERNAME=yourUsername \ ---from-literal=TARGET_DB_PASSWORD=yourPassword \ ---from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \ ---save-config --dry-run=client -o yaml | kubectl apply -f - -# Create or update target-db-ssl secret -kubectl create secret generic target-db-ssl --namespace=rdi \ ---from-file=ca.crt=/path/to/myca.crt \ ---save-config --dry-run=client -o yaml | kubectl apply -f - - -# With target mTLS -# Create or update target-db secret -kubectl create secret generic target-db --namespace=rdi \ ---from-literal=TARGET_DB_USERNAME=yourUsername \ ---from-literal=TARGET_DB_PASSWORD=yourPassword \ ---from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \ ---from-literal=TARGET_DB_CERT=/etc/certificates/target_db/client.crt \ ---from-literal=TARGET_DB_KEY=/etc/certificates/target_db/client.key \ ---from-literal=TARGET_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if TARGET_DB_KEY is password-protected ---save-config --dry-run=client -o yaml | kubectl apply -f - -# Create or update target-db-ssl secret -kubectl create secret generic target-db-ssl --namespace=rdi \ ---from-file=ca.crt=/path/to/myca.crt \ ---from-file=client.crt=/path/to/myclient.crt \ ---from-file=client.key=/path/to/myclient.key \ ---save-config --dry-run=client -o yaml | kubectl apply -f - -``` - -Note that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets. - -## Deploy a pipeline - -When you have created your configuration, including the [jobs]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}), you are -ready to deploy. Use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) +When you have created your configuration, including the +[jobs]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}), +and [set the authentication secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}) +for your source and target databases, you are ready to deploy. Use +[Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) to configure and deploy pipelines for both VM and K8s installations. For VM installations, you can also use the @@ -259,4 +29,4 @@ command to deploy a pipeline: ```bash redis-di deploy --dir -``` \ No newline at end of file +``` diff --git a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md index cbc3ebdf77..c86bfcfbb6 100644 --- a/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md +++ b/content/integrate/redis-data-integration/data-pipelines/pipeline-config.md @@ -25,7 +25,7 @@ if you want to apply custom transformations to the captured data. Below is an example of a `config.yaml` file. Note that the values of the form "`${name}`" refer to secrets that you should set as described in -[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets" >}}). +[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}). In particular, you should normally use secrets as shown to set the source and target username and password rather than storing them in plain text in this file. @@ -210,7 +210,7 @@ configuration contains the following data: to the source database, you may need to specify additional properties in the `advanced` section with references to the corresponding certificates depending on the source database type. Note that these properties **must** be references to - secrets that you should set as described in [Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets" >}}). + secrets that you should set as described in [Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}). - `databases`: List of all databases to collect data from for source database types that support multiple databases, such as `mysql` and `mariadb`. - `schemas`: List of all schemas to collect data from for source database types @@ -253,7 +253,7 @@ or [mTLS](https://en.wikipedia.org/wiki/Mutual_authentication#mTLS) to connect to the target database, you must specify the CA certificate (for TLS), and the client certificate and private key (for mTLS) in `cacert`, `cert`, and `key`. Note that these certificates **must** be references to secrets -that you should set as described in [Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets" >}}) +that you should set as described in [Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}) (it is not possible to include these certificates as plain text in the file). {{< note >}}If you specify `localhost` as the address of either the source or target server during diff --git a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md index 1e5b16966d..da15c95d69 100644 --- a/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md +++ b/content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md @@ -147,7 +147,7 @@ RDI supports two authentication methods for accessing Spanner: Before deploying the RDI pipeline, you need to configure the necessary secrets for the target database. Instructions for setting up the target database secrets are available in the -[RDI deployment guide]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets-for-k8shelm-deployment-using-kubectl-command" >}}). +[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets#set-secrets-for-k8shelm-deployment-using-kubectl-command" >}}). **Optional**: If you prefer to use a service account credentials file instead of Workload Identity authentication, you'll need to create a Spanner-specific secret named `source-db-credentials`. diff --git a/content/integrate/redis-data-integration/data-pipelines/secret-providers.md b/content/integrate/redis-data-integration/data-pipelines/secret-providers.md new file mode 100644 index 0000000000..1cedc29f82 --- /dev/null +++ b/content/integrate/redis-data-integration/data-pipelines/secret-providers.md @@ -0,0 +1,142 @@ +--- +Title: Using an external secret provider +alwaysopen: false +categories: +- docs +- integrate +- rs +- rdi +description: | + Configure RDI to obtain authentication secrets for your source and target databases + from an external provider. +group: di +linkTitle: External secret providers +summary: Redis Data Integration keeps Redis in sync with the primary database in near + real time. +type: integration +weight: 49 +--- + +For K8s deployments, you can use an external secret provider, such as +[Vault](https://developer.hashicorp.com/vault) or +[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) to provide +the authentication secrets for your source and target databases. +See the sections below to learn how to do this. If you prefer to set the secrets for RDI manually, see +[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}) +for more information. + +{{< note >}} +This page is a work in progress. The sections marked **TODO** below need +confirmation from the RDI developers before the page is published. Please +review and comment. +{{< /note >}} + +## Configure an external provider + +You define a secret provider in the `secret-providers` section of your +[`config.yaml`]({{< relref "/integrate/redis-data-integration/data-pipelines/pipeline-config" >}}) +file. Each provider has: + +- a unique **id** (the key under `secret-providers`) that you use to refer to it +- a **`type`** (`vault` or `aws`) +- a **`parameters`** object that configures the connection to the provider and + lists the secret **`objects`** to fetch from it. + +Each entry in `objects` maps a secret stored in the provider to a name that you +can reference elsewhere in the configuration: + +```yaml +secret-providers: + my-provider: + type: vault # or "aws" + parameters: + # connection parameters (see the sections below) + objects: + - objectName: credentials # the name you reference + secretPath: secret/data/db-pass + secretKey: password +``` + +You then reference a fetched secret using the syntax +`${secret:::}`. For example: + +```yaml +connection: + user: ${secret:my-provider:credentials:user} + password: ${secret:my-provider:credentials:password} +``` + +Note that this differs from the `${SECRET_NAME}` syntax used for +[secrets that you set manually]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}). + +> **TODO (needs dev input):** Confirm where a provider can be defined. The +> `config.yaml` reference lists `secret-providers` as a config section, but the +> API also exposes `PUT /api/v1/pipelines/secret-providers/{name}`. Can +> providers be configured in `config.yaml`, through the API, and/or in Redis +> Insight? + +## Vault + +Set `type: vault` and provide the following connection parameters: + +| Parameter | Description | +| :-- | :-- | +| `vaultAddress` | URL of the Vault server, for example `http://vault.default:8200`. | +| `roleName` | The Vault role that RDI uses to authenticate. | + +Each entry in `objects` has the following fields: + +| Field | Description | +| :-- | :-- | +| `objectName` | The name you use to reference the secret in the `${secret:...}` syntax. | +| `secretPath` | The path to the secret in Vault, for example `secret/data/db-pass`. | +| `secretKey` | The key within the secret whose value you want to read. | + +For example: + +```yaml +secret-providers: + my-vault: + type: vault + parameters: + vaultAddress: http://vault.default:8200 + roleName: database + objects: + - objectName: credentials + secretPath: secret/data/db-pass + secretKey: password +``` + +> **TODO (needs dev input):** +> +> - Which Vault **authentication method** does RDI use (token, Kubernetes auth, +> AppRole)? How are the credentials or token supplied? The `roleName` +> parameter suggests a role-based method. +> - The API example includes an extra `someField` parameter. What is the +> complete, real list of `parameters` for the Vault provider? +> - Are there any TLS options for the connection to Vault? + +## AWS Secrets Manager + +Set `type: aws`. + +> **TODO (needs dev input):** The `config.yaml` reference and the API schema +> currently document only `type: aws` with no parameters, so this section needs +> the full parameter set from the RDI developers, including: +> +> - **Region** and any endpoint configuration. +> - **Authentication** — IAM role, IRSA, or static access keys? +> - The **`objects`** mapping fields — the AWS equivalents of `objectName`, the +> secret name or ARN, and the key or JSON field to read. + +## Secret rotation + +*Secret rotation* is a technique where secrets are changed automatically +by the provider according to a schedule. +RDI versions 1.10.0 and above let you configure the pipeline to +restart the appropriate K8s pods automatically whenever a secret rotates in +the external provider that you have configured. + +> **TODO (needs dev input):** Document how to enable the automatic pod restart — +> which `config.yaml` setting controls it, and whether it has any per-provider +> requirements. diff --git a/content/integrate/redis-data-integration/data-pipelines/set-secrets.md b/content/integrate/redis-data-integration/data-pipelines/set-secrets.md new file mode 100644 index 0000000000..63e82a1fe6 --- /dev/null +++ b/content/integrate/redis-data-integration/data-pipelines/set-secrets.md @@ -0,0 +1,247 @@ +--- +Title: Set secrets +alwaysopen: false +categories: +- docs +- integrate +- rs +- rdi +description: Set authentication secrets for your source and target databases. +group: di +linkTitle: Set secrets +summary: Redis Data Integration keeps Redis in sync with the primary database in near + real time. +type: integration +weight: 48 +--- + +Before you +[deploy]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy" >}}) +your pipeline, you must set the authentication secrets for the +source and target databases. Each secret has a name that you can pass to the +[`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) +command (VM deployment) or the `rdi-secret.sh` script (K8s deployment) to set the secret value. +For K8s, you can also configure RDI to obtain the secrets from an +[external provider]({{< relref "/integrate/redis-data-integration/data-pipelines/secret-providers" >}}). + +The `config.yaml` file accesses these secrets with the syntax "`${SECRET_NAME}`" +(the sample +[config.yaml file]({{< relref "/integrate/redis-data-integration/data-pipelines/pipeline-config#example" >}}) +shows the secrets in use). + +The table below lists all valid secret names. Note that the +username and password are required for the source and target, but the other +secrets are only relevant for TLS/mTLS connections. + +| Secret name | Description | +| :-- | :-- | +| `SOURCE_DB_USERNAME` | Username for the source database | +| `SOURCE_DB_PASSWORD` | Password for the source database | +| `SOURCE_DB_CACERT` | (For TLS only) Source database CA certificate | +| `SOURCE_DB_CERT` | (For mTLS only) Source database client certificate | +| `SOURCE_DB_KEY` | (For mTLS only) Source database private key | +| `SOURCE_DB_KEY_PASSWORD` | (For mTLS only) Source database private key password | +| `TARGET_DB_USERNAME` | Username for the target database | +| `TARGET_DB_PASSWORD` | Password for the target database | +| `TARGET_DB_CACERT` | (For TLS only) Target database CA certificate | +| `TARGET_DB_CERT` | (For mTLS only) Target database client certificate | +| `TARGET_DB_KEY` | (For mTLS only) Target database private key | +| `TARGET_DB_KEY_PASSWORD` | (For mTLS only) Target database private key password | + +{{< note >}} +{{< embed-md "rdi-tls-secrets.md" >}} +{{< /note >}} + +## Set secrets for VM deployment + +Use [`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) +to set secrets for a VM deployment. + +The specific command lines for source secrets are as follows: + +```bash +# For username and password +redis-di set-secret SOURCE_DB_USERNAME yourUsername +redis-di set-secret SOURCE_DB_PASSWORD yourPassword + +# With source TLS, in addition to the above +redis-di set-secret SOURCE_DB_CACERT /path/to/myca.crt + +# With source mTLS, in addition to the above +redis-di set-secret SOURCE_DB_CERT /path/to/myclient.crt +redis-di set-secret SOURCE_DB_KEY /path/to/myclient.key +# Use this only if SOURCE_DB_KEY is password-protected +redis-di set-secret SOURCE_DB_KEY_PASSWORD yourKeyPassword +``` + +The corresponding command lines for target secrets are: + +```bash +# For username and password +redis-di set-secret TARGET_DB_USERNAME yourUsername +redis-di set-secret TARGET_DB_PASSWORD yourPassword + +# With target TLS, in addition to the above +redis-di set-secret TARGET_DB_CACERT /path/to/myca.crt + +# With target mTLS, in addition to the above +redis-di set-secret TARGET_DB_CERT /path/to/myclient.crt +redis-di set-secret TARGET_DB_KEY /path/to/myclient.key +# Use this only if TARGET_DB_KEY is password-protected +redis-di set-secret TARGET_DB_KEY_PASSWORD yourKeyPassword +``` + +## Set secrets for K8s/Helm deployment using the rdi-secret.sh script + +Use the `rdi-secret.sh` script to set secrets for a K8s/Helm deployment. To use this script, unzip the archive that contains the RDI Helm chart and navigate to the resulting folder. The `rdi-secret.sh` script is located in the `scripts` subfolder. The general pattern for using this script is: + +```bash +scripts/rdi-secret.sh set +``` + +The script also lets you retrieve a specific secret or list all the secrets that have been set: + +```bash +# Get specific secret +scripts/rdi-secret.sh get + +# List all secrets +scripts/rdi-secret.sh list +``` + +The specific command lines for source secrets are as follows: + +```bash +# For username and password +scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername +scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword + +# With source TLS, in addition to the above +scripts/rdi-secret.sh set SOURCE_DB_CACERT /path/to/myca.crt + +# With source mTLS, in addition to the above +scripts/rdi-secret.sh set SOURCE_DB_CERT /path/to/myclient.crt +scripts/rdi-secret.sh set SOURCE_DB_KEY /path/to/myclient.key +# Use this only if SOURCE_DB_KEY is password-protected +scripts/rdi-secret.sh set SOURCE_DB_KEY_PASSWORD yourKeyPassword +``` + +The corresponding command lines for target secrets are: + +```bash +# For username and password +scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername +scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword + +# With target TLS, in addition to the above +scripts/rdi-secret.sh set TARGET_DB_CACERT /path/to/myca.crt + +# With target mTLS, in addition to the above +scripts/rdi-secret.sh set TARGET_DB_CERT /path/to/myclient.crt +scripts/rdi-secret.sh set TARGET_DB_KEY /path/to/myclient.key +# Use this only if TARGET_DB_KEY is password-protected +scripts/rdi-secret.sh set TARGET_DB_KEY_PASSWORD yourKeyPassword +``` + +## Set secrets for K8s/Helm deployment using Kubectl command + +In some scenarios, you may prefer to use [`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) +to set secrets for a K8s/Helm deployment. The general pattern of the commands is: + +```bash +kubectl create secret generic \ +--namespace=rdi \ +--from-literal== +``` + +Where `` is either `source-db` for source secrets or `target-db` for target secrets. + +If you use TLS or mTLS for either the source or target databases, you also need to create the `source-db-ssl` and/or `target-db-ssl` K8s secrets that contain the certificates used to establish secure connections. The general pattern of the commands is: + +```bash +kubectl create secret generic -ssl \ +--namespace=rdi \ +--from-file== +``` + +The specific command lines for source secrets are as follows: + +```bash +# Without source TLS +# Create or update source-db secret +kubectl create secret generic source-db --namespace=rdi \ +--from-literal=SOURCE_DB_USERNAME=yourUsername \ +--from-literal=SOURCE_DB_PASSWORD=yourPassword \ +--save-config --dry-run=client -o yaml | kubectl apply -f - + +# With source TLS +# Create of update source-db secret +kubectl create secret generic source-db --namespace=rdi \ +--from-literal=SOURCE_DB_USERNAME=yourUsername \ +--from-literal=SOURCE_DB_PASSWORD=yourPassword \ +--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \ +--save-config --dry-run=client -o yaml | kubectl apply -f - +# Create or update source-db-ssl secret +kubectl create secret generic source-db-ssl --namespace=rdi \ +--from-file=ca.crt=/path/to/myca.crt \ +--save-config --dry-run=client -o yaml | kubectl apply -f - + +# With source mTLS +# Create or update source-db secret +kubectl create secret generic source-db --namespace=rdi \ +--from-literal=SOURCE_DB_USERNAME=yourUsername \ +--from-literal=SOURCE_DB_PASSWORD=yourPassword \ +--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \ +--from-literal=SOURCE_DB_CERT=/etc/certificates/source_db/client.crt \ +--from-literal=SOURCE_DB_KEY=/etc/certificates/source_db/client.key \ +--from-literal=SOURCE_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if SOURCE_DB_KEY is password-protected +--save-config --dry-run=client -o yaml | kubectl apply -f - +# Create or update source-db-ssl secret +kubectl create secret generic source-db-ssl --namespace=rdi \ +--from-file=ca.crt=/path/to/myca.crt \ +--from-file=client.crt=/path/to/myclient.crt \ +--from-file=client.key=/path/to/myclient.key \ +--save-config --dry-run=client -o yaml | kubectl apply -f - +``` + +The corresponding command lines for target secrets are: + +```bash +# Without target TLS +# Create or update target-db secret +kubectl create secret generic target-db --namespace=rdi \ +--from-literal=TARGET_DB_USERNAME=yourUsername \ +--from-literal=TARGET_DB_PASSWORD=yourPassword \ +--save-config --dry-run=client -o yaml | kubectl apply -f - + +# With target TLS +# Create of update target-db secret +kubectl create secret generic target-db --namespace=rdi \ +--from-literal=TARGET_DB_USERNAME=yourUsername \ +--from-literal=TARGET_DB_PASSWORD=yourPassword \ +--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \ +--save-config --dry-run=client -o yaml | kubectl apply -f - +# Create or update target-db-ssl secret +kubectl create secret generic target-db-ssl --namespace=rdi \ +--from-file=ca.crt=/path/to/myca.crt \ +--save-config --dry-run=client -o yaml | kubectl apply -f - + +# With target mTLS +# Create or update target-db secret +kubectl create secret generic target-db --namespace=rdi \ +--from-literal=TARGET_DB_USERNAME=yourUsername \ +--from-literal=TARGET_DB_PASSWORD=yourPassword \ +--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \ +--from-literal=TARGET_DB_CERT=/etc/certificates/target_db/client.crt \ +--from-literal=TARGET_DB_KEY=/etc/certificates/target_db/client.key \ +--from-literal=TARGET_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if TARGET_DB_KEY is password-protected +--save-config --dry-run=client -o yaml | kubectl apply -f - +# Create or update target-db-ssl secret +kubectl create secret generic target-db-ssl --namespace=rdi \ +--from-file=ca.crt=/path/to/myca.crt \ +--from-file=client.crt=/path/to/myclient.crt \ +--from-file=client.key=/path/to/myclient.key \ +--save-config --dry-run=client -o yaml | kubectl apply -f - +``` + +Note that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets.