Skip to main content

KOBIL SHIFT

KOBIL SHIFT helm chart 0.277.0

[[TOC]]

Prerequisites​

KOBIL Shift has the following dependencies:

  • Kubernetes version 1.23 - 1.34. Currently tested with 1.31 and 1.34.

  • Istio service mesh: Currently tested with 1.25.3 and 1.30.2. When using the mutual TLS use case, the minimum required version is 1.19.0.

  • Apache Kafka: Shift contains resources to create a Kafka cluster with Strimzi Kafka operator. This requires Strimzi Kafka operator version 0.47.0 with support for Kafka 4.0.0. Alternatively an external Kafka cluster can be used. See Section External Kafka clusters for details.

  • Redis: Shift supports Redis Standalone and Redis Cluster versions 7.x and 8.x.

  • Databases:

    • Most Shift components require a PostgreSQL database. Supported versions are 13.x and 16.x. Note: scram-sha-256 password hashing is NOT supported when using smartdashboard components.
    • Components of Shift-Lite (ast components, scp-notifier, idp components, otp-management) also support Oracle 19c.
    • SCP chat components (scp-addressbook, scp-presence, scp-messenger, scp-media, scp-gateway) require MongoDB version 4.4 or 6.

  • The Mercury Chat Platform has the following additional dependencies:

    • Centrifugo real-time messaging server. Currently tested with version 6.1.0.
    • S3 compatible object storage. Currently tested with AWS S3, Minio version RELEASE.2024-11-07T00-52-20Z, and SeaweedFS version 4.17.0.
    • ScyllaDB. Currently tested with ScyllaDB Enterprise version 2024.2.5.

    See Section Mercury for additional details.

KOBIL Shift runs on Red Hat OpenShift and supports the Red Hat variants of Istio OpenShift Service Mesh and Strimzi Kafka operator Streams for Apache Kafka. It is currently tested with the following versions

KOBIL Shift has the following additional requirements on the infrastructure:

  • Shift requires correct and synchronized system clocks. Shift uses X.509 certificates and JSON Web Tokens (JWT) that contain 'valid from' and 'valid until' timestamps. Inaccurate clocks can lead to rejection of certificates or JWTs because they appear to be expired or not valid yet.
  • The network infrastructure must support Server-sent events (SSE). In particular, application load balancers must be configured to immediately forward partial http responses to the clients and not perform any kind of response caching. Otherwise, delivery of server sent events to the client might be delayed.

Further requirements to install Shift:

  • Helm version 3.x.

  • Access to KOBIL chart museum

    helm repo add kobil https://charts.kobil.com --username {chart_username} --password {chart_password}

  • An imagePullSecret providing access to the relevant repositories at Azure (kobilsystems.azurecr.io)

    kubectl create secret docker-registry registry-azure \
    --docker-server=kobilsystems.azurecr.io \
    --docker-username=azure_user_name \
    --docker-password=azure_password

  • KOBIL Shift operator

Deploy KOBIL Shift-Operator​

KOBIL Shift Operator charts are available at https://charts.kobil.com. Before deployment, configure image pull secret, Docker image registry, and Helm Chart repository credentials in configuration file shift-operator-values.yaml:

global:
  imagePullSecrets:
    - registry-azure

  registry: kobilsystems.azurecr.io

helmRepo:
  url: https://charts.kobil.com
  username: ""
  password: ""
helm install shift-operator -f shift-operator-values.yaml -n shift kobil/shift-operator --version x.y.z

Verify that KOBIL Shift-Operator is running by executing:

kubectl -n shift get deployments

Also verify that the custom resource definition servicegroups.shift.kobil.com is available by executing kubectl get crd.

Deploy KOBIL Shift​

The next step is to deploy KOBIL Shift in the same namespace where the Shift Operator is running.

Set appropriate configuration for the KOBIL Shift services in configuration file shift-values.yaml. The included values.yaml can be used as template. See section Values and Issuer CA for additional information.

helm install shift -f shift-values.yaml -n shift kobil/shift --version x.y.z

Deploying Shift chart creates multiple servicegroups.shift.kobil.com objects which are managed by Shift Operator. Use

kubectl -n shift get servicegroups.shift.kobil.com

to obtain an overview of the deployed servicegroups. The READY column shows the status of the servicegroup. The status changes to true, once Shift Operator successfully deployed all services in the servicegroup and the corresponding workloads are in a ready state.

If a servicegroup fails to become ready, use command

kubectl -n shift describe servicegroups.shift.kobil.com <name-of-servicegroup>

to obtain information about the deployment error.

In addition, Shift chart creates resources which are managed by Strimzi Kafka Operator and Istio. The status of the Kafka cluster can be observed using command

kubectl -n shift get kafkas.kafka.strimzi.io

The deployed Istio resources can be viewed using commands

kubectl -n shift get gateways.networking.istio.io
kubectl -n shift get virtualservices.networking.istio.io
kubectl -n shift get destinationrules.networking.istio.io

Post deployment configuration​

The following settings need to be performed in IDP UI. Open https://idp.{{global.routing.domain}}/auth/admin and login using IDP master admin credentials:

username: {{ global.idp.adminUser.username }}
password: {{ global.idp.adminUser.passsord }}

Configure SMTP settings under 'Realm Settings -> Email'.

Create confidential OIDC client workspacemanagement and configure its client_id and client_secret in file shift-values.yaml.

smartdashboardKongConfigurationBackend:

# -- client_id and client_secret of an OIDC client in IDP master realm. Must be manually created.
  config:
    masterClientId: "workspacemanagement"
    masterClientSecret: "client_secret"

Then execute command

helm upgrade shift -f shift-values.yaml -n shift kobil/shift --version x.y.z

Open URL https://smartdashboard.{{global.routing.domain}}/dashboard/master/workspace-management to access Kobil Portal.

Issuer CA​

Part of every SHIFT deployment are multiple identities as Certificate Authorities (CA).

  • A CA must have a key pair (CA Key Pair).
  • A CA must have a digital certificate (CA Certificate) signed either by its own private key, as a self-signed root certificate, or be signed by another CA's private key, as an intermediate certificate.
  • The main CA identity of a SHIFT deployment will be named Issuer CA.
  • The main CA identity's certificate will be named Issuer CA Certificate.
  • The main CA identity's key pair will be named Issuer CA Key Pair.
  • A sub CA Certificate of SHIFT deployments, which are signed by the Issuer CA's private key will be named Tenant Signer CA Certificate.
  • Sub CA key pairs will be named Tenant Signer CA Key Pair.

An Issuer CA Certificate and Issuer CA Key Pair must be generated for each SHIFT deployment. The Issuer CA Key Pair cannot be changed afterwards. Generate the Issuer CA Certificate according to the following instructions.

  • The Issuer CA Key Pair must be in PKCS#8 format. Both the Issuer CA Certificate and Issuer CA Key Pair must be DER-encoded
  • The Issuer CA Key Pair must be of one of the supported algorithms:
    • RSA with >= 2048 bit keys
    • ECDSA with one of the supported curves:
      • secp256r1 (or P-256), secp384r1 (or P-384), secp521r1 (or P-521)
    • Ed25519
  • Set the key algorithm used by the Issuer CA Key Pair using ca.signers.key_generation.algorithm. If using an intermediate certificate, it is recommended to use the same key algorithm that is configured for the root certificate. The curve parameter (for ECDSA), or the strength parameter (for RSA), can be set to a more secure configuration, compared to the root certificate, to provide additional security.
  • The Issuer CA Certificate must have the Basic Constraints Certificate Extension with CA=True and pathLen unset or >= 1. The Certificate Extension must be marked as critical.
  • The Issuer CA Certificate must have the Key Usage Certificate Extension with at least the bits for keyCertSign and cRLSign set. The Certificate Extension must be marked as critical. Other usage bits should not be set.
  • The Issuer CA Certificate must have specific Certificate Extension policies set. These Certificate Extensions depend on the feature sets that are required. Alternatively, the Certificate Extensions may specify anyPolicy. These Certificate Extension policies should not be marked critical. The following Certificate Extension policies are supported:
    • Base policy with OID 1.3.6.1.4.1.14481.109.4.1. This policy must always be added.
    • SCP policy with OID 1.3.6.1.4.1.14481.109.4.2. This policy is needed when scp services are deployed and messaging features are used.
    • mTLS policy with OID 1.3.6.1.4.1.14481.109.4.3. This policy is needed when ast services are configured to enforce mutualTLS communication with clients.
  • The above policies are umbrella policies, combining multiple single policies, required by the associated feature set. It is recommended to use these umbrella policies. They combine the following single policies:
    • Base policy contains
      • 1.3.6.1.4.1.14481.109.1.0 (profile LEAF_CA)
      • 1.3.6.1.4.1.14481.109.1.4 (profile AST_DEVICE)
    • SCP policy contains
      • 1.3.6.1.4.1.14481.109.1.1 (profile SIGNATURE)
      • 1.3.6.1.4.1.14481.109.1.2 (profile AUTHENTICATION)
      • 1.3.6.1.4.1.14481.109.1.3 (profile ENCRYPTION)
      • 1.3.6.1.4.1.14481.109.1.6 (profile SIGNATURE_GATEWAY)
      • 1.3.6.1.4.1.14481.109.1.7 (profile AUTHENTICATION_GATEWAY)
      • 1.3.6.1.4.1.14481.109.1.8 (profile ENCRYPTION_GATEWAY)
    • mTLS policy contains
      • 1.3.6.1.4.1.14481.109.1.9 (profile TLS_CLIENT)
      • 1.3.6.1.4.1.14481.109.1.10 (profile TLS_CLIENT_AND_KEY)
  • The Issuer CA Certificate may have the Extended Key Usage Certificate Extension with the id_kp_OCSPSigning key purpose set. Other key purpose IDs should not be set.
  • Other Certificate Extensions should not be present.

Below is a simple example how to generate an Issuer CA Certificate as a self-signed root certificate using OpenSSL. The example generates an Issuer CA Certificate for all three umbrella policies described above.

  • Create file openssl.cnf with the following content

    [req]
    default_bits = 4096
    encrypt_key  = no
    default_md   = sha512
    prompt       = no
    utf8         = yes
    x509_extensions = v3_req
    distinguished_name = req_distinguished_name

    # Adjust below values as required
    [req_distinguished_name]
    C  = DE
    ST = Rheinland-Pfalz
    L  = Worms
    O  = KOBIL GmbH
    CN = KOBIL Shift Issuer CA

    [v3_req]
    basicConstraints = critical, CA:TRUE, pathlen:1
    keyUsage = critical, keyCertSign, cRLSign
    # explicit policies
    certificatePolicies = 1.3.6.1.4.1.14481.109.4.1, 1.3.6.1.4.1.14481.109.4.2, 1.3.6.1.4.1.14481.109.4.3
    # or alternatively anyPolicy
    # certificatePolicies = 2.5.29.32.0

  • Create an ECDSA key-pair for curve P-521, convert it to PKCS#8 format, and store it in file key.der.

    openssl ecparam -name P-521 -genkey -noout -outform DER | openssl pkcs8 -inform DER -topk8 -nocrypt -outform DER -out key.der

  • Create a self-signed certificate with a validity of 10 years for the public key generated in the previous step and store it in file cert.der.

    openssl req -nodes -x509 -days 3650 -config openssl.cnf -key key.der -keyform DER -out cert.der -outform DER

  • Base64 encode key and certificate. The content of resulting files key.b64 and cert.b64 can be added to values common.ast.issuer.key and common.ast.issuer.certs, respectively.

    openssl enc -a -A -in key.der -out key.b64
    openssl enc -a -A -in cert.der -out cert.b64

Updating policies​

It is possible to change the supported Certificate Extension policies of the Issuer CA Certificate. This can be done by reissuing the Issuer CA Certificate with the updated Certificate Extension policies.

When reissuing the Issuer CA Certificate, the Issuer CA Key Pair, both the public and private keys, must not be changed.

When using the above OpenSSL example, edit the openssl.cnf config file and adjust the policies accordingly. Then reissue the Issuer CA Certificate using the command:

openssl req -nodes -x509 -days 3650 -config openssl.cnf -key key.der -keyform DER -out cert.der -outform DER

Then base64 encode it using command

openssl enc -a -A -in cert.der -out cert.b64

Then update the value common.ast.issuer.certs with content of file cert.b64.

After updating the Certificate Extension policies of the Issuer CA Certificate, existing Tenant Signer CA Certificates must be manually updated using the following instructions:

  • For each tenant, in which a Tenant Signer CA Certificate exists, obtain an access token with admin write privileges

    • In the default permission configuration, the required role is ks-management/Admin
    • If the permission configuration was changed from the default, use a token with any of the api.security.jwtAuth.external.writeAccessRoles from the AST-CA service's values

  • Execute PATCH /v1/tenants/<tenant>/signers/admin with the admin token and an empty request body, e.g.

    curl -X 'PATCH' https://asts.example.com/v1/tenants/<tenant>/signers/admin \
      --header "authorization: bearer <token>"

  • If successful, the AST-CA service returns a body that looks like this:

    {
      "id": "<the new signer ID>",
      "tenant": "<tenant>",
      "name": "<signer name>"
    }

  • The AST-CA Service has enqueued the Tenant Signer CA Certificate to be reissued and will recreate it using the same Tenant Signer CA Key Pair

  • It is recommended to recreate the SDK Config JWT for tenants in which the Tenant Signer CA Certificate as been reissued, but it is not required.

Mutual TLS​

Shift supports a mode where clients are forced to perform mutual TLS authentication when accessing certain endpoints. This feature is configured using values section common.mutualTLS:.

Set common.mutualTLS.services.enabled: true to enable this feature. When enabled, the first instance that terminates TLS for client traffic must be configured to optionally perform mutual TLS authentication.

The trusted CA certificates to use when verifying client certificates must be the same certificates provided in value common.ast.issuer.certs or alternatively in the existing secret common.ast.issuer.existingSecretIssuerCa.

Client certificates must be added to the request headers of requests which are forwarded to upstream services. The names of the request header for client certificates must be specified as a list using the value common.mutualTLS.services.certRequestHeaders:. Multiple header names are supported.

common:
  mutualTLS:
    services:
      enabled: true
      certRequestHeaders:
        - "x-forwarded-client-cert"

In case Istio Ingress Gateway is the first instance that terminates TLS for client traffic, set common.mutualTLS.istioIngressGateway.enabled: true to configure it for mutual TLS.

The trusted CA certificates to use when verifying client certificates must also be configured. The required format is a single line base64 encoded list of certificates in PEM format. Either provide them directly using value common.mutualTLS.istioIngressGateway.cacerts: or manually put them in a Kubernetes secret and set common.mutualTLS.istioIngressGateway.useExistingCaCertsSecret: true. The name of the existing secret must be {{ .Values.global.routing.tlsSecret }}-cacert, e.g. tls-secret-cacert when using the default.

Note: When using the traffic routing configuration described in Section Traffic routing, the Kubernetes secret containing the trusted CA certificates must be created manually and placed in the namespace where the Istio Ingress Gateway workload is running. Using the parameter common.mutualTLS.istioIngressGateway.cacerts: to do so is no longer supported. For each enabled Gateway one secret must be created. The name of the CA certificate secrets must be the name of the Gateway TLS secrets, with -cacert appended. For example, when using

global:
  routing:
    istio:
      apiGroups:
        gateways:
          public:
            tlsSecretName: tls-secret-public
          integration:
            tlsSecretName: tls-secret-integration
          admin:
            tlsSecretName: tls-secret-admin

The names of the Kubernetes secrets, containing the trusted CA certificates, are: tls-secret-public-cacert, tls-secret-integration-cacert, and tls-secret-admin-cacert.

When using mutual TLS on the Istio Ingress Gateway, the value of common.mutualTLS.services.certRequestHeaders: must not be changed.

common:
  mutualTLS:
    services:
      enabled: true

    istioIngressGateway:
      enabled: true
      useExistingCaCertsSecret: true

Example for an existing CA certificates secret.

apiVersion: v1
kind: Secret
metadata:
  name: tls-secret-cacert
type: Opaque
data:
  cacert: "single line base64 encoded list of certificates in PEM format"

When using the mutual TLS feature, the certificate policy mTLS must be added to the Issuer CA Certificate. See Section Issuer CA for details on the policies. See Section Updating policies for details on how to add the mTLS policy to an existing Issuer CA Certificate and how to update existing Tenant Signer CA Certificates.

Kafka clusters generated using Strimzi Kafka Operator​

Sizing​

The size of the Kafka cluster can be configured using the value strimzi.sizing.mode. Supported values are 'basic', 'tuned', 'custom'. 'Basic' and 'tuned' are presets (preset details described below).

When using mode 'custom', values strimzi.sizing.custom.kafka and strimzi.sizing.custom.controller must be specified. Also, see the documentation on sizing and configuration

Note: Changing the sizing mode after deployment is highly discouraged, as it effects the topic replica count and partition assignment to nodes. It can even lead to data loss.

  • Performance mode 'basic' corresponds to the following configuration

    strimzi:
      sizing:
        mode: "custom"
        custom:
          kafka:
            replicas: 1
            resources:
              requests:
                memory: 2Gi
                cpu: "100m"
              limits:
                memory: 2Gi
            jvmOptions:
              -Xms: 1024m
              -Xmx: 1024m
            config:
              auto.create.topics.enable: "false"
              delete.topic.enable: "false"
              default.replication.factor: 1
              min.insync.replicas: 1
              offsets.topic.replication.factor: 1
              transaction.state.log.replication.factor: 1
              transaction.state.log.min.isr: 1
          controller:
            replicas: 1
            resources:
              requests:
                memory: 768Mi
                cpu: "50m"
              limits:
                memory: 768Mi
            jvmOptions:
              -Xms: 512m
              -Xmx: 512m

  • Performance mode 'tuned' corresponds to the following configuration

    strimzi:
      sizing:
        mode: "custom"
        custom:
          kafka:
            replicas: 3
            resources:
              requests:
                memory: 8Gi
                cpu: "2"
              limits:
                memory: 8Gi
            jvmOptions:
              -Xms: 4096m
              -Xmx: 4096m
            config:
              auto.create.topics.enable: "false"
              delete.topic.enable: "false"
              default.replication.factor: 3
              min.insync.replicas: 2
              offsets.topic.replication.factor: 3
              transaction.state.log.replication.factor: 3
              transaction.state.log.min.isr: 2
          controller:
            replicas: 3
            resources:
              requests:
                memory: 1536Mi
                cpu: "1"
              limits:
                memory: 1536Mi
            jvmOptions:
              -Xms: 1024m
              -Xmx: 1024m

Using Red Hat Streams for Apache Kafka on OpenShift​

Red Hat Streams for Apache Kafka supports less Kafka versions than the corresponding Strimzi Kafka operator. Due to this, the default version configured in Shift may not work. Change the Kafka version to a supported version as explained in the comments of the strimzi.kafkaVersion value. For example, in case of Shift 0.194.0, use Kafka version 3.7.1 for the Strimzi Kafka operator and Kafka version 3.7.0 for Red Hat Streams for Apache Kafka.

External Kafka clusters​

Shift supports external Kafka clusters.

The required topics must be manually created in the external Kafka cluster. See file topics.yaml for a list of required topics and their configuration (partitions, retention times). For Shift-lite, the topics from sections common:, asts:, scp:, and idp: must be created. When using Smartscreen services, the topics from section smartscreen: must be created as well. When using payment services, the topics from section payment: must be created as well.

To configure Shift to use an external Kafka cluster, the following parameters must be set:

  • Disable the creation of the custom resources for Strimzi Kafka operator

    strimzi:
      enabled: false

  • Enable usage of an external Kafka cluster and provide hostname and port

    common:
      datastores:
        kafka:
          external:
            enabled: true
            broker:
              host: kafka-broker
              port: 9092

Authentication​

Shift supports only one user for all connections to Kafka. Only the SASL mechanism SCRAM-SHA-512 is supported. Use the following parameters to enable authentication and configure the username.

common:
  datastores:
    kafka:
      auth:
        enabled: true
        username: shift-kafka-username

The password must be provided in an existing Kubernetes secret. The name of the secret must match the username. The password must be contained in the key password.

For username shift-kafka-username and password shift-kafka-password, the required Kubernetes secret can be generated using the following command:

kubectl create secret generic shift-kafka-username \
  --from-literal=password=shift-kafka-password

This is how the resulting secret should look like:

apiVersion: v1
kind: Secret
metadata:
  name: shift-kafka-username
type: Opaque
data:
  password: c2hpZnQta2Fma2EtcGFzc3dvcmQ=

TLS​

Shift supports TLS for Kafka connections. When TLS is used, authentication must also be enabled. The TLS trust store must be provided in an existing Kubernetes secret. Use the following parameters to enable TLS and configure the name of the existing Kubernetes secret containing the trust store.

common:
  datastores:
    kafka:
      external:
        tls:
          enabled: true
          trustStoreSecret: shift-kafka-tls-truststore

The existing Kubernetes secret must contain the trust store in two formats. A file containing all required certificates in PEM format must be provided in the key ca.crt encoded as a base64 string. A file containing all required certificates in PKCS#12 format must be provided in the key ca.p12 encoded as a base64 string. The import password for the PKCS#12 file must be provided in the key ca.password encoded as a base64 string.

Given files

  • /path/to/ca.crt containing all required certificates in PEM format
  • /path/to/ca.p12 containing all required certificates in PKCS#12

as well as import password ca-import-password, the required Kubernetes secret can be generated using the following command:

kubectl create secret generic shift-kafka-tls-truststore \
  --from-file=ca.crt=/path/to/ca.crt \
  --from-file=ca.p12=/path/to/ca.p12 \
  --from-literal=ca.password=ca-import-password

This is how the resulting secret should look like:

apiVersion: v1
kind: Secret
metadata:
  name: shift-kafka-tls-truststore
type: Opaque
data:
  ca.crt: LS0tLS1...LS0tLS0K
  ca.p12: MIIGogI...xAgInEA==
  ca.password: Y2EtaW1wb3J0LXBhc3N3b3Jk

Topics prefix​

Shift supports an optional prefix to add to all Kafka topics. This must be used when running multiple Shift deployments against the same external Kafka cluster to ensure each deployment uses unique topics. The prefix must contain only lowercase alphanumeric characters and dashes ('-'). The prefix must start and end with an alphanumeric character and consist of no more than 16 characters. Dot character ('.') will be inserted automatically as delimiter between prefix and internal topic name. For example, when using prefix prod, topic com.kobil.audit becomes prod.com.kobil.audit.

Use the following parameter to configure a topics prefix.

common:
  datastores:
    kafka:
      external:
        topics:
          prefix: ""

Full example​

Below is a full example to configure Shift against an external Kafka cluster using TLS, authentication, and a topics prefix 'test'.


# Disable custom resources for Strimzi Kafka operator
strimzi:
  enabled: false

# Configure Shift against external Kafka using authentication, TLS, and topics prefix.
common:
  datastores:
    kafka:

      auth:
        enabled: true
        username: shift-kafka-username

      external:
        enabled: true
        broker:
          host: kafka-broker
          port: 9092

        topics:
          prefix: "test"

        tls:
          enabled: true
          trustStoreSecret: shift-kafka-tls-truststore

Traffic routing​

This section covers the configuration and requirements of Shift regarding traffic routing, DNS configuration, and TLS certificates. The configuration described here is available since Shift version 0.232.0.

Note: These configurations are currently not fully supported by Smartdashboard and Payment components. When using these components, the previous traffic routing configuration should be used.

Shift uses Istio or alternatively Red Hat OpenShift Service Mesh for routing external traffic to the endpoints of the various Shift components. These endpoints are grouped in three API groups:

  • API group public contains endpoints consumed by end users (mobile apps, browsers). These must be reachable via public Internet.
  • API group integration contains endpoints for backend integration. These may be required to be reachable via public Internet, depending on where the backends that integrate with Shift are located.
  • API group admin contains endpoints for administrative access. These must not be reachable via public Internet.

Shift can create up to three Istio Gateway resources for routing the endpoints in the API groups:

  • Gateway public routes endpoints of the API group public.
  • Gateway integration routes endpoints of the API groups public and integration.
  • Gateway admin routes endpoints of the API groups public, integration, and admin.
ConsumerGatewayAPI Groups
End userspublicpublic
Backendsintegrationpublic, integration
Adminsadminpublic, integration, admin

These Gateway resources are used to configure Istio Ingress Gateways. Shift requires that the Istio Ingress Gateways already exist in the Kubernetes cluster. Part of the Gateway configuration is a TLS secret name. Shift requires that the corresponding Kubernetes TLS secrets already exists in the namespace where the Istio Ingress Gateway workload is running.

Gateway configuration options​

The gateway configuration provided by Shift allows various routing topologies to be realized. The following options exist for each Gateway:

  • enabled: Enable creation of the Istio Gateway resource.
  • ingressGatewaySelector: Object containing the Kubernetes labels of the Istio Ingress Gateway workload to be used as selector in the Gateway resource.
  • domain: The internal domain name under which the Istio Ingress Gateway is exposed to the outside of the Kubernetes cluster. The hosts added to the Istio Gateway are this domain and the subdomains of the Shift components.
  • tlsSecretName: The name of the Kubernetes TLS secret which the Gateway should use. The certificate must contain the Gateway's domain as well as all required subdomains. Alternatively a wildcard certificate can be used. The TLS secret must be created in the namespace where the Istio Ingress Gateway workload is running.
  • gatewayHttpsRedirect: If set to true, the Gateway is configured to send a redirect for all http requests asking clients to use https. If set to false, the Gateway accepts plain http traffic.
  • gatewayAddAllHosts: If set to true, all hosts required by Shift are explicitly added to the Gateway. This is required when multiple Gateway resources configure the same Istio Ingress Gateway. This requires that load balancers in front of the Istio Ingress Gateway correctly set or forward the SNI. Otherwise the Istio Ingress Gateway cannot select the correct hosts to route incoming traffic. See here for further information. If set to false, a wildcard (*) is configured as host in the Gateway. This should be used if load balancers in front of the Istio Ingress Gateway do not forward the SNI. This requires a dedicated Istio Ingress Gateway for each of the Gateways.
  • additionalSubdomains: List of additional subdomain prefixes. The corresponding subdomains are added to the Gateway hosts. Only relevant when gatewayAddAllHosts: true is set.

Note: It is possible to use the same Istio Ingress Gateway for multiple Gateways by setting the same value for ingressGatewaySelector. In this case, gatewayAddAllHosts must be set to true.

Note: It is possible to use the same domain for multiple Gateways. In this case, different Istio Ingress Gateways must be used for each Gateway to avoid conflicts.

Note: All domains configured above must be unique per Shift instance. If several Shift instances are used, e.g. for staging purposes, distinct domains must be configured.

Note: The routing configuration described here is not enabled by default. Instead, the legacy approach for configuring traffic routing is enabled by default. This must be explicitly disabled when using the approach described here, by setting the following in shift-values.yaml:

global:
  routing:
    istio:
      gateways:
        public: false
        external: false
        admin: false

External domains​

Certain Shift components must know the external domain under which they are reachable by consumers. This is mainly required by OpenID Connect (OIDC) authentication flows. The external domain is not necessarily the domain specified in the Gateway configuration, as there might be a Web Application Firewall (WAF) which rewrites the external domain to the internal domain. Shift provides two parameter for configuring the external domains:

  • global.routing.domain configures the public external domain used by end users.
  • global.routing.adminDomain configures the admin external domain used by admins. This only needs to be configured when using a distinct external domain for admins.

The integration external domain used by backends does not need to be configured.

Note: OIDC is used for authenticating to the admin interfaces of IDP-Core and Smartdashboard. HTTP requests are redirected to the public external domain (global.routing.domain) during authentication. It is required that admins are able to resolve and access this public external domain, even tough they use the admin external domain to interact with Shift.

Note: While doing OIDC discovery via the well-known endpoint of IDP-Core, using the integration external domain, backends will receive URLs containing the public external domain (global.routing.domain) in the responses. It is therefore required that backends are able to resolve and access this public external domain. Alternatively, the OIDC configuration in the backends can be done manually using the integration external domain.

Note: All external domains must be unique per Shift instance. If several Shift instances are used, e.g. for staging purposes, distinct external domains must be used.

Subdomains, TLS certificates, and DNS entries​

The Fully Qualified Domain Names (FQDN) used by consumers to access Shift endpoints are subdomains of the domains configured above. This is true for both the internal and external domains.

Shift uses several subdomains, which are not configurable and depend on the installed components.

For example, for the domain example.com, a Shift lite installation uses subdomains:

  • asts.example.com
  • idp.example.com
  • scp.example.com

A complete Shift deployment uses additional subdomains

  • pay.example.com
  • smartdashboard.example.com
  • smartscreen.example.com
  • profile.example.com
  • audience.example.com
  • mercury.example.com

The TLS certificates used by the Istio Gateways must include all required subdomains. For sake of simplicity, wildcard certificates can be used, i.e. *.example.com.

The DNS entries pointing to the Istio Ingress Gateway must also include all required subdomains. For sake of simplicity, wildcard entries can be used, i.e. *.example.com.

Example configuration​

Minimalistic example using only the Gateway admin​

This example uses the Gateway admin and a single Istio Ingress Gateway with the label istio: "istio-ingressgateway".

This setup is suitable for testing or when an additional firewall ensures that only the public API group endpoints are reachable from the Internet.

All consumers (end users, backends, admins) use the external domain shift.external.com to access Shift APIs. This domain points to the Istio Ingress Gateway and is used by the Gateway admin for routing the API groups public, integration, and admin. The TLS certificate for the subdomains shift.external.com and *.shift.external.com is stored in the Kubernetes TLS secret tls-secret.

ConsumerExternal domainInternal domainGatewayAPI Groups
End usersshift.external.comshift.external.comadminpublic, integration, admin
Backendsshift.external.comshift.external.comadminpublic, integration, admin
Adminsshift.external.comshift.external.comadminpublic, integration, admin

The configuration for this example in shift-values.yaml is as follows.

global:
  routing:
    domain: "shift.external.com"

    istio:
      enabled: true

      apiGroups:
        enabled: true

        gateways:
          admin:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway"
            domain: "shift.external.com"
            tlsSecretName: tls-secret

      gateways:
        public: false
        external: false
        admin: false

Complex example using all Gateways and distinct external domains​

This example uses all three Gateways (public, integration, admin) and a single Istio Ingress Gateway with label istio: "istio-ingressgateway".

End users use the external domain shift.public.external.com to access Shift APIs. A Web Application Firewall (WAF) rewrites this external domain to the internal domain public.internal. This internal domain points to the Istio Ingress Gateway and is used by the Gateway public for routing the API group public. The TLS certificate for subdomains public.internal and *.public.internal is stored in the Kubernetes TLS secret tls-secret-public.

Backends use the external domain shift.integration.external.com to access Shift APIs. A Web Application Firewall (WAF) rewrites this external domain to the internal domain integration.internal. This internal domain points to the Istio Ingress Gateway and is used by the Gateway integration for routing the API groups public and integration. The TLS certificate for the subdomains integration.internal and *.integration.internal is stored in the Kubernetes TLS secret tls-secret-integration.

Admins use the external domain shift.admin.external.com to access Shift APIs. A Web Application Firewall (WAF) rewrites this external domain to the internal domain admin.internal. This internal domain points to the Istio Ingress Gateway and is used by the Gateway admin for routing the API groups public, integration, and admin. The TLS certificate for the subdomains admin.internal and *.admin.internal is stored in the Kubernetes TLS secret tls-secret-admin.

ConsumerExternal domainInternal domainGatewayAPI Groups
End usersshift.public.external.compublic.internalpublicpublic
Backendsshift.integration.external.comintegration.internalintegrationpublic, integration
Adminsshift.admin.external.comadmin.internaladminpublic, integration, admin

The configuration for this example in shift-values.yaml is as follows.

global:
  routing:
    domain: "shift.public.external.com"
    adminDomain: "shift.admin.external.com"

    istio:
      enabled: true

      apiGroups:
        enabled: true

        gateways:
          public:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway"
            domain: "public.internal"
            tlsSecretName: tls-secret-public

          integration:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway"
            domain: "integration.internal"
            tlsSecretName: tls-secret-integration

          admin:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway"
            domain: "admin.internal"
            tlsSecretName: tls-secret-admin

      gateways:
        public: false
        external: false
        admin: false

Migration​

This section explains how to migrate from the previous traffic routing configuration to the new traffic routing configuration introduced above.

Previous configuration in shift-values.yaml

global:
  routing:
    domain: shift.example.com

    tlsSecret: tls-secret

    istio:
      enabled: true

      gateways:
        public: true
        external: true
        admin: true

      options:
        gatewayNamePrefix: istio-ingressgateway
        gatewayAddAllHosts: false
        gatewayHttpsRedirect: true

  certs:
    managed: true
    issuerName: cert-issuer

New configuration in shift-values.yaml

global:
  routing:
    domain: shift.example.com

    istio:
      enabled: true

      apiGroups:
        enabled: true

        gateways:
          public:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway-public"
            domain: "shift.example.com"
            tlsSecretName: tls-secret
            gatewayHttpsRedirect: true
            gatewayAddAllHosts: false

          integration:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway-external"
            domain: "shift.example.com"
            tlsSecretName: tls-secret
            gatewayHttpsRedirect: true
            gatewayAddAllHosts: false

          admin:
            enabled: true
            ingressGatewaySelector:
              istio: "istio-ingressgateway-admin"
            domain: "shift.example.com"
            tlsSecretName: tls-secret
            gatewayHttpsRedirect: true
            gatewayAddAllHosts: false

      gateways:
        public: false
        external: false
        admin: false

  certs:
    managed: false

The parameter gatewayNamePrefix: istio-ingressgateway is converted to the parameters ingressGatewaySelector. Here, not only the prefix, but the full label needs to be specified.

The parameters tlsSecret, gatewayAddAllHosts, and gatewayHttpsRedirect now need to be specified for each Gateway separately.

While migrating it is recommended to disable the cert-manager Certificate resource created by Shift as it is deprecated and will be removed in a future release. The Certificate can be created manually by applying the following manifest in the namespace where the Istio Ingress Gateway workload is running.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: shift-tls
spec:
  secretName: tls-secret
  duration: 2160h0m0s
  renewBefore: 360h0m0s
  privateKey:
    size: 2048
    algorithm: RSA
    encoding: PKCS1
  usages:
    - server auth
  dnsNames:
    - "shift.example.com"
    - "*.shift.example.com"
  issuerRef:
    name: cert-issuer
    kind: ClusterIssuer

Note, when migrating to API group routing, the feature to create optional Ingress resources for the Istio Ingress Gateway is no longer supported. Ingresses configured via the following parameters must now be created manually.

routing:
  istio:
    ingress:
      admin:
        enabled: false
        class: ~
        annotations: {}

      external:
        enabled: false
        class: ~
        annotations: {}

      public:
        enabled: false
        class: ~
        annotations: {}

The following yaml snippet is an example of manually creating the Ingress resource for the Admin Istio Ingress Gateway. It must be applied in the namespace where the Istio Ingress Gateway workload is running.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shift-ingress-admin
  annotations:
    <annotations from `routing.istio.ingress.admin.annotations:`>
spec:
  ingressClassName: <ingress class from `routing.istio.ingress.admin.class:`>
  tls:
  - hosts:
    - "*.shift.example.com"
    secretName: tls-secret
  rules:
    - host: "*.shift.example.com"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: <svc name of Istio Ingress Gateway>
                port:
                  number: 443

Note, when using Mutual TLS, the Kubernetes secret containing the trusted CA certificates must be created manually and placed in the namespace where the Istio Ingress Gateway workload is running. Using the parameter common.mutualTLS.istioIngressGateway.cacerts: to do so is no longer supported. For each enabled Gateway one secret must be created. The name of the CA certificate secrets must be the name of the Gateway TLS secrets, with -cacert appended.

Istio Sidecar Proxy Injection​

Shift supports injecting Istio sidecar proxies into it's workloads to add Shift components to the Istio service mesh. This is enabled by setting

global:
  routing:
    istio:
      options:
        inject: true

in shift-values.yaml. Shift controls sidecar proxy injection by adding the label sidecar.istio.io/inject: "true" to all workloads. Enabling sidecar injection on the namespace level is not required.

Sidecar proxy configuration using annotations​

By default, shift adds the proxy config annotation

annotations:
  proxy.istio.io/config: |
    holdApplicationUntilProxyStarts: true

to all workloads in order to delay application startup until the Istio proxy is ready. This avoids startup race conditions. Additional Istio related annotations can be configured using the value

global:
  routing:
    istio:
      resourceAnnotations: |
        proxy.istio.io/config: |
          holdApplicationUntilProxyStarts: true

The content must be a yaml formatted text block. To disable annotations, e.g. in case they are set via global mesh options, set this value to the empty string (""). See Resource Annotations and Proxy Config for supported options.

Istio sidecar proxy injection on OpenShift​

The following instructions are not required when native sidecar containers are enabled in OpenShift and OpenShift Service Mesh. See Section Using Native Sidecar Containers

OpenShift Service Mesh does not allow Init Containers to establish network connections to outside of the service mesh, see Enabling sidecar injection.

Shift components use Init Containers for performing database migrations which will fail if the database is outside of the service mesh.

The workaround suggested by RedHat is to exclude the database port from being redirected through the sidecar proxy. This can be achieved by setting

global:
  routing:
    istio:
      resourceAnnotations: |
        proxy.istio.io/config: |
          holdApplicationUntilProxyStarts: true
        traffic.sidecar.istio.io/excludeOutboundPorts: "1521"

in shift-values.yaml, where 1521 is the database port.

In addition, the various values for database.host: in shift-values.yaml must be set to the IP address, and not the hostname, of the database.

PeerAuthentication resources​

Shift creates a PeerAuthentication resource with

spec:
  mtls:
    mode: STRICT

to ensure that mutual TLS is required for all inbound traffic to the sidecar proxies and therefore the Shift services.

For some ports mTLS mode STRICT cannot be used, e.g. additional ports serving prometheus metrics. Shift creates additional PeerAuthentication resources to configure mTLS mode PERMISSIVE for such ports and the workload in question.

The creation of PeerAuthentication resources can be disabled by setting

global:
  routing:
    istio:
      options:
        createPeerAuthentication: false

in shift-values.yaml. When no PeerAuthentication resources exist, the default mode PERMISSIVE is used. This allows Shift services to accept both plaintext and mutual TLS traffic. See PeerAuthentication and Mutual TLS Migration for details.

Sidecar resource​

To reduce memory usage of Istio sidecar proxies, when the mesh is large, Shift creates a Sidecar resource with

spec:
  egress:
  - hosts:
    - "./*"

This configures the sidecar proxies to allow egress traffic only to other workloads in the same namespace. This affects only egress traffic to services which are part of the service mesh. Egress traffic to services outside of the service mesh is not restricted. See Sidecar for details.

The creation of the Sidecar resource can be disabled by setting

global:
  routing:
    istio:
      options:
        createSidecar: false

in shift-values.yaml.

Using Native Sidecar Containers​

Several workarounds for Istio sidecar proxy injection become obsolete once native sidecar containers are enabled in Kubernetes and Istio. Support for native sidecar containers is stable in both Kubernetes and Istio. See KEP-753: Sidecar containers for additional details on this feature.

The following workarounds are no longer required:

  • The holdApplicationUntilProxyStarts: true annotation, which makes the Istio proxy start before the application to prevent race conditions, is no longer needed.

    This annotation can be removed by adding the following to custom shift-values.yaml:

    global:
      routing:
        istio:
          resourceAnnotations: ""

  • The Envoy-Sidecar-Helper container used in ephemeral pods (jobs and tests) to stop the Istio proxy after the main container finishes is no longer needed.

    The Envoy-Sidecar-Helper container can be disabled by adding the following to custom shift-values.yaml:

    global:
      routing:
        istio:
          options:
            createSidecarHelper: false

    Note: The Envoy-Sidecar-Helper container is deprecated and will be removed in a future Shift version.

  • The workarounds described in Section Istio sidecar proxy injection on OpenShift are no longer required.

How to enable native sidecar support in Istio and Kubernetes​

Support for native sidecar containers in Kubernetes has been enabled by default since version 1.29 and cannot be disabled as of version 1.33.

Support for native sidecars in Istio can be enabled in the global mesh configuration since version 1.19 using the following settings.

pilot:
  env:
    ENABLE_NATIVE_SIDECARS: "true"

Since version 1.24, Istio supports the sidecar.istio.io/nativeSidecar annotation to enable native sidecar support on a per-pod basis. In Shift, this can be applied by adding the following to the custom shift-values.yaml file.

global:
  routing:
    istio:
      resourceAnnotations: |
        sidecar.istio.io/nativeSidecar: "true"

Since Istio version 1.27, native sidecar support has been enabled by default when running on Kubernetes 1.33.

OpenShift Service Mesh has supported native sidecars since version 2.6 by adding the following configuration to the ServiceMeshControlPlane resource.

spec:
  runtime:
    components:
      pilot:
        container:
          env:
            ENABLE_NATIVE_SIDECARS: "true"

Mercury Chat Platform​

This section contains additional details on the configuration of the Mercury chat platform and its dependencies.

Mercury endpoints are exposed using subdomain mercury.

S3​

Mercury uses a S3 compatible object storage for media files and optionally for archiving messages. The media bucket must be publicly accessible to clients and provide support for presigned URLs. Clients access the buckets only using presigned URLs which are generated by Mercury. Therefore, the credentials (access key and secret access key) for accessing the buckets should only be provided to Mercury. Both AWS S3 and Minio provide the required functionality.

Centrifugo​

Mercury uses Centrifugo real-time messaging server for message delivery to clients. The Centrifugo unidirectional Server-Sent Events (SSE) endpoints with prefix /connection/uni_sse must be publicly accessible to clients. Centrifugo must be correctly configured to properly function with Mercury. This includes setup of the required channels and integration with IDP.

The following config.json shows the required settings. The hostname in issuer_regex must be set to the publicly reachable hostname of idp-core. The hostname in jwks_public_endpoint must be set to an idp-core hostname that is reachable by Centrifugo, e.g. the internal Kubernetes service name of idp-core if Centrifugo is deployed in the same namespace as Shift.

{
    "http_api": {
        "key": "api_key"
    },
    "http_server": {
        "internal_port": "9000",
    },
    "channel": {
        "namespaces": [
            {
                "name": "personal",
                "presence": true
            },
            {
                "name": "global",
                "presence": false,
                "allow_subscribe_for_client": true
            }
        ],
        "without_namespace": {
            "presence": true
        }
    },
    "client": {
        "subscribe_to_user_personal_channel": {
            "enabled": true,
            "personal_channel_namespace": "personal"
        },
        "token": {
            "issuer_regex": "https://idp.example.com/auth/realms/(?P<realm>[^/]+)",
            "jwks_public_endpoint": "http://idp-core:80/auth/realms/{{realm}}/protocol/openid-connect/certs"
        }
    },
    "health": {
        "enabled": true
    },
    "uni_sse": {
        "enabled": true
    }
}

Credentials from existing Kubernetes secrets​

Mercury supports reading the following credentials from an existing Kubernetes secret.

  • Postgres credentials used by addressbook, broadcast, and conversation.
  • ScyllaDB credentials used by backend and archive.
  • Postgres credentials used by smartscreen.
  • Centrifugo API key.
  • S3 credentials.

If this feature is used in Shift, all these credentials must be added to the secret configured in the value common.existingSecretDatastoreCredentials. See Section Credentials and other sensitive data from existing Kubernetes secrets for details on the required structure of the secret.

Chat messages from MiniApps​

Mercury expects MiniApps to address recipients of messages using their user ID (claim sub from the user's access tokens). This is different from the previous chat system which requires the username (claim preferred_username in the user's access token).

Mercury supports this legacy feature. It requires idp-core admin credentials to be provided to Mercury miniapp connector. Use the following valuesOverride to enable it:

mercury:
  miniappConnector:
    valuesOverride:
      api:
        security:
          checkByUsername: true
          adminLogin: "admin"
          adminPassword: "password"

Migration from SCP Notifier to AST Push Notification​

The task of sending push notifications to mobile devices is currently handled by component SCP Notifier. This component is deprecated and users are advised to migrate to the new component AST Push Notification. From deployment perspective, migration consists of four steps. The migration is designed such that there is no downtime.

  1. Deploy AST Push Notification

    Component AST Push Notification is disabled by default. It requires a database. To enable it, add the following to shift-values.yaml:

    astPushNotification:
      enabled: true

      database:
        host: postgres
        port: 5432
        name: "ast_push_notification"
        auth:
          username: user
          password: "password"

    When using the Shift feature Credentials and other sensitive data from existing Kubernetes secrets, add keys AST_PUSH_NOTIFICATION_DB_USERNAME and AST_PUSH_NOTIFICATION_DB_PASSWORD to the Kubernetes secret defined in the parameter common.existingSecretDatastoreCredentials.

    After enabling AST Push Notification, the component is running but not actively used for sending push notifications to mobile devices. When mobile devices register new push tokens, they are sent to SCP Notifier and mirrored to AST Push Notification.

  2. Migrate existing push tokens

    A database migration must be executed manually to migrate existing push tokens from the database of SCP Notifier to the database of AST Push Notification. Please contact KOBIL service team to obtain the database migration scripts. The database migration can be performed online, i.e. while both SCP Notifier and AST Push Notification are running and actively using the database. The database migration is idempotent, i.e. it can be executed multiple times.

    After migration, the push tokens in both databases are in sync. They also remain in sync, because newly registered push tokens are added to both databases.

  3. Enable the Integration of Shift components with AST Push Notification

    To enable the actual usage of AST Push Notification in Shift, add the following to shift-values.yaml:

    astPushNotification:
      enableIntegration: true

    After enabling the integration, AST Push Notification is actively used to send push notifications to mobile devices and SCP Notifier is no longer used.

  4. Disable SCP Notifier.

    Since SCP Notifier is no longer needed, it can be disabled.

    1. If SCP Notifier is the only SCP component currently deployed, disable it by adding the following to shift-values.yaml:

      scp:
        enabled: false

    2. If other SCP components are used, disable it by adding the following to shift-values.yaml:

      scpNotifier:
        enabled: false

    When uninstalling SCP Notifier, Shift adds the required routing configuration (Istio VirtualService) to ensure that the endpoint used by mobile devices to register push tokens remains reachable. The endpoint in question is https://scp.example.com/notifier/push/token, which means the scp subdomain must remain in TLS certificates and WAF rules even if SCP Notifier is disabled.

The previously described changes to the deployment configuration will become the default in a future Shift release. This will be marked as a breaking change. Updating to this future Shift release without downtime will only be possible if these migration steps were executed.

Upgrading​

Upgrading from versions before 0.267.0​

This Shift version updates the version of the included Kafka cluster from 3.9.0/3.9.1 to 4.0.0. Required Strimzi Kafka Operator version changes from 0.45.1 to 0.47.0. Required Red Hat Streams for Apache Kafka on OpenShift version changes from 2.9.x to 3.0.x.

Kafka 4.0.0 removes ZooKeeper support. KRaft is now the only supported architecture. Before updating to this Shift version, the migration from ZooKeeper to KRaft must be completed.

When using Strimzi Kafka Operator on Kubernetes, ensure that version 0.47.0 is installed before applying the update. This is the only version that supports both Kafka 3.9.0/3.9.1 and 4.0.0 - see supported versions of Strimzi Kafka Operator.

When using Red Hat Streams for Apache Kafka on OpenShift, ensure that version 3.0.x is installed before applying the update. This is the only version that supports both Kafka 3.9.0 and 4.0.0, see Kafka 4.0 support.

Also ensure that a Shift version using Kafka 3.9.0/3.9.1 is running before applying the upgrade, i.e. Shift version 0.236.0 or newer.

The removal of ZooKeeper support introduces breaking changes to the Helm configuration (values.yaml). Update your custom shift-values.yaml accordingly before applying the update:

Old parameterAction
strimzi.kraftRemoved. KRaft is always enabled and this parameter must be removed.
strimzi.storage.size.zookeeperRenamed to strimzi.storage.size.controller.
strimzi.storage.class.zookeeperRenamed to strimzi.storage.class.controller.
strimzi.sizing.custom.zookeeperRenamed to strimzi.sizing.custom.controller.

Upgrading from versions before 0.263.0​

Important Deployment Notice​

The IDP Core update included in this release performs database migrations that require scheduled downtime. Plan the deployment carefully to avoid service interruptions.

Update procedure:

  • Manually scale down the affected components to 0 replicas.
  • Perform the update.
  • After the update completes, the components will scale up automatically.

Upgrading from versions before 0.262.0​

Breaking change in the configuration of Smartdashboard Workspace Management: the previously used parameter smartdashboardWorkspaceManagement.config.loginAccountAdminTheme: 'kobil-admin' has been removed and replaced by the following parameters:

smartdashboardWorkspaceManagement:
  config:
    accountTheme: 'keycloak.v3'
    adminTheme: 'kobil-admin'

If your deployment uses non-default values, update them in your custom shift-values.yaml before upgrading to this version.

Upgrading from versions before 0.258.0​

Important Deployment Notice​

The IDP Core update included in this release performs database migrations that require scheduled downtime. Plan the deployment carefully to avoid service interruptions.

Update procedure:

  • Manually scale down the affected components to 0 replicas.
  • Perform the update.
  • After the update completes, the components will scale up automatically.

Upgrading from versions before 0.257.0​

Important Deployment Notice​

This release introduces database schema changes that may result in a brief period of inconsistency during deployment. Downtime is not required, but ongoing app version registrations may be temporarily impacted while the update is in progress.

To reduce the risk of user-visible effects, schedule the deployment during a period of minimal system load.

Upgrading from versions before 0.256.0​

Important Deployment Notice​

This release includes components that perform database migrations requiring scheduled downtime when using Oracle database. Zero-downtime updates are supported for PostgreSQL, but not for Oracle. Plan your deployment carefully to avoid service interruptions.

Affected components:

  • AST Client Management

Update procedure (Oracle only):

  • Manually scale down the affected components to 0 replicas.
  • Perform the update.
  • After the update completes, the components will scale up automatically.

Upgrading from versions before 0.251.0​

Important Deployment Notice​

This release includes components that perform database migrations requiring scheduled downtime when using Oracle database. Zero-downtime updates are supported for PostgreSQL, but not for Oracle. Plan your deployment carefully to avoid service interruptions.

Affected components:

  • AST Certificate Authority
  • AST Login
  • AST Trusted Message Sign

Update procedure (Oracle only):

  • Manually scale down the affected components to 0 replicas.
  • Perform the update.
  • After the update completes, the components will scale up automatically.

Upgrading from versions before 0.236.0​

This Shift version updates the version of the included Kafka cluster from 3.8.0 to 3.9.0. Required Strimzi Kafka Operator versions change from 0.43.0 - 0.45.0 to 0.45.1. Required Red Hat Streams for Apache Kafka on OpenShift version changes from 2.8.x to 2.9.x.

When using Strimzi Kafka Operator on Kubernetes, ensure that version 0.45.1 is installed before applying the update. This is the only version that supports both Kafka 3.8.0 and 3.9.0/3.9.1 (see supported versions of Strimzi Kafka Operator) as well as the migration from ZooKeeper to KRaft.

When using Red Hat Streams for Apache Kafka on OpenShift, ensure that version 2.9.x is installed before applying the update. This is the only version that supports both Kafka 3.8.0 and 3.9.0, see Kafka 3.9.x support.

Also ensure that a shift version using Kafka 3.8.0 is running before applying the upgrade, i.e. Shift version 0.222.0 or newer.

Upgrading from versions before 0.234.0​

This Shift release introduces a migration of roles in IDP Core performed by Smartdashboard. The OIDC client configured for Smartdashboard (parameter smartdashboardKongConfigurationBackend.config: in Helm values) must be granted the required permissions before applying this release.

Required configuration in the IDP Core Admin UI:

  • Enable Service account roles for the OIDC client under Settings -> Capability config.
  • Assign the admin role to the OIDC client under Service account roles.

The admin role is only required during the migration and can be removed after the migration succeeds.

Upgrading from versions before 0.228.0​

This Shift version uses IDP Core 5 per default. Customers using an IDP Core 4 based custom IDP must set the following parameter in shift-values.yaml:

idp:
  idpCoreBaseVersion: "4"

Upgrading from versions before 0.227.0​

This version contains a breaking change in Payment Services configuration. Parameters for configuring the hosts of payment providers iyzico and Sipay are no longer part of the values provided during installation/update in the values.yaml file. If you previously configured these values in your values.yaml they will be ignored. This config must now be done in Payment GUI after upgrading to this version.

Upgrading from versions before 0.222.0​

This Shift version updates the version of the included Kafka cluster from 3.7.0/3.7.1 to 3.8.0. Required Strimzi Kafka Operator versions change from 0.42.0 - 0.44.0 to 0.43.0 - 0.45.0. Required Red Hat Streams for Apache Kafka on OpenShift version changes from 2.7.0 to 2.8.0.

When using Strimzi Kafka Operator on Kubernetes, ensure that version 0.43.0 or 0.44.0 is installed before applying the update. These are the only versions that support both Kafka 3.7.1 and 3.8.0, see supported versions of Strimzi Kafka Operator.

When using Red Hat Streams for Apache Kafka on OpenShift, ensure that version 2.8.0 is installed before applying the update. This is the only version that supports both Kafka 3.7.0 and 3.8.0, see Kafka 3.8.0 support.

Also ensure that a shift version using Kafka 3.7.0/3.7.1 is running before applying the upgrade, i.e. shift version 0.194.0 or newer.

Upgrading from versions before 0.220.0​

This Shift version contains breaking changes in Mercury Pay Connector:

  • Mercury Pay Connector now requires a database. The database must be created and configured using parameter mercury.payConnector.postgres: before updating to this version.

    If Shift is configured to read database credentials from an existing Kubernetes secret, the database credentials used by Mercury Pay Connector must be added to this secret in keys MERCURY_PAYCONNECTOR_POSTGRES_DB_USERNAME and MERCURY_PAYCONNECTOR_POSTGRES_DB_PASSWORD before updating to this version.

  • Mercury Pay Connector now depends on Mercury Conversation. Enable Mercury Conversation (mercury.conversation.enabled: true) and configure a database (mercury.conversation.postgres:) before updating to this version.

Upgrading from versions before 0.194.0​

This Shift version updates the version of the included Kafka cluster from 3.6.1 to 3.7.1. Required Strimzi Kafka Operator versions change from 0.39.0 - 0.42.0 to 0.42.0 - 0.44.0.

Before applying the update, ensure that Strimzi Kafka Operator version 0.42.0 is installed. This is the only version that supports both Kafka 3.6.1 and 3.7.1, see supported versions of Strimzi Kafka Operator.

Also ensure that a shift version using Kafka 3.6.1 is running before applying the upgrade, i.e. shift version 0.179.0 or newer.

Upgrading from versions before 0.189.0​

This version adds support for reading Redis credentials from an existing Kubernetes secret to pay services.

If Shift is configured to read database credentials from an existing Kubernetes secret, the Redis credentials used by pay services must be added to this secret in key PAY_SERVICES_REDIS_PASSWORD before updating to this version.

Upgrading from versions before 0.186.0​

The service ast-webhooks included in Shift version 0.186.0 requires a database. Before updating to Shift version 0.186.0, create the database and configure it in shift-values.yaml:

astWebhooks:
  database:
    host: postgres
    port: 5432
    name: "ast_webhooks"
    auth:
      username: user
      password: "password"

If Shift is configured to read database credentials from an existing Kubernetes secret, the ast-webhooks database credentials must be added to this secret in keys AST_WEBHOOKS_DB_USERNAME and AST_WEBHOOKS_DB_PASSWORD.

Upgrading from versions before 0.179.0​

This Shift version updates the version of the included Kafka cluster from 3.5.1 to 3.6.1. Required Strimzi Kafka Operator versions change from 0.36.1 - 0.39.0 to 0.39.0 - 0.42.0.

Before applying the update, ensure that Strimzi Kafka Operator version 0.39.0 is installed. This is the only version that supports both Kafka 3.5.1 and 3.6.1, see supported versions of Strimzi Kafka Operator.

Also ensure that a shift version using Kafka 3.5.1 is running before applying the upgrade, i.e. shift version 0.171.0 or newer.

Upgrading from versions before 0.171.0​

This shift version updates the version of the included Kafka cluster from 3.4.0 to 3.5.1. Supported Strimzi Kafka Operator versions change from 0.33.2 - 0.37.0 to 0.36.1 - 0.39.0.

Before applying the update, ensure that Strimzi Kafka Operator version 0.36.1 or 0.37.0 is installed. These are the only versions that support both Kafka 3.4.0 and 3.5.1, see supported versions of Strimzi Kafka Operator.

Also ensure that a shift version using Kafka 3.4.0 is running before applying the upgrade, i.e. shift version 0.153.0 or newer.

Upgrading from versions before 0.168.0​

Shift version 0.168.0 no longer requires the Kafka topics com.kobil.ast.stream.sse and com.kobil.ast.stream-statestoreastmessages-changelog. Since the included Kafka cluster by default is set to prevent topic deletion, the Kafka topics are not actually deleted. This has no negative impact. Optionally, the following steps can be performed after updating to shift version 0.168.0 to permanently delete the Kafka topics:

  • Enable topic deletion in Kafka by adding the following to shift-values.yaml and upgrade the shift helm release.

    strimzi:
      valuesOverride:
        kafka:
          config:
            delete.topic.enable: "true"

  • Delete the Kafkatopic Kubernetes resources com.kobil.ast.stream.sse and com.kobil.ast.stream-statestoreastmessages-changelog.

  • Remove the valuesOverride block and upgrade the shift helm release to disable topic deletion.

Upgrading from versions before 0.153.0​

This shift version updates the version of the included Kafka cluster from 3.2.0 to 3.4.0. Supported Strimzi Kafka Operator versions change from 0.29.0 - 0.33.2 to 0.33.2 - 0.37.0.

Before applying the update, ensure that Strimzi Kafka Operator version 0.33.2 is installed. This is the only version that supports both Kafka 3.2.0 and 3.4.0, c.f. supported versions of Strimzi Kafka Operator.

Also ensure that a shift version using Kafka 3.2.0 is running before applying the upgrade, i.e. shift version 0.133.0 or newer.

Upgrading from versions before 0.134.0​

Shift version 0.134.0 removes two no longer needed Kafkatopic Kubernetes resources (com.kobil.smartscreen.resource-changes and com.kobil.smartscreen.events). Since the included Kafka cluster by default is set to prevent topic deletion, the Kafka topics are not actually deleted and the Kafkatopic Kubernetes resources will be automatically recreated by the Strimzi topic operator. This has no negative impact. Optionally, the following steps can be performed after updating to shift version 0.134.0 to permanently delete the Kafka topics:

  • Enable topic deletion in Kafka by adding the following to shift-values.yaml and upgrade the shift helm release.

    strimzi:
      valuesOverride:
        kafka:
          config:
            delete.topic.enable: "true"

  • Delete the Kafkatopic Kubernetes resources com.kobil.smartscreen.resource-changes and com.kobil.smartscreen.events:

    kubectl -n shift delete kafkatopics.kafka.strimzi.io com.kobil.smartscreen.resource-changes com.kobil.smartscreen.events

  • Remove the valuesOverride block and upgrade the shift helm release to disable topic deletion.

Upgrading from versions before 0.133.0​

This shift version updates the version of the included Kafka cluster from 3.0.0 to 3.2.0. Supported Strimzi Kafka Operator versions change from 0.26.0 - 0.29.0 to 0.29.0 - 0.33.2.

Before applying the update, ensure that Strimzi Kafka Operator version 0.29.0 is installed. This is the only version that supports both Kafka 3.0.0 and 3.2.0, c.f. supported versions of Strimzi Kafka Operator.

Also ensure that a shift version using Kafka 3.0.0 is running before applying the upgrade, i.e. shift version 0.74.0 or newer.

Upgrading from versions before 0.93.0​

Updating to this version causes down time of Smart Screen until manual migration is performed. The existing Smart Screen must be migrated manually after performing the upgrade. Migration is done using a http request to smartscreen-services. If migration is omitted, the services will launch, but clients will see an empty Smart Screen. Any changes to Smart Screen happening after the update and before the migration will be overwritten. Since this service is not exposed outside of the cluster, port forwarding must be used, e.g.

kubectl -n <shift-namespace> port-forward svc/<svc-name-smartscreen-services> 8080:80

Then execute the following curl command.

curl -X 'POST' http://localhost:8080/v1/commands/migrate

Upgrading from versions before 0.86.0​

Due to an incompatibility in the Infinispan version used by idp-core, a rolling update is not possible when using the default idp-core image. The idp-core deployment must be scaled down to 0 before applying the upgrade. This causes downtime.

Upgrading from versions before 0.80.0​

Shift version 0.80.0 removes built in defaults for security related configurations. This includes ast-services session and database encryption keys as well as the issuer private key and certificate. To allow existing installations to keep functioning, a new value testInstallation was added. When set to true, the previous defaults are used.

Note that testInstallation: true must only be used for test and demo deployments and is not suitable for productive usage.

Upgrading from versions before 0.68.0​

  • Prepare for the upgrade

    • Some Kafka topics were renamed, which means that the Kafkatopic resource is reinstalled. To avoid topic deletion, ensure that the kafka option delete.topic.enable: "false" is set (this is the default since shift version 0.47.0).

    • To avoid that the persistent volumes (pv) are deleted if the corresponding persistent volume claims (pvc) are accidently removed, edit the persistent volume resources and change spec.persistentVolumeReclaimPolicy from Delete to Retain.

    • Some CD tools prune the pvc resources during the upgrade. In case of ArgoCD, this can be avoided by adding the following to values.yaml (this requires shift version 0.45.0 or newer).

      kafka:
        kafka:
          extraValues:
            template:
              persistentVolumeClaim:
                metadata:
                  annotations:
                    argocd.argoproj.io/sync-options: Prune=false
        zookeeper:
          extraValues:
            template:
              persistentVolumeClaim:
                metadata:
                  annotations:
                    argocd.argoproj.io/sync-options: Prune=false

    • Any of the above config changes must be applied to the currently used version of shift.

  • Before the upgrade:

    • In values.yaml, remove global.kafka.enabled and replace it with strimzi.enabled.
    • Additional Kafka topics added via value kafka.topics must be migrated to value strimzi.additionalTopics. See Section Values for details.

  • After performing the update, ast-stream application needs to be manually reset using Kafka Streams Application Reset Tool. This is required because the partitions of corresponding Kafka topics were increased.

    • Enable topic deletion in Kafka by adding the following to values.yaml and upgrade the shift helm release.

      strimzi:
        valuesOverride:
          kafka:
            config:
              delete.topic.enable: "true"

    • Scale down ast-stream service to zero. This can be done using command:

      kubectl -n <shift-namespace> scale --replicas=0 deployment <ast-stream-deployment-name>`

    • Reset the stream application using command:

      kubectl -n <shift-namespace> exec -it <kafka-pod-name-0> \
      /bin/bash -- bin/kafka-streams-application-reset.sh \
      --bootstrap-servers localhost:9092 \
      --application-id com.kobil.ast.stream

      The output of the command will be similar to

      No input or intermediate topics specified. Skipping seek.
      Deleting all internal/auto-created topics for application com.kobil.ast.stream
      Done.

    • Scale up ast-stream to the previous replicas using command:

      kubectl -n <shift-namespace> scale --replicas=1 deployment <ast-stream-deployment-name>`

    • Disable topic deletion in Kafka by removing the values added in the first step and upgrade the shift helm release.

  • After performing the update, old topics can be deleted. This step is optional.

    • Enable topic deletion in kafka by adding the following to values.yaml and upgrade the shift helm release.

      strimzi:
        valuesOverride:
          kafka:
            config:
              delete.topic.enable: "true"

    • Open a shell in the running Kafka container

      kubectl -n <shift-namespace> exec -it <kafka-pod> sh

    • Execute the following commands to delete old topics:

      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic ast.audit
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic audit
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic health-check-topic
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.client.management
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.client.management.event
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.ast.healthCheck
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.ast.ca.signerCreated
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.astlogin.events
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.astmanagement.events
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.otp.management.events
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.scp.notifier.push_messages
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.vertx.smartscreen.events
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic com.kobil.vertx.smartscreen.smartScreenTopic
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic checkStatus
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic createOperation
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic createTenant
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic createTransaction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic creditAction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic creditActionResult
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic digitalAction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic digitalActionResult
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic digitalBalanceTransaction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic initiateCancelTransaction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic initiateTransactionCreationAndPayment
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic initiateTransactionPayment
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic operationAction
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic operationActionResult
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic operationCallback
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic responseNotification
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic securityNotification
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic statusCallback
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic statusNotification
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic transactionCallback
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic transactionNotificationDeliveryData
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic transactionRequestDeliveryData
      ./bin/kafka-topics.sh --bootstrap-server localhost:9092  --delete --topic transactionRequestNotification

    • Remove the valuesOverride block and upgrade the shift helm release to disable topic deletion.

Configuration​

Smartscreen-search supports additional optional search providers which are accessible with the new endpoint /tenants/{tenantId}/provider-search. This new endpoint calls search APIs that can be configured via value smartscreenSearch.searchProviders (default []). A sample configuration looks like this

searchProviders:
  - name: Test
    uriTemplate: https://test.com/{tenantId}
    headers:
      Authorization: Bearer search-test
      Content-Type: application/json
    timeout: 100
    httpMethod: GET
    requestBody: '{ "request": "{query}" }'

The search configuration is templated. Templates will be replaced with variables when called. The following variables are currently available:

  • The search query
  • The languange used by the app (if available)
  • The tenantId where the query was called
  • The OIDC token used by the mobile app to authenticate to smartscreen components (if available)

By default, no custom search providers are configured. Calling /provider-search without search providers will return an empty result. It is possible to define multiple search providers, in which case, they will all be queried sequentially and /provider-search will only return when all searches are finished (or timed out). It is advised to take this into consideration when configuring production systems.

Share Miniapp Configuration​

Configuration of the Share Miniapp feature in Smartscreen is done via a http request to an internal endpoint. Use kubectl port forwarding to make this endpoint reachable.

kubectl port-forward svc/smartscreendashboard 8080:80

Then use the following command to configure the feature:

curl -X 'PUT' \
  'http://localhost:8080/v1/share/config' \
  -H 'accept: */*' \
  -H 'Content-Type: application/json' \
  -d '{
  "redirectUrl": "https://www.example.com/app.html",
  "appStoreLink": "https://apps.apple.com",
  "playStoreLink": "https://play.google.com",
  "defaultTitle": "App",
  "defaultImageUrl": "https://www.example.com/image.png",
  "defaultDescription": "Description",
  "tenant": "tenant",
  "appleAppSiteAssociation": {},
  "assetLinksJson": []
}'

Credentials and other sensitive data from existing Kubernetes secrets​

Shift supports reading certain credentials and security sensitive data from existing Kubernetes secrets. If this feature is used, it is not required to add them in values.yaml, because they will be ignored.

Shift currently supports four existing secrets for database credentials, admin credentials, encryption keys, and issuer CA. Usage of these secrets can be configured independent of each other:

common:
  # The name of an existing secret with datastore credentials.
  # See README.md for details and the required structure.
  # NOTE: When it's set, the datastore credentials configured in this file
  # are ignored.
  existingSecretDatastoreCredentials: "shift-datastore-credentials"

  # -- The name of an existing secret with admin credentials.
  # See README.md for details and the required structure.
  # NOTE: When it's set, the admin credentials configured in this file
  # are ignored.
  existingSecretAdminCredentials: "shift-admin-passwords"

  ast:
    # -- The name of an existing secret with encryption keys.
    # See README.md for details and the required structure.
    # NOTE: When it's set, the encryption keys configured in this file
    # are ignored.
    existingSecretEncryptionKeys: "shift-encryption-keys"

    # -- The issuer CA certificate and private key used to generate tenant signers.
    # See README.md section [Issuer CA](#issuer-ca) for requirements on issuer CA generation.
    issuer:
      # -- The name of an existing secret with issuer CA.
      # See README.md for details and the required structure.
      # NOTE: When it's set, the issuer CA configured in this file
      # is ignored.
      existingSecretIssuerCa: "shift-issuer-ca"

These secrets must be created in the same namespace where shift is deployed.

Required structure for datastore secrets​

Create a secret using below structure and add

  • Database credentials for ast services, idp-core, idp-scp-connector, scp-notifier, pay services, and Mercury components.
  • Redis password used by ast services, pay services, and idp-scp-connector.

Credentials for all supported and enabled services must be added to the secret. Not used (disabled) services can be omitted.

apiVersion: v1
kind: Secret
metadata:
  name: shift-datastore-credentials
type: Generic
stringData:
  AST_SERVICES_REDIS_PASSWORD: "change-me"
  PAY_SERVICES_REDIS_PASSWORD: "change-me"
  IDP_SERVICES_REDIS_PASSWORD: "change-me"
  MERCURY_SERVICES_REDIS_PASSWORD: "change-me"
  IDP_CORE_DB_USERNAME: "change-me"
  IDP_CORE_DB_PASSWORD: "change-me"
  IDP_SCP_CONNECTOR_DB_USERNAME: "change-me"
  IDP_SCP_CONNECTOR_DB_PASSWORD: "change-me"
  AST_CA_DB_USERNAME: "change-me"
  AST_CA_DB_PASSWORD: "change-me"
  AST_CLIENT_MANAGEMENT_DB_USERNAME: "change-me"
  AST_CLIENT_MANAGEMENT_DB_PASSWORD: "change-me"
  AST_CLIENT_PROPERTIES_DB_USERNAME: "change-me"
  AST_CLIENT_PROPERTIES_DB_PASSWORD: "change-me"
  AST_LOGIN_DB_USERNAME: "change-me"
  AST_LOGIN_DB_PASSWORD: "change-me"
  AST_VERSION_DB_USERNAME: "change-me"
  AST_VERSION_DB_PASSWORD: "change-me"
  AST_LOCALIZATION_DB_USERNAME: "change-me"
  AST_LOCALIZATION_DB_PASSWORD: "change-me"
  AST_TMS_DB_USERNAME: "change-me"
  AST_TMS_DB_PASSWORD: "change-me"
  AST_KEY_PROTECTION_DB_USERNAME: "change-me"
  AST_KEY_PROTECTION_DB_PASSWORD: "change-me"
  AST_WEBHOOKS_DB_USERNAME: "change-me"
  AST_WEBHOOKS_DB_PASSWORD: "change-me"
  AST_PUSH_NOTIFICATION_DB_USERNAME: "change-me"
  AST_PUSH_NOTIFICATION_DB_PASSWORD: "change-me"
  SCP_NOTIFIER_DB_USERNAME: "change-me"
  SCP_NOTIFIER_DB_PASSWORD: "change-me"
  PAY_DB_USERNAME: "change-me"
  PAY_DB_PASSWORD: "change-me"
  SMARTSCREEN_SERVICES_DB_USERNAME: "change-me"
  SMARTSCREEN_SERVICES_DB_PASSWORD: "change-me"
  MERCURY_ADDRESSBOOK_POSTGRES_DB_USERNAME: "change-me"
  MERCURY_ADDRESSBOOK_POSTGRES_DB_PASSWORD: "change-me"
  MERCURY_CONVERSATION_POSTGRES_DB_USERNAME: "change-me"
  MERCURY_CONVERSATION_POSTGRES_DB_PASSWORD: "change-me"
  MERCURY_BROADCAST_POSTGRES_DB_USERNAME: "change-me"
  MERCURY_BROADCAST_POSTGRES_DB_PASSWORD: "change-me"
  MERCURY_PAYCONNECTOR_POSTGRES_DB_USERNAME: "change-me"
  MERCURY_PAYCONNECTOR_POSTGRES_DB_PASSWORD: "change-me"
  MERCURY_BACKEND_SCYLLA_DB_USERNAME: "change-me"
  MERCURY_BACKEND_SCYLLA_DB_PASSWORD: "change-me"
  MERCURY_ARCHIVE_SCYLLA_DB_USERNAME: "change-me"
  MERCURY_ARCHIVE_SCYLLA_DB_PASSWORD: "change-me"
  MERCURY_CENTRIFUGO_API_KEY: "change-me"
  MERCURY_S3_ACCESS_KEY: "change-me"
  MERCURY_S3_SECRET_ACCESS_KEY: "change-me"

Required structure for admin credentials​

Create a secret using below structure and add admin credentials for idp-core. Not used (disabled) services can be omitted.

apiVersion: v1
kind: Secret
metadata:
  name: shift-admin-passwords
type: Generic
stringData:
  IDP_CORE_ADMIN_USERNAME: "admin"
  IDP_CORE_ADMIN_PASSWORD: "password"

Required structure for encryption keys​

Create a secret using below structure and add database encryption master key and session encryption master key. Both keys must be alphanumeric (UTF-8) strings of length 64.

apiVersion: v1
kind: Secret
metadata:
  name: shift-encryption-keys
type: Generic
stringData:
  DATABASE_ENCRYPTION_MASTER_KEY: ""
  SESSION_ENCRYPTION_MASTER_KEY: ""

Required structure for issuer CA​

Create a secret using below structure and add issuer CA certificate and key. Only a single self-signed certificate is supported. The certificate must be a base64 encoded self-signed certificate. The key must be the issuer private and public key in PKCS#8 format as base64 string.

apiVersion: v1
kind: Secret
metadata:
  name: shift-issuer-ca
type: Generic
data:
  ISSUER_CA_CERTIFICATE: ""
  ISSUER_CA_KEY: ""

Internal Features​

ServiceGroup for additional helm charts​

Shift chart has experimental support for adding arbitrary add-on helm charts as ServiceGroup to be managed by shift operator. This feature is configured using values:

# -- Section for configuring `add-on` helm charts to be managed by shift operator.
addons:

  # -- Name of the `add-on` helm chart
  chartname:

    # -- enable/disable deployment
    enabled: true

    # -- Chart version
    version: 0.1.0

    # -- ServiceGroup readiness check by shift operator. Set to `true` to enable.
    # A servicegroup is considered ready if all pods created by the ServiceGroup
    # are running. Shift operator uses label `app.kubernetes.io/instance` for
    # selecting pods to check. The add-on helm chart must set label
    # `app.kubernetes.io/instance: {{ .Release.Name }}` on all pods to
    # ensure readiness check works properly.
    readycheck: false

ServiceGroup's sub chart aliases​

Shift supports deploying the same helm chart multiple times using aliases. This requires shift operator version 0.9.0 or higher.

The following example demonstrates it in the spec context of a service group custom resource. The helm chart with name service-chart is deployed twice using aliases service-one and service-two.

spec:
  service-one:
    chart: service-chart
    version: 1.0.0
    fullnameOverride: {{ include "ks.siblingFullname" (merge (dict "sibling" "service-one") .) }}
    serviceTwoUrl: http://{{ include "ks.siblingFullname" (merge (dict "sibling" "service-two") .) }}
  service-two:
    chart: service-chart
    version: 1.0.0
    fullnameOverride: {{ include "ks.siblingFullname" (merge (dict "sibling" "service-two") .) }}

Shift operator uses the 'alias' to generate the helm release names. Note, that when overriding the full name (fullnameOverride) in a custom resource, the alias must be used in the sibling parameter. The same holds when configuring service names of other services, see the example value serviceTwoUrl.

Overriding arbitrary values of service helm charts​

Arbitrary default values of the services helm charts can be overridden using object valuesOverride: in custom shift-values.yaml. This feature can be used to change defaults for values that are not exposed by shift chart.

For example, to increase the memory requests and limits of idp-core to 4Gi use

idpCore:
  valuesOverride:
    mainContainer:
      resources:
        requests:
          memory: "4Gi"
        limits:
          memory: "4Gi"

Values containing helm templates can be overridden using object valuesOverrideTpl:. Values provided in valuesOverrideTpl: have a higher priority than values provided in valuesOverride:.

For example, to set the value baseUrl: of service service to the svc name of idp-core use

service:
  valuesOverrideTpl:
    baseUrl: 'http://{{ include "ks.siblingFullname" (merge (dict "sibling" "idp-core") .) }}:80/auth'