Using Openshift *oc route* for ServiceMesh exposure
OpenShift sample setup using "oc route"
Document for OpenShift specific "Route" "Gateway" "Services " setup here when using RedHat OpenShift Service Mesh (maistra.io - https://maistra.io/ )
The KOBIL Shift to "maistra.io"-ServiceMesh integration is done by:
- run standard deployment for KOBIL Shift-Operator (into project/namespace "kobil-shift" / i.e. release-name "kobil-shift-operator")
- run standard deployment for KOBIL Shift (into project/namespace
"kobil-shift" i.e. release-name "kobil-shift") here to make use of default configuration for
global.routing.istio.gateway.admin:trueand domain settingrouting.domain:<your-domain.net> - Add the OC project to the ServiceMeshMemberRole (smmr)
- Create ServiceMeshControlPlane object referencing to the KOBIL Shift Gateway object in the project/namespace
- Create a
oc routedeclaration routing to the ServiceMesh-Gateway service which is created at time of successful ServiceMesh setup
Deploy Shift on Red Hat OpenShift
This document describes additional steps that are required to deploy Shift on Red Hat OpenShift Container Platform.
Install and configure OpenShift Service Mesh
OpenShift Service Mesh is required to access Shift services from outside the OpenShift cluster. The first step is to install the required operators as described here.
Then follow the instructions here to create the ServiceMeshControlPlane. See below for an example configuration.
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
name: basic
spec:
version: v2.3
addons:
grafana:
enabled: true
jaeger:
install:
storage:
type: Memory
kiali:
enabled: true
prometheus:
enabled: true
policy:
type: Istiod
profiles:
- default
proxy:
accessLogging:
file:
name: /dev/stdout
telemetry:
type: Istiod
tracing:
sampling: 10000
type: Jaeger
gateways:
enabled: true
egress:
enabled: false
ingress:
enabled: false
openshiftRoute:
enabled: false
Store this configuration in a file named servicemeshcontrolplane.yaml and use the following command to deploy it
oc create -n istio-system -f servicemeshcontrolplane.yaml
The next step is to create a Service Mesh member roll and add the project (namespace) where Shift will be deployed as described here. The following example assumes that Shift will be deployed in project (namespace) "kobil-shift".
apiVersion: maistra.io/v1
kind: ServiceMeshMemberRoll
metadata:
name: default
spec:
members:
- kobil-shift
Store this configuration in a file named servicemeshmemberroll.yaml and use the following command to create it
oc create -n istio-system -f servicemeshmemberroll.yaml
Configure ingress gateways
The next step is to create the required ingress gateway workloads. We assume that Shift is deployed in project (namespace) shift and that Shift's default configuration is used for the following values:
global:
routing:
istio:
# -- Enable/disable istio ingress gateways for the three API groups.
# `public` gateway only routes endpoints marked as public.
# `external` gateway routes endpoints marked as public and external.
# `admin` gateway routes all endpoints.
gateways:
public: false
external: false
admin: true
options:
# -- Global prefix of istio ingress gateway name resources.
gatewayNamePrefix: istio-ingressgateway
That is, we only require one ingress gateway admin that routes all
traffic and the default gateway prefix istio-ingressgateway is used. One
additional ingress gateway must be added to the ServiceMeshControlPlane
under spec.gateways.additionalIngress. Edit the ServiceMeshControlPlane
resource basic created above to add the required configuration.
Note: The additionalIngress: object must be added to the existing gateways: object in the ServiceMeshControlPlane.
spec:
gateways:
additionalIngress:
shift-admin:
enabled: true
namespace: shift
runtime:
container:
resources:
limits:
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
deployment:
autoScaling:
enabled: false
replicas: 2
service:
metadata:
labels:
istio: istio-ingressgateway-admin
type: ClusterIP
After updating the ServiceMeshControlPlane, a deployment shift-admin and corresponding pods will be created in project (namespace) kobil-shift.
In addition, a OpenShift route is created. Since this automatic created route does not work properly, a custom route needs to be created. It is currently an open issue in OpenShift, that the automatic creation of these routes cannot be disabled.
The following example assumes that the external reachable domain name
global.routing.domain in custom "shift-values.yaml" is set to "example.com".
Create route object for Gateway service
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: shift-admin-gateway
spec:
host: wildcard.example.com
port:
targetPort: https
tls:
insecureEdgeTerminationPolicy: Redirect
termination: passthrough
to:
kind: Service
name: shift-admin
weight: 100
wildcardPolicy: Subdomain
Store this configuration in a file named shift-routes.yaml and use the following command to create it
oc create -n kobil-shift -f shift-routes.yaml
You can now proceed with the deployment of Shift.
Using multiple ingress gateways
This section explains how to configure multiple ingress gateways. We assume that
- Shift is deployed in project (namespace) shift.
- All three ingress gateways
global.routing.istio.gatewaysin custom shift-values.yaml are enabled. - The gateway prefix
global.routing.istio.options.gatewayNamePrefixin custom shift-values.yaml is set to shift-ingressgateway.
global:
routing:
istio:
# -- Enable/disable istio ingress gateways for the three API groups.
# `public` gateway only routes endpoints marked as public.
# `external` gateway routes endpoints marked as public and external.
# `admin` gateway routes all endpoints.
gateways:
public: true
external: true
admin: true
options:
# -- Global prefix of istio ingress gateway name resources.
gatewayNamePrefix: shift-ingressgateway
For each enabled gateway, one additional ingress gateway must be added to the ServiceMeshControlPlane.
spec:
gateways:
additionalIngress:
shift-admin:
enabled: true
namespace: kobil-shift
runtime:
container:
resources:
limits:
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
deployment:
autoScaling:
enabled: false
replicas: 2
service:
metadata:
labels:
istio: shift-ingressgateway-admin
type: ClusterIP
shift-external:
enabled: true
namespace: kobil-shift
runtime:
container:
resources:
limits:
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
deployment:
autoScaling:
enabled: false
replicas: 2
service:
metadata:
labels:
istio: shift-ingressgateway-external
type: ClusterIP
shift-public:
enabled: true
namespace: kobil-shift
runtime:
container:
resources:
limits:
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
deployment:
autoScaling:
enabled: false
replicas: 2
service:
metadata:
labels:
istio: shift-ingressgateway-public
type: ClusterIP
After updating the ServiceMeshControlPlane, three deployments
shift-admin, shift-external, and shift-public and corresponding pods will be created in project (namespace) kobil-shift.
Issues
When initially creating the ServiceMeshControlPlane resource, we received an error from OpenShift that a validating webhook could not be called. The problem was that OpenShift created a service object with name istiod-basic in the istio-system namespace. But the control plane tried to reach the webhook using hostname istiod. To work around this issue, we created a copy of service istiod-basic and renamed it to istiod.
ServiceMesh for Openshift / ServiceMeshControlPlane SMCP- find details here: https://docs.openshift.com/container-platform/4.10/service_mesh/v2x/ossm-reference-smcp.html