BTP Kyma ORY Hydra OAuth2 Client migration… ver. extended, revised

If you are an active SAP BTP Kyma user, under certain conditions you may see a notification in your dashboard.

Note says:

You are using Ory Hydra Auth2, which has been deprecated since May 19, 2022 and is scheduled to be removed on October 14, 2023. During the Kyma upgrade on October 14, 2023, workloads that still rely on deprecated Ory Hydra Auth2 may experience minor outages. To prevent this, follow the procedure outlined in the blog post below before the specified date. If you are unable to follow the procedure, Ory Hydra Auth2 will be migrated to the hydra-deprecated Namespace to prevent outages caused by API call failures.
However, you will then be fully responsible for managing ORY Hydra Auth2.
This includes troubleshooting, maintenance, security updates, vulnerability fixes, and adherence to product standards. If you want to continue using the Auth2 functionality provided as part of the SAP BTP, Kyma runtime offering, delete the hydra-deprecated Namespace or follow the migration steps outlined in the blog post below.

There is a link to SAP Blog, (nope, not a part of official SAP documentation)
SAP BTP, Kyma runtime ORY Hydra OAuth2 Client migration, which partially covers topic.

NOTE
There are no blog posts, documentation, or SAP Support KBA how to install or "troubleshooting, maintenance, security updates, vulnerability fixes, and adherence to product standards." for your custom ORY installation.

I am not going to cover that too, as there is too much of an effort in doing that. Sorry.
What I will add here are just steps to make your migration less painful.

Partially, as that one covers only setting up authentication rule in APIRule.

The blog post discusses adding a new rule to secured endpoint by APIRule, which checks that standard OAuth2 Kyma client (that part is that mysterious ORY thingy from the API).

There is a lot of gray area here, to be frank.

Firstly, placeholders used in scripts, where do I get this information to fill values?

OAuth2 Credentials

The first code snippet describes a sample OAuth2 client, which should be replaced.

cat <<EOF | kubectl apply -f -
apiVersion: hydra.ory.sh/v1alpha1              
kind: OAuth2Client       
metadata:
  name: test_app                   
  namespace: $NAMESPACE
spec:
  grantTypes:      
    - "client_credentials"
  scope: "test"                 
  secretName: test_app            
EOF

$NAMESPACE – that is the easiest one, you can get it from top right corner in dashboard. That should be the namespace where your OAuth2 configuration is located.

In my case that is services.

$ export NAMESPACE=services

These namespaces are part of Kubernetes environment.

API Rule

Second and third code snippet contains another variables:

curl -ik -X POST "https://oauth2.$KYMA_DOMAIN/oauth2/token" -H "Authorization: Basic $ENCODED_CREDENTIALS" -F "grant_type=client_credentials" -F "scope=test"

$KYMA_DOMAIN – is the URL, which you can take directly from dashboard. Just open main page for your Kyma cluster.

You can find a URL to your API service. $KYMA_DOMAIN is that one without api. subdomain.

$ export KYMA_DOMAIN=c-9XXXXXX9.kyma.ondemand.com

The second placeholder is $ENCODED_CREDENTIALS. It is strictly related to secrets to given OAuth2 client.
When you open: Configuration > OAuth2 Clients on the left menu and view one of entry, you can find on the bottom client_id and client_secret. Firstly Decode them, and then copy for later. These data have to be Base64 encoded

export CLIENT_ID="<client_id>"
export CLIENT_SECRET="<client_secret>"
export ENCODED_CREDENTIALS=$(echo -n "$CLIENT_ID:$CLIENT_SECRET" | base64)

Adding SAP Cloud Identity Services

It is explicitly mentioned to use SAP Cloud Identity Services, so be it.

Let's go to your BTP cockpit and search in Services > Service Marketplace that particular service:

I have opened it and clicked Create and followed the 2-step wizard to complete the process.

Error

In my case, instance creation failed with a note to SAP Help: Establish Trust and Federation Between SAP Authorization and Trust Management Service and Identity Authentication

Tip
There is an official SAP BTP Learning video about that step available on YT: Cloud Identity Services Identity Authentication | SAP Business Technology Platform.

I have followed steps from help page and added as Sub-Account Administrator new entry to Trust Configuration.

NOTE
Read through restrictions on help page, as that operation may not be possible in some cases.

Gray Area

The part related to IAS provisioning and making it available as a tenant in Trust area on BTP is beyond my knowledge, as that operation is done... via IT ticket. I wasn't able to do that so had to ask my colleagues to request it.

Configuration

Tip
Pieter Janssens wrote nice blog about technical users configuration. I recommend to go through that later on SAP Blog: Managing technical users for BTP platform access

When you log in to IAS, by default you are redirected to profile page: https://<instance-id>.accounts<optional-id>.ondemand.com/ui/protected/profilemanagement

In order to access admin cockpit, you need to visit sub-page:
https://<instance-id>.accounts<optional-id>.ondemand.com/admin

You need to create new application:

After clicking on create, I have configured it as a OpenID Connect:

SAP Cloud Identity Services Configuratoin

In order to get IAS environment variables, we need to open auto-discovery link: https://<instance-id>.accounts<optional-id>.ondemand.com/.well-known/openid-configuration.

In my case that is: https://sandbox.accounts.ondemand.com/.well-known/openid-configuration

export IAS_ENC_CREDS={APPLICATION_ENCODED_CREDENTIALS}
export IAS_TOKEN_URL={TOKEN_URL}
export IAS_INTRO_URL={INTROSPECTION_URL}
export IAS_INTRO_ENC_CREDS={INTROSPECTION_ENCODED_CEDENTIALS}

Credentials Migration

The next step is to move login data to the new service.

Fetching data from Kyma

In order to get data from Kyma we need to extract data stored in Kubernetes Secret resources.

First, it is required to set up your KUBECONFIG value pointing to kubeconfig.yaml for your environment. File is available from your BTP and Kyma cockpits.

export KUBECONFIG=<path-to-kubeconfig.yaml>

Then we can filter out our resources:

kubectl get secrets

Extracting the right data is up to you. Depending on your data, you can filter out by namespace and name of the secret by using --field-selector. In my case, filtering out was done after fetching data from Kubernetes.

Data are encoded with base64. In order to import user data to IAS, you need to decode them first.

Importing User Profile to IAS/CIS

In IAS admin cockpit you can find possibility to upload CSV data. Then read about Full User Profile and this: Import or Update Users for a Specific Application

https://me.sap.com/notes/0002607696

NOTE
CSV file format is tricky here. Documentation says that:

• there are 3 mandatory columns: userName (or loginName), emails[0].value (or mail), name.familyName (or lastName),
• columns are separated by commas,
• multi-values for one column are separated by semicolons,
• column names are case-sensitive,
• mapping commas to columns is strict, meaning there should not be any orphan commas, otherwise will end up with error
• there is no clear information what is wrong with file-content
• import will fail if file contains additional spaces

I never thought that I will get so many import errors with simple CSV file… At the end, I was not able to find what was the import error and created users manually.

CSV Import process does not allow you to set up same password as it used to be. It is possible to set it manually.