Declarative configuration of system emulators: simplified testing of external registry integrations

🌐 This document is available in both English and Ukrainian. Use the language toggle in the top right corner to switch between versions.

1. General overview

There is a problem that the current mocking solution is done with SoapUI and the integration point mocking services come with the platform. This leads to the fact that it is not possible to quickly add a new integration point mock or change an existing one without providing a platform update.

To solve this problem, it is necessary to replace the current approach for creating mocks with the implementation of a new unified strategy that makes it possible to configure custom mocks for integration points in real time without delivering a new version of the platform.

2. Actors and user roles

Registry developer
Platform developer

3. Glossary

WM - WireMock
WMS - WireMock Studio
Mocks - integration point stubs
Control Plane Console - platform admin console

4. Current operation of mocking integration points

In this diagram you can see that to update/add moku you need:

The platform developer creates a mock in the SOAP UI
The platform developer adds a change to the installer
Registry developer can use shared mock after platform upgrade

It is also worth noting that the mock is at the platform level, not the registry, that is, it is common to all registries on the cluster.

4.1. Disadvantages of the current solution

Mocks are delivered together with the platform, which makes it impossible to change the regulations by the developer
There is no way to quickly integrate with a new external registry admin system without the help of a platform developer
There is no way to customize the integration point mock for the registry

5. Technical solution design

5.1. Mocks settings via Control Plane

Figure 1. Setting up the mock configuration of the external integration point

It is important to note that after the configuration is saved, the flow described in the design is executed Management of settings and secrets of external integrations including the following remarks:

If the user wants to lock the external REST system that was created - it is necessary to delete the service entry for it
If the user wants to add a mock of an external REST system, it is necessary to create a secret without a service entry

5.1.1. Steps to deploy your own mock of integration point

Figure 2. Configuring registry integration with an external API mock

Platform administrator adds external integration for registry dev with Trembita (Secure exchange gateway) or another external system
In Gerrit Control Plane Console, the configuration file has been updated to integrate with the external system

control-plane-gerrit:<registry>.git/deployment-templates/values.yaml

#Ukrainian specific implementation
trembita:
#SOAP external systems integration
  registries:
    edr-registry:
      #Link on registry wiremock
      url: "http://wiremock:9021/"
      mock: true
      ...
    dracs-registry:
      #Link on registry wiremock
      url: "http://wiremock:9021/"
      mock: true
      ...
    idp-exchange-service-registry:
      #Link on registry wiremock
      url: "http://wiremock:9021/"
      mock: true
      ...
external-systems:
#REST external systems integration
  #Ukrainian specific implementation
  diia:
    #Link on registry wiremock
    url: "http://wiremock:9021/"
    mock: true
    ...
  http-bin:
    #Link on registry wiremock
    url: "http://wiremock:9021/"
    mock: true
    ...
  secured-service:
    #Link on registry wiremock
    url: "http://wiremock:9021/"
    mock: true
    ...

The link changes only for the selected system to the registry WM, which is accessed via the service address (http://wiremock:9021/).

If mocking is selected for the external system, a flag is stored in the file that allows you to identify the system that refers to the mock.

5.1.2. Mocking configuration management

This functionality is specific to the Ukrainian implementation and may not apply or function as described in other contexts or regions. Please consult the local guidelines or documentation if you are implementing this outside Ukraine.

Figure 3. Setting up the registry interaction via Trembita for registry dev

Figure 4. Setting up the interaction with mock of Trembita integration points for dev registry

Figure 5. Setting up the interaction of the registry with an external system

Figure 6. Setting up the interaction of the registry with the mock of the integration points of external systems

5.1.3. Integration scenarios

System name	Level	Protocol	Integration type
diia (Citizen-facing solution)	Registry	REST	External integration
trembita (Secure exchange gateway)	Registry	SOAP	External integration
<Other external systems>	Registry	REST/SOAP	External integration

System name

Level

Protocol

Integration type

diia (Citizen-facing solution)

Registry

REST

External integration

trembita (Secure exchange gateway)

Registry

SOAP

External integration

<Other external systems>

Registry

REST/SOAP

External integration

5.2. Mocking of external integration point

It is suggested to use WM for mocking integration points. With this approach, it is possible to mock both REST and SOAP APIs.

Figure 7. Using custom mocks for the registry

Figure 8. Add custom mock for registry using

5.3. Access from services

For certain registry and platform services, it is necessary to enable interaction with the registry WM service on port 9021.

Network policies that are necessary for interaction with the mocking service of external integrations at the registry level

Service name	Service label
admin-tools-jenkins	app=jenkins
bpms	app=bpms
ddm-notification-service	app.kubernetes.io/instance = ddm-notification-service

Service name

Service label

admin-tools-jenkins

app=jenkins

bpms

app=bpms

ddm-notification-service

app.kubernetes.io/instance = ddm-notification-service

Istio policies that are necessary for interaction with the external integration mocking service at the platform level

Service name	Service label
keycloak-manag	app.kubernetes.io/name=keycloak

Service name

Service label

keycloak-manag

app.kubernetes.io/name=keycloak

6. Modeling of regulations

To save mappings in the registry repository, it is necessary to follow the structure. The mapping file also stores the response provided by the mock of the external integration point

Figure 9. Example of setting mock-integrations

6.1. Example of mapping files and response processing

WM supports both SOAP and REST requests, so below is an example of processing both types of requests

File with mappings for EDR mock edr-registry.json (Download)

{
  "mappings": [
    {
      "request": {
        "urlPath": "/mockEdr",
        "method": "POST"
      },
      "response": {
        "status": 200,
        "body": "<soap11env:Envelope\n        xmlns:soap11env=\"http://schemas.xmlsoap.org/soap/envelope/\"\n        xmlns:tns=\"http://nais.gov.ua/api/sevdeir/EDR\"\n        xmlns:xroad=\"http://x-road.eu/xsd/xroad.xsd\"\n        xmlns:id=\"http://x-road.eu/xsd/identifiers\">\n    <soap11env:Header>\n        <tns:AuthorizationToken>token</tns:AuthorizationToken>\n        <xroad:userId>MDTUDDM</xroad:userId>\n        <xroad:client id:objectType=\"SUBSYSTEM\">\n            <id:xRoadInstance>SEVDEIR-TEST</id:xRoadInstance>\n            <id:memberClass>GOV</id:memberClass>\n            <id:memberCode>43395033</id:memberCode>\n            <id:subsystemCode>IDGOV_TEST_01</id:subsystemCode>\n        </xroad:client>\n        <xroad:service id:objectType=\"SERVICE\">\n            <id:xRoadInstance>SEVDEIR-TEST</id:xRoadInstance>\n            <id:memberClass>GOV</id:memberClass>\n            <id:memberCode>00015622</id:memberCode>\n            <id:subsystemCode>2_MJU_EDR_prod</id:subsystemCode>\n            <id:serviceCode>SearchSubjects</id:serviceCode>\n        </xroad:service>\n        <xroad:protocolVersion>4.0</xroad:protocolVersion>\n        <xroad:id>MDTUDDM</xroad:id>\n        <xroad:requestHash algorithmId=\"http://www.w3.org/2001/04/xmldsig-more#gost34311\">YzS4MYmFiW8tkoncbQL624RllowfcK8B8FGNTWZ5QFE=</xroad:requestHash>\n    </soap11env:Header>\n    <soap11env:Body>\n        <tns:SearchSubjectsResponse>\n            <tns:SubjectList>\n                <!--Темплейтинг мапінгу-->\n                {{#each (soapXPath request.body '/SearchSubjects/code/text()') as |thing|}}\n                {{#if (contains thing '101')}}\n                <tns:state>1</tns:state>\n                <tns:state_text>зареєстровано</tns:state_text>\n                <tns:name>Сидоренко Василь Леонідович</tns:name>\n                <tns:url>http://zqedr-api.nais.gov.ua/1.0/subjects/3</tns:url>\n                <tns:code>{{thing}}</tns:code>\n                <tns:id>3</tns:id>\n                {{else if (contains thing '123213123')}}\n                <tns:state>1</tns:state>\n                <tns:state_text>зареєстровано</tns:state_text>\n                <tns:name>Петренко Петро Петрович</tns:name>\n                <tns:url>http://zqedr-api.nais.gov.ua/1.0/subjects/3</tns:url>\n                <tns:code>{{thing}}</tns:code>\n                <tns:id>3</tns:id>\n                {{/if}}\n                {{/each}}\n                <tns:SubjectInfo>\n                </tns:SubjectInfo>\n            </tns:SubjectList>\n        </tns:SearchSubjectsResponse>\n    </soap11env:Body>\n</soap11env:Envelope>",
        "headers": {
          "Content-Type": "text/xml"
        },
      "transformers": [
        "response-template",
        "body-transformer"
      ]
      }
    }
  ]
}

Formatted response mock-integration/edr-registry.json (Download)

<soap11env:Envelope
        xmlns:soap11env="http://schemas.xmlsoap.org/soap/envelope/"
        xmlns:tns="http://nais.gov.ua/api/sevdeir/EDR"
        xmlns:xroad="http://x-road.eu/xsd/xroad.xsd"
        xmlns:id="http://x-road.eu/xsd/identifiers">
    <soap11env:Header>
        <tns:AuthorizationToken>token</tns:AuthorizationToken>
        <xroad:userId>MDTUDDM</xroad:userId>
        <xroad:client id:objectType="SUBSYSTEM">
            <id:xRoadInstance>SEVDEIR-TEST</id:xRoadInstance>
            <id:memberClass>GOV</id:memberClass>
            <id:memberCode>43395033</id:memberCode>
            <id:subsystemCode>IDGOV_TEST_01</id:subsystemCode>
        </xroad:client>
        <xroad:service id:objectType="SERVICE">
            <id:xRoadInstance>SEVDEIR-TEST</id:xRoadInstance>
            <id:memberClass>GOV</id:memberClass>
            <id:memberCode>00015622</id:memberCode>
            <id:subsystemCode>2_MJU_EDR_prod</id:subsystemCode>
            <id:serviceCode>SearchSubjects</id:serviceCode>
        </xroad:service>
        <xroad:protocolVersion>4.0</xroad:protocolVersion>
        <xroad:id>MDTUDDM</xroad:id>
        <xroad:requestHash algorithmId="http://www.w3.org/2001/04/xmldsig-more#gost34311">YzS4MYmFiW8tkoncbQL624RllowfcK8B8FGNTWZ5QFE=</xroad:requestHash>
    </soap11env:Header>
    <soap11env:Body>
        <tns:SearchSubjectsResponse>
            <tns:SubjectList>
                <!--Темплейтинг мапінгу-->
                {{#each (soapXPath request.body '/SearchSubjects/code/text()') as |thing|}}
                {{#if (contains thing '101')}}
                <tns:state>1</tns:state>
                <tns:state_text>зареєстровано</tns:state_text>
                <tns:name>Сидоренко Василь Леонідович</tns:name>
                <tns:url>http://zqedr-api.nais.gov.ua/1.0/subjects/3</tns:url>
                <tns:code>{{thing}}</tns:code>
                <tns:id>3</tns:id>
                {{else if (contains thing '123213123')}}
                <tns:state>1</tns:state>
                <tns:state_text>зареєстровано</tns:state_text>
                <tns:name>Петренко Петро Петрович</tns:name>
                <tns:url>http://zqedr-api.nais.gov.ua/1.0/subjects/3</tns:url>
                <tns:code>{{thing}}</tns:code>
                <tns:id>3</tns:id>
                {{/if}}
                {{/each}}
                <tns:SubjectInfo>
                </tns:SubjectInfo>
            </tns:SubjectList>
        </tns:SearchSubjectsResponse>
    </soap11env:Body>
</soap11env:Envelope>

File with mappings diia.json (Citizen-facing solution)(Download)

{
  "mappings": [
    {
      "request": {
        "urlPath": "/api/partner-token/post-info",
        "method": "POST",
        "headers": {
          "Accept": {
            "contains": ".*"
          }
        },
        "bodyPatterns": [{
          "matchesJsonPath": "$.info"
        }]
      },
      "response": {
        "status": 200,
        "body": "{\n  \"info\": \"string\"\n}",
        "headers": {
          "Content-Type": "text/xml"
        },
        "transformers": [
          "response-template",
          "body-transformer"
        ]
      }
    }
  ]
}

After adding the mock, the user will be able to go to the address http:registry-wiremock/mockEdr to get a mocked response

Link to the documentation according to which you need to create your own mappings

During the execution of the registry-regulations-management pipeline, if mappings are available, a query is executed

curl -v -d @mapping.json http://wiremock:9021/__admin/mappings/import

Mappings can be stored in several files, so you need to import all files with mappings using the API

6.2. Migration plan for current mocking solutions

Create mock mappings of current external systems
Add mapping import to post-upgrade scripts
Update the user instructions to create new mocks at the user’s request
Mark existing mochas as those components that will be replaced in future releases

6.3. An example of creating your own WMS mock

Mapping for import can be created in WMS, for this you need:

Go to WMS and click the Mock API button

Choose an empty template and fill in the name

Go to the Stubs tab

Press the New button

Fill in the mapping settings and save it

Export the file to upload it to the Registry Gerrit

7. High-level development plan

7.1. Technical examinations

DevOps
Dev

7.2. Development plan

WM deployment is performed only during the deployment of dev registers and there is no root for it
Updating the Control Plane Console with new functionality
Update the Control Plane Jenkins pipeline for registry deployment to account for described changes
Adding a stage to the registry-regulations pipeline for importing mappings
Develop WM mappings of current solutions and mappings for Diia and add them to the demo registry
Development of user instructions with instructions on how to develop, add, update mappings with a link to the demo registry