Decommissioning of saving previous versions of data factory API services

🌐 This document is available in both English and Ukrainian. Use the language toggle in the top right corner to switch between versions.

1. Overview

Currently, during the registry regulations deployment, the code and Docker images of all previous versions of data API are saved. Additionally, the registry developer is required to change the version of the regulations every time they modify the data model.

Since the old versions are not used when operating or maintaining the registry, it is recommended to discontinue saving them and cancel the version check when deploying the regulations.

2. Actors and user roles

  • Regulations developer

  • Registry administrator

3. General provisions

  • Only the latest versions of the generated data API services are stored in the registry Gerrit.

  • One service build pipeline instance per service. Pipelines are not created for individual versions.

  • Only the latest versions of the generated data API services are stored in the registry Nexus.

  • In Kafka, topics that are not used by the latest version of the API are deleted.

  • Mandatory change of the regulations version is canceled.

4. Functional scenarios

  • Developing the registry data model

  • Deploying the registry regulations

5. Current implementation

Deploying the registry regulations data model
Figure 1. Deploying the registry regulations data model

6. Target design

Deploying the registry and registry regulations data model
Figure 2. Deploying the registry and registry regulations data model
It makes sense to consider the possibility of not including the creation of the codebase and data API pipelines in the registry deployment pipeline. Their creation can remain in the data model deployment pipeline, following the "create if not exists" principle. This way, there won’t be a need for changes in the regulations creation procedures (which are different for CI/CD and target environments) and the creation of migration upgrade scripts.

6.1. System components and their purpose within the solution design

This section provides a list of system components that are used or require modification/creation as part of implementing the functional requirements according to the technical design of the solution.

Component Service name Purpose / What’s changed

JobProvisioner

jenkins-operator

Creating codebase and pipelines for data API services.

JobProvisionerDefault

control-plane-jenkins

Creating codebase and pipelines for data API services.

Registry regulations publication pipeline

registry-regulations-publication-pipeline

Publishing the regulations without saving the previous versions of the generated components.

Liquibase scripts for pre- and post-deployment data model

data-model

Removing the regulations version check.

7. Registry regulations modeling

7.1. Regulations version

The requirement to change the version when changing the data model is canceled.

Regulations administrators can use the settings.general.version attribute in the regulations settings file settings.yaml at their discretion.

Registry regulations structure
Figure 3. Registry regulations structure

7.2. Migration

When updating the registry, you need to create and run a migration script that will create unversioned codebases and data API pipelines.

After the update, to clean up the codebase and outdated API artifacts, it is recommended to perform the current version cleanup procedures - a cleanup pipeline for development environments and manual cleaning for production environments.

8. High-level development plan

8.1. Technical expertise

  • DEVOPS

8.2. Development plan

  • Make changes to the process of registry deployment and regulations data model publishing.

  • Update documentation for the regulations developers.