Working with geodata in the registry

🌐 This document is available in both English and Ukrainian. Use the language toggle in the top right corner to switch between versions.

1. General description

Registry administrators and regulations developers have the ability to configure work with geospatial data (geodata)[1] within the framework of business processes using the GIS[2] GeoServer, which has been implemented in the system.

At the core of the solution lies the GeoServer component, an open-source server that allows retrieving data from the database in the form of GeoJSON.

The main types are GeoJSON, Feature, and Layer.

  • GeoJSON — a data format that can be interpreted by LeafletJS and displayed on a map.

  • Feature — an object that contains geometry (information about the geographic location of the object) and attributes (additional information about the object) of a specific geographic entity.

  • Layer (layer) — a raster or vector dataset represented by a set of geographic objects that can be displayed on a map. A layer can contain information about geographic entities such as points, lines, polygons, and also their attributes and metadata.

The LeafletJs library is used to display geospatial data on UI forms in the User portals, integrating all aspects of the GeoServer and enabling map display on HTML pages.

1.1. GeoServer use cases

The platform provides the following functionality for working with geospatial data in the registry:

  • Displaying maps and satellite images supported by third-party systems in the portal for officials or service recipients, with the ability to switch between them and change the scale.

  • Displaying registry objects with a spatial reference on the map as layers.

  • Enabling multiple layers on the map.

  • Searching registry objects on the map by attributes.

  • Ability to select coordinate points, draw lines or polygons by overlaying them on the map within a business process.

  • Geocoding — searching for coordinates on the map by a public address or object name.

  • Reverse geocoding - searching for an address or object name by coordinates.

1.2. Utilizing GeoServer in the registry

Use the following action plan to configure and utilize the GeoServer in the registry:

Basic configuration:
Working with data:
Additional information:

2. GeoServer deployment

First, deploy the registry with the GeoServer.

GeoServer is a component of a separate registry template and is automatically deployed together with the registry from the corresponding template. When deploying the registry from such a template, GeoServer and Nominatim[3] are additionally installed.

Configuration of GeoServer and publication of layers occur during the regulation deployment.

To deploy the registry with GeoServer configuration in the Control Plane administrative panel , follow these steps:

  1. Access the Control Plane and open the Registries section.

  2. During the configuration process, on the Registry template step, select a template with the GeoServer. Use the recommended configuration:

    templates/registry-tenant-template-geo-server-recommended

    geoserver 1

    After making and confirming all the changes, the Jenkins process MASTER-Build-<registry-name> will start, deploying the registry with GeoServer.

    For more details, see Deploying a registry.

3. Modeling regulations for using geospatial data in business processes

3.1. Creating a registry data model

After deploying the registry, you can create a data model and use geospatial data in the regulations.

First, it is necessary to create a data model. To use geospatial data in the registry, a special attribute type="geometry" has been implemented, which extends the capabilities of the standard Liquibase library.

This parameter can be used at the column level both in table construction and search conditions (view tables).
Example 1. Modeling a table with the geometry type
<changeSet id="table geometry type" author="registry owner">
    <createTable tableName="entity_with_geo_type" ext:historyFlag="true" remarks="Сутність з геотипом">
        <column name="entity_id" type="UUID" defaultValueComputed="uuid_generate_v4()">
            <constraints nullable="false" primaryKey="true" primaryKeyName="pk_entity_id"/>
        </column>
        <column name="name" type="TEXT">
            <constraints nullable="false"/>
        </column>
        <column name="address" type="TEXT">
            <constraints nullable="false"/>
        </column>
        <column name="entity_location" type="geometry">
            <constraints nullable="false"/>
        </column>
    </createTable>
</changeSet>
Example 2. Modeling a view table (Search Condition) with the geometry type
<changeSet author="registry owner" id="create SC get_entity_with_geo_type_not_equals">
    <ext:createSearchCondition name="get_entity_with_geo_type_not_equals">
        <ext:table name="entity_with_geo_type">
            <ext:column name="entity_id"/>
            <ext:column name="name" searchType="notEqual"/>
            <ext:column name="address"/>
            <ext:column name="entity_location"/>
        </ext:table>
    </ext:createSearchCondition>
</changeSet>

After applying the changes to the master branch of the registry regulations, the Jenkins process MASTER-Build-registry-regulations is triggered, which publishes structures containing the geometry type as layers to the GeoServer.

As a result, GeoServer will contain published entities entity_with_geo_type and get_entity_with_geo_type_not_equals_v, which can be accessed to display geodata on UI forms in business processes.

For more details, see Working with GeoServer.

3.2. Modeling business process scenarios and forms

After deploying the registry data model and creating data layers according to the data model, you will be able to write to or read from the database objects that contain coordinates of specific points, lines, polygons, etc.

3.2.1. Manually entering coordinates into the database

Manually enter the address and coordinates of the object (latitude and longitude) in the corresponding fields of the data entry form.

Use the standard Text Field component when modeling forms.

Use Groovy scripts to retrieve data from the form and save the data to the data factory.

3.2.1.1. Modeling the business process

Model a business process that allows you to enter data with object coordinates and save them to the data factory.

  1. Model a User Task for data entry and associate it with the corresponding UI form using the Form key parameter.

    geoserver 2

  2. Model a User Task for signing data with a Qualified Digital Signature and associate it with the corresponding UI form using the Form key parameter.

    Pass the data for signing from the previous form through the submission() function in the Form data pre-population field. For example:

    ${submission('addGeoActivity').formData}

    geoserver 3

  3. Model a script task to retrieve data from a UI form based on the corresponding ID for further processing and saving the object’s coordinates to the database.

    Script to retrieve coordinates from a UI form and create an object for storing geodata 
    def signedFormData = submission('signGeoActivity').formData

def entityLocation = [:]
entityLocation.type = 'point'
entityLocation.latitude = signedFormData.prop('latitude').value()
entityLocation.longitude = signedFormData.prop('longitude').value()

signedFormData.prop('entityLocation', S(entityLocation, 'application/json'))
signedFormData.deleteProp('latitude')
signedFormData.deleteProp('longitude')

set_transient_variable('payload', signedFormData)

    This script combines latitude and longitude values into a single object named entityLocation and stores this object in signedFormData.

    1. It retrieves form data (formData) using the submission() function, passing the signature form ID signGeoActivity, and saves them into the variable signedFormData.

    2. It creates an object (a map of key-value pairs) named entityLocation.

      The entityLocation parameter corresponds to the entity_location column name in the registry database, which you have defined as the one to store your geodata.

      The column is created according to your data model with the attribute type="geometry".

      <column name="entity_location" type="geometry">
    <constraints nullable="false"/>
</column>

    3. It defines the type as point (point on map) for entityLocation.

    4. It saves the latitude (latitude) and longitude (longitude) values from signedFormData into entityLocation.

      It adds a new attribute entityLocation to signedFormData and assigns the JSON representation of the entityLocation object as its value.

    5. It sets the payload variable as a temporary variable storing signedFormData. It can be further used in the business process.

    geoserver 4

  4. Model a Service Task to sign data with a system key.

    Configuring:

    • Use the delegate System signature by DSO service from the template catalog for applying a system signature.

    • Pass the input data as a variable ${payload} in the corresponding field.

    • Pass the token of the executor of the last user task in the business process: ${completer('signGeoActivity').accessToken}.

    • Store the response in the variable system_signature_key.

    geoserver 5

  5. Save the data to the database. Create a new record in the database, storing the value of the entityLocation object in the corresponding column.

    • Use the delegate Create entity in data factory to create an entity in the database.

    • Specify the resource/API endpoint entity-with-geo-type corresponding to the name of the table with geodata defined in the data model registry - entity_with_geo_type.

    • Pass the input data as a variable ${payload} in the corresponding field.

    • Pass the token of the executor of the last user task in the business process: ${completer('signGeoActivity').accessToken}.

    • Specify the source of the system signature. Use the function sign_submission():
      ${sign_submission('signGeoActivity').signatureDocumentId}.

    • Specify as the variable ${system_signature_key} the Ceph document key containing information about the signed data.

    • Save the response to the result variable, for example, createGeoResponse.

    geoserver 6

3.2.1.2. Modeling data input UI forms

Model the data input forms for your business process. The form’s names should correspond to the value of the Form key parameter in the respective user tasks of the business process.

  1. Model a UI form for entering data about an object: name, address, and coordinates (latitude and longitude).

    • Use the Text Field component for all four fields.

    • For each field, define a business label (Tab Display > Label) and a parameter name for the API (Tab API > Property Name).

      geoserver 7

      geoserver 7 1

    • The UI form in the User portal may look like this:

      geoserver 8

    • The parameters to be stored in the data factory will have the following format:

      geoserver 9

  2. Model a UI form for verifying the entered data before signing it with a qualified digital signature. This form will have the same fields as the data input form. On this form, the user can only verify the accuracy of the entered data before signing.

3.2.2. Selecting coordinates on the map and saving them to the database

Model a UI form with a Map component for using the map in the business process.

Use Groovy scripts to retrieve data from the form and save it to the data factory.

3.2.2.1. Modeling the business process

Model a business process that allows you to select coordinates (point, line, or polygon) on the map and save them to the data factory.

  1. Model a user task for selecting coordinates on the map and integrate it with the corresponding UI form using the Form key parameter.

geoserver 10

  1. Model a script task for retrieving data from the UI form with the map based on the corresponding ID, for further processing and saving the object’s coordinates to the database.

    Script for retrieving coordinates from the map and creating an object to store geodata: 
    def formDataForm = submission('show-map').formData

def data = S([:], 'application/json')
	data.prop("name", formDataForm.prop("name"))
	data.prop("address", formDataForm.prop("address"))
	data.prop("entityLocation", formDataForm.prop('entityLocation').prop('geometry').toString())

println "data: " + data

execution.removeVariable('payload')
set_transient_variable('payload', data)

    In general, this script retrieves data from a form, creates a new JSON object with the received data, and stores it in a temporary variable payload:

    1. It creates a variable formDataForm and retrieves form data with the identifier show-map using the JUEL submission() function.

      • It creates a new JSON object called data with an empty dictionary and the data type application/json.

    2. It populates the data object with properties "name," "address," and "entityLocation", extracting the corresponding values from the formDataForm object.

      Note that the nested object 'geometry' in the "entityLocation" property is converted into a string.

    3. It sets a new variable 'payload' using the values of the data object, which can be further used in the business process.

    geoserver 11

  2. Model a user task (User Task) for signing the data with a digital signature and link it to the corresponding UI form using the Form key parameter.

    Pass the data to be signed as the variable ${payload} in the Form data pre-population field.

    geoserver 12

  3. Model a script task for processing and saving the signed data. The script used here is almost identical to the previous one, with the only difference being that the 'geometry' object in the 'entityLocation' property is not converted to a string but passed as a JSON object.

    Script for processing and saving data, signed with the qualified electronic signature: 
    def formDataForm = submission('show-map').formData

def data = S([:], 'application/json')
	data.prop("name", formDataForm.prop("name"))
	data.prop("address", formDataForm.prop("address"))
	data.prop("entityLocation", formDataForm.prop('entityLocation').prop('geometry'))

println "data: " + data

execution.removeVariable('payload')
set_transient_variable('payload', data)

  4. Model a service task for signing the data with a system key.

    • Use the delegate System signature by DSO service from the template catalog for applying the system signature.

    • Pass the input data as the variable ${payload} in the respective field.

    • Pass the token of the executor of the last user task in the business process: ${completer('signGeoActivity').accessToken}.

    • Store the response in the variable system_signature_key.

      geoserver 5

  5. Save the data to the database. Create a new record in the database, saving the value of the entityLocation object to the corresponding column.

    • Use the delegate Create entity in data factory to create an entity in the database.

    • Specify the resource/API endpoint entity-with-geo-type, which corresponds to the table name of the geodata you defined when creating the registry data model — entity_with_geo_type.

    • Pass the input data as the variable ${payload} in the respective field.

    • Pass the token of the executor of the last user task in the business process: ${completer('ID of the task for signing the data with a qualified digital signature').accessToken}.

    • Specify the source of the system signature. Use the function sign_submission():
      ${sign_submission('*ID of the task for signing the data with a qualified digital signature').signatureDocumentId}.

    • Specify as the variable ${system_signature_key} the key of the Ceph document that contains information about the signed data.

    • record your response in the result variable, for example, createGeoResponse.

    geoserver 6

3.2.2.2. Modeling data input UI forms

Model the data input UI forms. Unlike the previous case where we manually enter coordinates, we will now consider the possibility of entering object coordinates directly from the map into the database.

  1. Create a form for selecting coordinates on the map using the MAP component.

    • Define a label, for example, entityLocation.

    • Configure it on the Data tab.

    • Go to the API tab and define the Property Name as entityLocation. This parameter is used for data exchange through the API.

      For more details, see Map component

      map 1

  2. Create a form for signing the data with the qualified digital signature. Model three text fields for the data that will be saved to the database after digital signing:

    • address — object address;

    • name — object name;

    • entityLocation - object coordinates (a point on the map, line, or polygon).

    geoserver 16

3.2.3. Changing coordinates and information about them

You can modify previously entered coordinates. To do this, simply initiate the corresponding business process, select a specific geographic object on the map (point, line, or polygon) that needs to be changed, then select a new object and overwrite the values in the database.

3.2.3.1. Modeling the business process

  1. Model a user task to select coordinates on the map that need to be changed, and link it to the corresponding UI form using the Form key (form’s internal name).

    geoserver 21

  2. Using a script, obtain the entity ID in the database (entityId) that needs to be changed.

    geoserver 22

    Script for retrieving form data, including the entityId entity 
    def formDataForm = submission('choose-coordinates-id').formData
println "formDataForm: " +  formDataForm

def data = S([:], 'application/json')

	data.prop("entityId", formDataForm.prop('map').prop('properties').prop("id").value())
	data.prop("name", formDataForm.prop('map').prop('properties').prop("name").value())
	data.prop("address", formDataForm.prop('map').prop('properties').prop("address").value())

execution.removeVariable('payload')
set_transient_variable('payload', data)

    This script performs the following actions:

    1. It defines the formDataForm variable, which retrieves data from the form that was submitted with the identifier choose-coordinates-id using the JUEL submission() function.

    2. It creates a new object called data with an empty dictionary and the data type application/json.

    3. It populates the data object with data from formDataForm, such as entityId, name, and address.

    4. It sets the payload variable as a temporary variable and assigns it the value of data.

  3. Next, create a user task to select new coordinates on the map and link it to the corresponding UI form using the Form key (form’s internal name).

    Pass the data from the script to the form as a variable ${payload} in the Form data pre-population field.

    geoserver 23

  4. Using a script, retrieve the updated entity data that needs to be saved in the database.

    geoserver 24

    Script for retrieving updated data and coordinates from the form 
    def formDataForm = submission('ID користувацької задачі для вибору нових координат').formData
println "formDataForm: " +  formDataForm

def data = S([:], 'application/json')

	data.prop("entityId", formDataForm.prop('map').prop('properties').prop("id").value())
	data.prop("name", formDataForm.prop('map').prop('properties').prop("name").value())
	data.prop("address", formDataForm.prop('map').prop('properties').prop("address").value())

execution.removeVariable('payload')
set_transient_variable('payload', data)

    This script performs the following actions:

    1. It defines the variable formDataForm, which receives data from the form submitted with the identifier choose-coordinates-id using the JUEL function submission().

    2. Creates a new object called data with an empty dictionary and the data type application/json.

    3. Populates the data object with the data from formDataForm, such as entityId, name, and address.

    4. Sets the variable payload as a temporary variable and assigns it the value of data.

  5. Then, create a user task for signing the data with the qialified digital signature and link it to the corresponding UI form using the Form key (form’s internal name).

    Pass the data for signature from the script to the form as a variable ${payload} in the Form data pre-population field.

    geoserver 25

  6. Using a script, obtain the data signed with the qualified digital signature that needs to be saved in the database.

    geoserver 24

    Script for retrieving signed data from a form 
    def formDataForm = submission('choose-new-coord').formData
println "formDataForm choose-new-coord " + formDataForm

def data = S([:], 'application/json')

	data.prop("entityId", formDataForm.prop("entityId"))
	data.prop("name", formDataForm.prop("name"))
	data.prop("address", formDataForm.prop("address"))
	data.prop("entityLocation", formDataForm.prop('entityLocation').prop('geometry'))

execution.removeVariable('payload')
set_transient_variable('payload', data)

println "payloadData: " + data

    1. It defines the variable formDataForm, which receives data from the form submitted with the identifier choose-coordinates-id using the JUEL function submission().

    2. Creates a new object called data with an empty dictionary and the data type application/json.

    3. Populates the data object with the data from formDataForm, such as entityId, name, and address.

    4. Sets the variable payload as a temporary variable and assigns it the value of data.

    This script is similar to the previous one, but with one small difference: it does not invoke the toString() method for the geometry property of the entityLocation object. As a result, the value of entityLocation remains in its original format (an object) instead of being converted to a string.

  7. Model a service task for data signing with a system key.

    • Use the delegate System signature by DSO service from the template catalog to apply a system signature.

    • Pass the input data as the variable ${payload} in the corresponding field.

    • Pass the token of the executor of the last user task in the business process: ${completer('ID останньої користувацької задачі для підпису даних').accessToken}.

    • Store the response in the variable system_signature_key.

      geoserver 5

  8. Update the entity in the database.

    Use the Update entity in data factory delegate or the generic connector Connect to data factory with the PUT method.

    geoserver 27

    For example, pass the resource value and entity identifier as follows, using the submission function:

    entity-with-geo-type/${submission('ID of the user task for selecting new coordinates').formData.prop('entityId').value()}

    • entity-with-geo-type - the resource/endpoint corresponding to the entity_with_geo_type table in the database.

    • entityId - the identifier of the entity to be updated, obtained from the respective form.

    For more info, see bp-modeling/bp/element-templates/bp-element-templates-installation-configuration.adoc.
3.2.3.2. Modeling data input forms

  1. Model a form for selecting coordinates on the map using the MAP component.

    • Define a label, for example, Map.

    • Configure it on the Data tab.

    • Go to the API tab and define the Property Name as map. This parameter is used for data exchange through the API.

      For mor details, see Map component

      geoserver 14

  2. Next, model another form for updating coordinates and object information. Use Text Field components for text fields and the MAP component for selecting new coordinates on the map.

    • Define a label, for example, entityLocation.

    • Configure it on the Data tab.

    • Go to the API tab and define the Property Name as entityLocation. This parameter is used for data exchange through the API.

      For more details about the MAP component, see Map component

      geoserver 15

  3. Create a form for signing data with the qualified digital signature. Model three text fields for the data that will be saved to the database after digital signing:

    • address — object address;

    • name — object name;

    • entityLocation - object coordinates (point on the map, line, or polygon).

    geoserver 16

3.2.4. Searching for geographic objects on the map with geocoding functionality

Users have the ability to view all geographic objects on the map that are stored in the database and search for these objects using geocoding.

3.2.4.1. Modeling the business process

To display a map with coordinates of all available objects, it is sufficient to model a simple process with a start form.

Connect the start task to a data input form using the Form key.

geoserver 13

3.2.4.2. Modeling the object search form

Visualizing geodata on user interface forms in user dashboards can be achieved using the FormIO component Map. This component provides full functionality for working with geospatial data in the registry.

Geocoding (searching for geographic objects) is activated directly on the UI forms through the settings of the Map component.

map 5

For more information about the MAP component, refer to Map component.

3.3. Working with geodata in user dashboards

Users can utilize pre-modeled business processes to work with maps and geodata in the registry.

To do this, simply go to the personal dashboard, find the section Available services, and launch one of the available processes (e.g., entering object coordinates into the database, and so on).

geoserver 28

4. Working with GeoServer

GeoServer — сервіс, який дозволяє отримувати дані з БД у вигляді GeoJSON для їх подальшої обробки та відображення на мапі у бізнес-процесах.

All data structures within the regulations that contain the geometry type are published as layers to the GeoServer.

The configuration is published during the deployment of the regulations, in the publish-geoserver-configuration step of the main Jenkins process MASTER-Build-registry-regulations.

To manage the GeoServer settings, a web interface is provided, which can be accessed through the environment of your registry: https://geo-server-<registry-name>.apps.envone.dev.registry.eua.gov.ua/geoserver.

Viewing layers is GeoServer

Layer (Layer) — is a collection of objects (Features).

Feature is an individual object on the map that contains geometric and attribute data.

Objects can be:

  • points ("type": "Point");

  • lines ("type": "Polyline");

  • polygons ("type": "Polygon").

They represent various elements on the Earth’s surface, such as buildings, rivers, lakes, roads, etc. Each object feature contains geometry that indicates its spatial location (e.g., entityLocation) and properties that provide additional information about the object (e.g., name and address).

In the context of working with the GeoServer of the registry, a published layer is either a table or a view (Search Condition).

To view all layers published to the GeoServer, follow these steps:

  1. Log in to the GeoServer as an administrator.

    geoserver 18

  2. Open the Layer Preview section.

    geoserver 19

    You will see all layers (tables or views from your registry database) that contain the type geometry.

  3. Select the GeoJSON data format from the drop-down list next to the corresponding layer for data preview.

    geoserver 19 1

    As a result, you will see a large FeatureCollection object with a set of geometric (coordinates) and attribute (object name on the map, address, etc.) data.

    geoserver 20

5. Working with tables in the registry database

Geospatial data is stored in a specialized table in the registry database, which you define as the repository for this data. The geometric elements themselves (coordinates of points, lines, and polygons) are stored in a dedicated column that supports the geometry data type, according to your data model (for more details, see Creating a registry data model).

geoserver 29
Figure 1. Example of storing geodata in the column entity_location of the table entity_with_geo_type
geoserver 30
Figure 2. Visualization of geodata on the map

6. Working with the API

Information about all features (geometry and attributes) for each layer can be obtained directly from the registry’s API in the registry-rest-api service.

The corresponding access points will be automatically created based on the specified tables and search conditions in the data model. For example, entity-with-geo-type, and so on.

All generated API endpoints for the respective registry are presented in the openapi specification and can be accessed at: https://registry-rest-api-<registry-name>.apps.envone.dev.registry.eua.gov.ua/openapi.

Make sure to add /openapi at the end of the link, otherwise, you will be directed to the Swagger testing environment (sandbox).

7. Related pages

1. Geospatial data (geodata) — is data that includes information related to geographic location and can be associated with specific geographic objects, such as cities, rivers, forests, buildings, and so on.
2. GIS (Geographic Information System) — is software that enables the collection, storage, analysis, visualization, and even forecasting of various geospatial data.
3. Nominatim — is a geocoder that can convert addresses or place names into their corresponding geographic coordinates, and vice versa — geographic coordinates into addresses or place names.