Downloading digital documents from external sources: scripting capabilities

🌐 This document is available in both English and Ukrainian. Use the language toggle in the top right corner to switch between versions.

1. Overview

To correctly process the data obtained as a result of interaction with external systems within business processes, there is a need to download digital documents from external sources and store them into the registry repository.

As a solution, the implementation of a JUEL function has been considered, which enables the initiation of remote address file retrieval from the business process script tasks, and saving of the received file to the interim business process object storage for further use in generating a request to the registry data factory.

2. Actors and user roles

Registry regulations developer

3. Functional scenarios

File retrieval from a remote address and loading it into the interim business process object storage in the business process script tasks.

4. General provisions

Initiating file retrieval and storage from the business process through the JUEL function is performed by a system user
Saving files from a remote address is allowed only for service/internal use within the scope of business process execution
The size of retrievable files is limited by system-level configurations
File retrieval and storage are performed outside the business process by the Digital documents service.
Digitally stored documents through the JUEL function are subject to deletion upon completion of the business process execution

5. Technical design of the solution

This diagram illustrates the involved services and their interactions to meet the requirements. Additionally, important considerations to be taken into account within the implementation are highlighted.

Downloading digital documents from external sources requires the presence of a corresponding Istio Service Entry, which was created in automatic or manual mode.

In the target solution, for retrieving files from a remote system during the BP execution, the registry’s technical administrator must preconfigure the respective external system through the admin console, following the design in Managing settings and secrets for external integrations

5.1. API for saving a digital document from a remote address

The purpose of the API route is to retrieve a file located at a remote address within configured size constraints and subsequently save it to the Interim business process object storage.

Retrieving a file from a remote server and loading it into the object repository should be implemented using streaming and appropriate memory usage constraints for the server application.

5.1.1. Access authorization

Non-public / for internal cross-service interaction

5.1.2. API route specification

POST /internal-api/documents/{processInstanceId}/

Table 1. Request headers
Header	Type	Description
X-Access-Token	JWT	Access token

Table 2. Request parameters
Parameter	Type	Request part	Description
processInstanceId	Text	Request parameter	Business process identifier for file upload
remoteFileLocation	URL	Request parameter	Remote file location for upload
filename	Text	Request parameter	File name

Table 3. Response body structure
Json Path	Type	Description
$.id	UUID	Unique identifier of the digital document generated using a pseudo-random number generator
$.name	Text	Original file name
$.type	Text	File content type (application/pdf, image/png, image/jpeg, etc.)
$.checksum	Text	Automatically generated content hash using SHA256 algorithm
$.size	Numeric	File size

Response example

{
  "id": "{UUID}",
  "name": "{fileName}",
  "type": "{contentType}",
  "checksum": "{sha256}",
  "size": 0
}

Table 4. Response codes
Code	Description
201	Created with response body
400	Malformed request
401	Authentication error (access token missing)
422	Validation error (invalid file size, etc.)
500	Server request processing error

5.2. System settings

Expand the configuration of the Digital documents service with additional settings:

max-remote-file-size - Limit on the file size for remote upload (default: 10MB)

6. Modeling registry regulations

6.1. Extension for modeling

The following JUEL function needs to be implemented:

save_digital_document_from_url(String remoteFileUrl, String targetFileName): DocumentMetadata

Digital document metadata object structure

class DocumentMetadata {
  String id       // Unique identifier of the digital document
  String name     // Original file name
  String type     // File content type
  String checksum // SHA256 hash of the file content
  Long size       // File size
}

6.2. Reference modeling examples

Example of using save_digital_document_from_url in the development of a script task in a business process:

try {
  def documentMetadata = save_digital_document_from_url("http://...", "digital-document.ext") // Temporary save file to object storage

  def fileReference = [
    id: documentMetadata.id,
    checksum: documentMetadata.checksum
  ]
} catch (ValidationException ex) {
 // File size exceeded "max-remote-file-size" value
}

BE / Camunda

8.2. Development plan

Extension of the Digital documents service API with a utility route for file upload and storage
Extension / implementation of a Java client for the Digital documents service
Implementation of a JUEL function for uploading files on behalf of a system user to the Digital documents service
Development of guidelines and reference examples for regulations developers