This is a technical question related to what is possible to accomplish in a plugin and may require a developer to help answer.
From testing we have found that it appears that all dictionary entries are loaded during a deployment to an environment even when they are not used by any deployables. With standard dictionaries this is fine as the data is simply loaded from a database. This however presents a problem when dictionary entries come from external resources.
I'm looking at developing a plugin that pulls in dictionary values from an external resource. The dictionary would have keys and know how to fetch values but the actual values would be pulled from an external resource. What I want is for the dictionary values to be "lazy loaded" - namely that they will not be resolved from the external resource unless a deployable needs them.
For example:
Environment:
- Dictionary 1
- key1 : <empty> // value resolved during deployment from external resource
- Dictionary 2
- key2 : <empty> // value resolved during deployment from external resource
Application:
- Deployable:
- File that uses {{key2}}
When deploying the application to the environment the ideal situation would be that since only values from dictionary 2 are used then only the value of key2 will be loaded from an external resource with key1 being skipped. Currently both key1 and key2 are loaded as the current deploy code does not check if a dictionary needs to be used or not.
This does not scale well when there are a lot of dictionaries in an environment. I am aware that dictionaries can be restricted to applications but that would be a large effort to switch to doing this for our existing setup. Furthermore reorganizing environments to be smaller would also be a massive effort.
With the current API is it possible to achieve delayed dictionary evaluation like I want above?
If not maybe this could be a feature request?
I think what would be needed is for the dictionary API to also have a getKeys() method that defaults to returning getEntries().keyset(). The deployment logic would look at all the placeholders used by deployables and compare that against all the keys from participating dictionaries. If none of the keys in a particular dictionary are used by any deployables then loading the entries from said dictionary could be skipped. In my case getKeys() and getEntries() could be overridden such that keys are known up front and getEntries() end up hitting the external resources. Since getEntries() would only be called if a key is used by a deployable in the depolyment loading the values from the external resource could be skipped when it is not needed.
NOTE: This was supposedly moved to a feature request under R-03853 but cannot monitor it
by: Bryan G. | about a year ago | Deployments
Comments
The current API for fetching a list of entries can be found at https://apidocs.digital.ai/xl-deploy/23.1.x/rest-api/com.xebialabs.deployit.engine.api.DeploymentService.html#/deployment/dictionary:GET. This API returns all dictionary entries for the specified environment and application, depending on the parameters restrictedContainers and restrictedApplications.
Are we correct in understanding that your use case necessitates a resolved list of dictionaries for the provided application and environment? Currently, there isn't a dedicated API that offers this specific information. While an API exclusively for resolved dictionaries is not available, the existing API at https://apidocs.digital.ai/xl-deploy/23.1.x/rest-api/com.xebialabs.deployit.engine.api.DeploymentService.html#/deployment/dictionary:GET, when followed by a POST to /deployment/prepare/deployeds, can be utilized to retrieve the resolvedPlaceholders in the response.
To gain a clearer understanding of your use case, it would be beneficial to learn how you are presently resolving and passing these values to the deployment within the plugin logic.