I am looking for a concept that handles the needs of some data to be distributed in several environments (lets say we have dev, test, stage and prod).
The native tool for it is GIT – if you put any kind of data (i.e. files with configs or migrations in database) there, it will be eventually properly distributed to all the environments.
Another example can be some external translation tool, where you want same translations for all environments.
I have two use-cases where I want same data to be shared across multiple environments:
We are setting up planned outages of banks we are using and share that information through REST API with our clients. Now banks basically do not provide sandbox/dev environments (its just some mock versions and its VERY different to real use-cases) while clients need to test it even on sandbox. Therefore sandbox and prod are both connected to real banks. As clients are using sandbox to testing a lot of things, they should know about outages there as well. At this moment we have to fill it twice – on sandbox and on prod environment.
We need to setup some dynamic configuration which can be changed in real-time. For example there are some error-handling rules that we need to update quickly and we want same rules to be applied to all the environments – as they appear same on all environments.
Now what is good way to handle the above? I would love to keep all envs as separated as possible, but I do not want to introduce duplicated manual work
same data to be shared across multiple environments …
… changed in real-time
Universally available data that is up-to-the-minute.
That sounds like “the internet”.
You mentioned config files that are stored in git,
and presumably are available on github.
I will assume that all environments enjoy outbound
internet connectivity, or equivalently that all
environments can access private web server(s) that you control.
Use a web browser to visit the
config.txt file on github,
click the “Raw” button, and have your various environments
use that URL.
Wrap a suitable “cache to local /tmp directory” layer
around such access if desired.
As far as access control goes, there will be three cases:
config.txtis publicly available, and environments simply fetch it.
config.txt.encryptedis publicly available, you disseminate suitable symmetric (AES-256) or asymmetric keying material to each environment, and they fetch + decrypt the file.
config.txtis in a Private repo, you disseminate suitable github credentials such as a PAT to each environment, and they fetch that file.