App Configuration, where does it go? Config files, env vars, external service?
Vložit
- čas přidán 4. 10. 2023
- How do you manage your App Configuration? Things like connection strings, hostnames, 3rd party API keys, and application-specific settings like log levels. There are config files, environment variables, and external configuration services that you can use. Let me sort it out and explain the different situations so you can choose the best option. It's not a one size fits all.
🔗 EventStoreDB
eventsto.re/codeopinion
🔔 Subscribe: / @codeopinion
💥 Join this channel to get access to a private Discord Server and any source code in my videos.
🔥 Join via Patreon
/ codeopinion
✔️ Join via CZcams
/ @codeopinion
📝 Blog: codeopinion.com
👋 Twitter: / codeopinion
✨ LinkedIn: / dcomartin
📧 Weekly Updates: mailchi.mp/63c7a0b3ff38/codeo... - Věda a technologie
How are you storing your configuration? In code, config files, external services like AWS ParameterStore, Secrets Manager, Azure App Config, Key Vault? Let me know!
We use appsettings.[env].json. But for prod we use a find-and-replace step in the pipeline to inject the sensitive data from the KeyVault. It strikes a great balance between ease of development and testing, while still maintaining security for prod.
I tend to prefer using environment vars or something that doesn't require restarting for feature flags. Basically we use common sense and a bit of logic in deciding how to do things instead of going all-out on one solution that doesn't fit any scenario 100%.
Azure App Config for basic config, connected to KeyVault for storing secret. We use Configuration as Code approach connected to Deployment pipeline, so whenever there is update of code and some service needs new variables, they are automatically added to App Config with the release
I mainly use environment variables, but also key vaults. We've configured Kubernetes to fetch secrets automatically from key vaults based on certain roles, and those secrets are used as arguments when starting apps. So the apps themselves doesn't integrate with the key vaults.
how do you handle it when there is a need to refresh the secrets? Restarting the app is the only viable option I am seeing here.
I am really keen on reducing complexity in my apps as much as possible. So I like env variables the most. They are likely the oldest tech of all configuration options you mentioned, they are available in all systems, they are reliable, you can put them in your deployment pipelines and with a bit of fiddling you can even do some more. Under Linux you have up to 128 kb (131k Characters) available. I once did a base64 on a json config file and put it as an env variable. Worked like a charm (as long as you don't do Windows, which has a limit of 32k Characters...)
This video was extremely good. I seldom subscribe to anything, but this was to the point, words cherry picked pefectly. Cudos!
Thanks, hopefully you find some other of my videos helpful.
what credentials is your app using to access the external configuration store, and where are those stored? if your app locally caches configuration, where does it cache it, and why is that anymore secure than a flat file? the idea of storing credentials is a key vault suddenly got very popular, but I haven't heard anyone explain how it's actually better - it sounds like it's just creating a new version of the same problems? plus, if there is actually a good solution to this problem, why aren't APIs and service just natively using the same strategy? I would love to see a video that actually explains these things.
Finally! ♥
Where do you store the credentials to connect to a configuration store itself?
In Azure you can use managed identity for this, but you still need to configure the name of the store, just not the access keys.
Often its identity of the service. Example in AWS you give the task in ECS a role that has access to Parameter Store.
We use Azure App Configuration to populate environment variables in k8s during deployment for config, and Azure Key Vault for secrets, and managed identity where possible.
AAC has a 30k req/hr limit though which is causing us headaches as we try to use it for feature flag management, even though it refreshes in the background rather than every time you need to check a flag. They seem to want you to make an AAC instance per service, which is nuts.
I'm currently writing something to get the config out of AAC and broadcast it to all services as it changes but trying not to over engineer something that should be so simple.
How can we make the communication between services and Configuration service secure?
Hi Derek, thanks for the video! I don't think many people think about availability in the context of application configs and it is definitely worth it, especially in the case of scalable applications...
Could you give any example of what could be that config store? 🤔
I also think that it is worth of reconsideration of what COULD be stored in env vars even if it's kind of "against the rules" e.g. if we have a virtual machine hidden in some virtual private network and limited access, one could say that it is ok if we store some technical user credentials etc. without paying extra fees for azure key vault or aws secrets storages.
Out of the box cloud solutions like AWS Parameter Store, Secrets Manager, Azure App Config, Key Vault as examples. The problem with env vars is they have to get set somehow. Depending on how you deploy and infrastructure, so as long as that isn't defined in some plain text file like I was illustrating with a docker compose or AWS ECS Task definition.
What's your opinion about constants in application?
Basically it's configuration in code. I don't have a problem with it assuming it's fairly static.
Caching sensitive data / secrets gives a new problem right? How to protect those secrets.
It could. Depends how you persist it and what tooling you have available to you (eg, AWS KMS thats IAM policy based.)
After fetching the secrets from configuration service (ex: AWS ParameterStore), where do you store them? The secrets that I am trying to store are static, and so there is no point of keep calling the ParameterStore to fetch the same secrets every time. For the storing mechanism, I am thinking either store these secrets in local variable of the application, in-memory cache such as Redis, or environment variables using some injection sidecar container. Any recommendation?
I mostly use Laravel and its strategy goes like this:
1) .env file with secrets and other env-specific things
2) those get loaded and saved to an in-memory config object so they can be modified at runtime
3) Classes that depend on the above convifuration are usually injected into the DI container with the default config but if you want to both change the config and keep resolving the class from DI for testing purposes, you can modify the config just before resolving the class