Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.
Dotenv is definitely something that should not exist.
Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser!
Seems really silly to have a library do what two lines of shell can reasonably accomplish.
The concept is fine, but I'm a big fan of doing it with mise instead. It does the exact same thing plus a lot of other things really well. The environment for a project is not just things that get stored in environment variables, it also includes specific versions of tools, common project tasks, git hooks and such. Being able to manage all of that in one place ends up being really useful.
I also really like that for sensitive variables (AWS tokens, prod DB passwords, etc), I can keep those in a dedicated secret store (I use 1Password for personal projects and there are more enterprise focused alternatives for work).
direnv does one thing really well, and it enables a large ecosystem of other components that you can pick and choose to build developer environments. SecretSpec, from the featured article, is one such component which pairs really well with direnv, and has the features you’re describing for secrets.
I think direnv’s approach is the one with more staying power.
direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
> But migrating away from .env starts with reading it correctly.
We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well.
Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other.
It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see.
Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.
I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs.
We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it.
The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake.
And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together.
Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?
.env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
How exactly changing a default that users rely on is going to make it easy to move away from it?
Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely.
edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec.
I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
It’s an open source project, so you don’t have to wish it.
“Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”
The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell.
I have a shell function for this in my zsh config:
function lenv() {
export $(cat .env | xargs)
}
It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.
You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed.
If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.
Dotenv is definitely something that should not exist.
Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser!
Seems really silly to have a library do what two lines of shell can reasonably accomplish.
The concept is fine, but I'm a big fan of doing it with mise instead. It does the exact same thing plus a lot of other things really well. The environment for a project is not just things that get stored in environment variables, it also includes specific versions of tools, common project tasks, git hooks and such. Being able to manage all of that in one place ends up being really useful.
I also really like that for sensitive variables (AWS tokens, prod DB passwords, etc), I can keep those in a dedicated secret store (I use 1Password for personal projects and there are more enterprise focused alternatives for work).
mise is trying to do everything.
direnv does one thing really well, and it enables a large ecosystem of other components that you can pick and choose to build developer environments. SecretSpec, from the featured article, is one such component which pairs really well with direnv, and has the features you’re describing for secrets.
I think direnv’s approach is the one with more staying power.
I'm biased but I think it's silly to think either one is going away anytime soon
No, I also use direnv. It does everything I want, nothing I don't, and has been stable for me for years.
direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
A bonus is that with that approach, you actually have real environment variables.
I also used to use direnv (and likewise eschewed .env), but more recently I moved env definitions to mise. Been working out nicely.
I use Mise for env. I primarily use it for installing tools and running tasks, having env support is just a cherry on top.
Direnv's first stable release was in 2013, .env was 2012 (adopted by Heroku).
So no, if anything direnv makes you hip and edgy.
> But migrating away from .env starts with reading it correctly.
We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well.
Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other.
It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see.
Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.
I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs.
We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
> the guard over the missile launch switch
That's called a molly guard, but also a Debian package that asks you before you run shutdown if you're on the server you intended to shut down.
I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it.
The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake.
And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together.
Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
Maybe this is a dumb question but what does -ng mean? I see a lot of forks seem to follow the <project>-ng naming convention.
Next generation.
It’s not a great convention.
Ah, ok. That makes sense.
Next Generation
I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?
What does the ng at the end means? I have seen a few products/projects that added it and no explanation of the meaning.
I came in and thought it had something to do with Angular since `ng` is a popular pre/post-fix for Angular libraries.
Probably next generation.
Next generation I guess.
Can’t you just escape the dollar sign?
If you really believe .env was a mistake (it was), just let it die. You are the ones keeping it alive now.
.env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
How exactly changing a default that users rely on is going to make it easy to move away from it?
Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely.
edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec.
I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
I think mise offers a better solution for the problem.
well mise uses dotenvy right now (for env._file anyhow), it may be worth switching to dotenv-ng but I haven't looked into it yet https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758...
I assume you just mean normal [env] in mise.toml which is different, but this still is relevant to mise
Yes - I think .env files are not a good idea (works for one person but even then it's not great)
I love AI development, but I wish product announcements were self-authored.
It’s an open source project, so you don’t have to wish it.
“Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”
The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell.
I have a shell function for this in my zsh config:
It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed.
If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
Use this wrapper instead:
doppler all the way