Step one in making a new repo is disabling Dependabot. Automate intentionally or get pwned your choice.
Also, is automated version bumps really such a good thing? Many times I have wasted hours tracking down a bug that was introduced by bumping library. Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does! It is so much better to update intentionally, test, deploy. Though this does assume you have a modest number of dependencies which pretty much excludes any kind of server-side javascript project.
I don’t disagree about automating intentionally, but it’s worth noting that Dependabot isn’t enabled by default: you have to explicitly configure it.
(The larger problem here isn’t even Dependabot per se, since all Dependabot does is fire PRs off. The problem is that people then try to automate the merging of those PRs, and end up shooting themselves in the foot with GHA’s more general footguns. It also doesn’t help that, until recently, GitHub’s documentation recommended using these kinds of dangerous triggers for automating Dependabot.)
> Really? Dependabot runs on a number of my repositories without my having consciously enabled it.
I've never experienced this. Do you have a `.github/dependabot.yml` file in your repository? That's how it's enabled.
(GitHub has muddied the water here a bit by having two related but distinct things with the same name: there's "Dependabot" the subject of this post, and then there's "Dependabot security updates" which are documented separately and appear to operate on a different cycle[1]. I don't know if this latter one is enabled by default or not, but the "normal" one is definitely disabled until you configure it.)
My understanding, and it may be wrong, is you may be grandfathered in to an ancient Personal, Public Repo opt-out from a brief window of time just after GitHub was excited to announce the first/earliest version of Dependabot and was hoping it would clean up some Open Source supply chain attacks and just before GitHub realized Dependabot was a useful thing to charge people an upcharge on (now under the umbrella known as GitHub Advanced Security). I believe that GitHub auto-opted in a lot of personal accounts with "significant" Public repos (anything with a bunch of forks/stars, or a package identifier visible in the dependency graphs of the Ruby or npm ecosystems, or any of the things that awarded "badges" like Mars Rover badge or the Artic Vault badge). There's a page buried in your Personal Account Settings to turn off that ancient Dependabot option. (I'm on a work machine without access to my personal account at the moment or I'd directly tell you where to find it.)
I'm at a loss to explain that! My only other guess is that you might have enabled Dependabot at some point further back in history, when it was a third-party integration and directly owned by or integrated into GitHub.
Do you have a Dependbot entry in your account/org-level applications?
Okay, I have no idea then. I guess perhaps at one point Dependabot was enabled by default for some people, although that strikes me as a bad idea and I can only assume they've disabled it since then, since I haven't seen this on any new repository I've made.
How do you scale this besides keeping the dep list short? Are you reading every item or just scanning for words like "deprecated" or "breaking change"?
How do you prevent exposing yourself to supply chain attacks like the tj-actions/changed-files one [0] if you don't?
I get your question regarding scaling, but that's the job: you can choose to outsource code to 3rd-party libraries, and eternal vigilance is the trade-off.
Assume your 3rd-party dependencies will try to attack you at some point: they could be malicious; they could be hacked; they could be issued a secret court order; they could be corrupted; they could be beaten up until they pushed a change.
Unless you have some sort of contract or other legal protection and feel comfortable enforcing them, behave accordingly.
That's a judgement call. It would be too much to review all code change of all dependencies unfortunately.
The corollary of reviewing all code on all dependency updates is you should review all code or the new deps you add, including the transformation by build processes that might mean what is in the package manager might be different and same for all transitive dependencies.
> Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does!
Still have flashbacks from that one time when some dependency in our Go project dropped support for go1.18 in its patch version update, and we almost couldn't rebuild the project before the Friday evening. Because obviously /s being literally unable to build the dependency is a backwards-compatible change.
> Also, is automated version bumps really such a good thing?
Depends. Do you want to persist the belief that software requires constant maintenance because it's constantly changing? Then yes: automate your version bumps and do it as often as possible.
If you want software to be stable then only update versions when you have a bug.
I'm going through SOC2 right now, and it is essentially requiring it to be enabled. I'll just do the minimal to pass, but it isn't something you can just disable in some cases.
FWIW: this might be a suggestion of the specific audit team you’re working with or a requirement of one of the “follow our playbook and you’ll pass” vendors if you’re using one of those, but the SOC 2 on its own doesn’t really impose specific technical feature/control requirements like this.
I don’t have the exact exam language in front of me right now but the requirement would be something like “you have some process for learning about, assessing, and mitigating vulnerabilities in software dependencies that you use”.
Enabling an automated scan and version bump tool like dependabot is a common and easy way to prove your organization has those capabilities. But you could implement whatever process you want here and prove that you do it on the schedule you say you do in order to satisfy the audit requirement.
True on all counts. But the lowest effort is "just turn off dependabot", which is what I suspect most of the people trying to get past SOC2 will do (like myself).
Curious if auto-merge philosophy changes between libraries and applications. The library definitely has a larger user base to break and a wider matrix of use-cases. IMO, auto-merge is more palatable for an application – do you agree? Especially when you're under SOC2/FedRAMP/etc.
Work has been fiddling around with Dependabot and force-enabling it everywhere (not auto-merging, just having it generating the PRs)... my feedback was that it is built on the presumption that All Updates Are Automatically Good, but this is transparently a false statement. In fact, Dependabot, by being so fast and automatic, may actually raise the probability of some project getting malicious code injected into it! Consider the timeline of malicious code injection:
1. Malicious code is injected into some project.
2. People have a chance to pick it up and put it into their code.
3. The malicious code is found, publicized, and people react.
The faster you act after step 1, the more chance you'll have time to put it into your system before the world reaches step 3. Dependabot maximizes the speed of reaction after step one. If I'm doing things somewhat more manually then I'm much more likely to experience the world finding out about a corrupted dependency before I start incorporating it.
Now, just typing it out it may sound like I'm more freaked out than I actually am. While supply-chain attacks are a problem, they are getting worse, and they will continue to get worse, they are also still an exotic situation bubbling on the fringe of my awareness, as opposed to something I'm encountering regularly. For a reasonable project the most likely outcome is that dependabot enhancing this exposure window will still not have any actual real-world impact, and I'm aware of that. However, where this is relevant is if you are thinking of Dependabot and its workflow as a way of managing security risk, because you imagine updates as likely carrying security improvements, and that's your primary purpose for using it (as opposed to other uses, such as, your system slowly falling behind in dependencies until it calcifies and can't be updated without a huge degree of effort, a perfectly reasonable threat and Dependabot is a sensible response to that), then you also need to consider the ways in which is may actually increase your vulnerability to threats like supply-chain attacks.
And of course, projects do not start out with all their vulnerabilities on day one and then monotonically remove them. Many vulnerabilities are introduced later. For each such vulnerability, there is a first release that includes them, and for which treating that update as if it was just a Good Thing was in fact not true, and anyone who pushed it in as quickly as possible made a mistake. Unfortunately, sometimes hard problems are just hard problems.
Though I have wondered about the idea of programming something like Dependabot, but telling it, hey, tell me about known CVEs and security releases, but otherwise, let things cook for 6 months before automatically building a PR for me to update. That would radically reduce this risk I'm outlining here.
(In fact, after pondering, I'm kind of reminded of how Debian and a lot of Linux distros work, with their staged Cutting Edge versus Testing versus Stable versus Long Term Support. Dependabot sort of builds in the presumption that you want that Cutting Edge level of updates... but in many cases, no, I really don't. I'd much rather build with Stable or Long Term Support for a lot of things, and dip into the riskier end of the pool for specific things if I need to.)
I have only skimmed the docs and I wouldn't be completely shocked if there's a "wait X weeks/months to notify about non-security updates" but I don't know about it if it's there. If it is there, hey, great! Won't be the first time I really wished that X did Y and found out that yes it already does.
The folks at Synaktiv had a nice detailed blog post on this same vector last year[1].
The bottom line with these kinds of things is that virtually nobody should be using `pull_request_target`, even with “trusted” machine actors like Dependabot. It’s a pretty terrible footgun.
If people are this obsessed with making the little version number of their dependencies as big as possible why don't they just make downloading the dependencies part of the build system?? Or better yet make the user supply their own dependencies and link them in as shared libraries? This problem was solved at least 30 years ago...
Wait, how is it possible for anyone who opens a PR to issue dependabot commands for main repository? There should be some kind of authorization in place to avoid it, right? Should it not ignore any commands coming from outside users who do not have commit access?
> Here's the trick: github.actor does not always refer to the actual creator of the Pull Request. It's the user who caused the latest event that triggered the workflow.
Seems like a bit forced scenario to me. I have never seen anyone auto-merge Dependabot PR automatically using GitHub Action.
Also pull_request_target is a big red flag in any GHA and even highlighted in GHA docs. It’s like running untrusted code with all your secrets handed over to it.
> So, they created workflows to auto-merge PRs if the creator was Dependabot. Seems safe, doesn't it?
No? In what world would it be safe to merge code, AI-generated or not, which you haven't reviewed, much less do it automatically without you even knowing it happened?
How do you know that you need the changes (whether bug or CVE)? How do you know the code isn't malicious? How do you know your systems are compatible with the change? How do you know you won't need to perform manual work during the migration?
I think this response is understandable. But also feels disingenuous. If a dependency update appears to be an automated version bump, and you trust your test base, and you trust the dependency author (did you read the whole dependency's dependency graph before importing it? You probably didn't.) it's only a small step to auto-merge. The horror here is that the bot isn't the author and the version update isn't legitimate. That's on Github's presentation and settings, in my opinion.
Relying on a human reviewer, regardless, is a weak guarantee. If your security posture is "Joe shouldn't make mistakes", you still have a weak security posture.
> ... and you trust the dependency author ... it's only a small step to auto-merge
I disagree for the reasons listed above, but we're talking about 3rd-party dependencies here, not trusted ones. Given the numerous scenarios I listed above, it's a huge step from "using a 3rd party library" to "trusting the author of it".
We'd have to start with "...and you don't trust the author", because for most 3rd-party dependencies, the author has given us neither sufficient evidence to do so, nor sufficient recourse if that trust is violated.
> Relying on a human reviewer, regardless, is a weak guarantee.
Relying on countless random 3rd party to not own you when they know people are pulling in their code as a dependency is a far, far worse guarantee. How would that strategy have protected someone against this supply-chain attack?:
Step one in making a new repo is disabling Dependabot. Automate intentionally or get pwned your choice.
Also, is automated version bumps really such a good thing? Many times I have wasted hours tracking down a bug that was introduced by bumping library. Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does! It is so much better to update intentionally, test, deploy. Though this does assume you have a modest number of dependencies which pretty much excludes any kind of server-side javascript project.
I don’t disagree about automating intentionally, but it’s worth noting that Dependabot isn’t enabled by default: you have to explicitly configure it.
(The larger problem here isn’t even Dependabot per se, since all Dependabot does is fire PRs off. The problem is that people then try to automate the merging of those PRs, and end up shooting themselves in the foot with GHA’s more general footguns. It also doesn’t help that, until recently, GitHub’s documentation recommended using these kinds of dangerous triggers for automating Dependabot.)
> Dependabot isn’t enabled by default: you have to explicitly configure it.
Really? Dependabot runs on a number of my repositories without my having consciously enabled it.
Could it have been a Dependabot security update? These are different from normal Dependabot updates and do not require `.github/dependabot.yml`.
Are these repos forks of projects that already set up dependabot? Or maybe created from templates that included dependabot configuration?
> Really? Dependabot runs on a number of my repositories without my having consciously enabled it.
I've never experienced this. Do you have a `.github/dependabot.yml` file in your repository? That's how it's enabled.
(GitHub has muddied the water here a bit by having two related but distinct things with the same name: there's "Dependabot" the subject of this post, and then there's "Dependabot security updates" which are documented separately and appear to operate on a different cycle[1]. I don't know if this latter one is enabled by default or not, but the "normal" one is definitely disabled until you configure it.)
[1]: https://docs.github.com/en/code-security/dependabot/dependab...
> I've never experienced this. Do you have a `.github/dependabot.yml` file in your repository? That's how it's enabled.
Nope. Example: https://github.com/m50d/tierney/pull/55
My understanding, and it may be wrong, is you may be grandfathered in to an ancient Personal, Public Repo opt-out from a brief window of time just after GitHub was excited to announce the first/earliest version of Dependabot and was hoping it would clean up some Open Source supply chain attacks and just before GitHub realized Dependabot was a useful thing to charge people an upcharge on (now under the umbrella known as GitHub Advanced Security). I believe that GitHub auto-opted in a lot of personal accounts with "significant" Public repos (anything with a bunch of forks/stars, or a package identifier visible in the dependency graphs of the Ruby or npm ecosystems, or any of the things that awarded "badges" like Mars Rover badge or the Artic Vault badge). There's a page buried in your Personal Account Settings to turn off that ancient Dependabot option. (I'm on a work machine without access to my personal account at the moment or I'd directly tell you where to find it.)
I'm at a loss to explain that! My only other guess is that you might have enabled Dependabot at some point further back in history, when it was a third-party integration and directly owned by or integrated into GitHub.
Do you have a Dependbot entry in your account/org-level applications?
> Do you have a Dependbot entry in your account/org-level applications?
I don't think so. I have no memory of such a thing, and there is no org.
Okay, I have no idea then. I guess perhaps at one point Dependabot was enabled by default for some people, although that strikes me as a bad idea and I can only assume they've disabled it since then, since I haven't seen this on any new repository I've made.
I always check changelogs of the dependencies. I treat a dependant PR as seriously as any PR.
"Is backward compatibility even possible?" -> https://digitalseams.com/blog/is-backward-compatibility-even...
How do you scale this besides keeping the dep list short? Are you reading every item or just scanning for words like "deprecated" or "breaking change"?
It's not a huge part of the job to read every item. Looking at code changes in deps though is a whole other thing.
How do you prevent exposing yourself to supply chain attacks like the tj-actions/changed-files one [0] if you don't?
I get your question regarding scaling, but that's the job: you can choose to outsource code to 3rd-party libraries, and eternal vigilance is the trade-off.
Assume your 3rd-party dependencies will try to attack you at some point: they could be malicious; they could be hacked; they could be issued a secret court order; they could be corrupted; they could be beaten up until they pushed a change.
Unless you have some sort of contract or other legal protection and feel comfortable enforcing them, behave accordingly.
0: https://www.wiz.io/blog/github-action-tj-actions-changed-fil...
changelogs, but not the code?
That's a judgement call. It would be too much to review all code change of all dependencies unfortunately.
The corollary of reviewing all code on all dependency updates is you should review all code or the new deps you add, including the transformation by build processes that might mean what is in the package manager might be different and same for all transitive dependencies.
Same with the language and runtime tooling.
It is too hard to be perfect!
You run CI on your dependabot PRs, right? IF so, how is it any different than doing the same thing manually?
> Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does!
Still have flashbacks from that one time when some dependency in our Go project dropped support for go1.18 in its patch version update, and we almost couldn't rebuild the project before the Friday evening. Because obviously /s being literally unable to build the dependency is a backwards-compatible change.
> Also, is automated version bumps really such a good thing?
Depends. Do you want to persist the belief that software requires constant maintenance because it's constantly changing? Then yes: automate your version bumps and do it as often as possible.
If you want software to be stable then only update versions when you have a bug.
I'm going through SOC2 right now, and it is essentially requiring it to be enabled. I'll just do the minimal to pass, but it isn't something you can just disable in some cases.
FWIW: this might be a suggestion of the specific audit team you’re working with or a requirement of one of the “follow our playbook and you’ll pass” vendors if you’re using one of those, but the SOC 2 on its own doesn’t really impose specific technical feature/control requirements like this.
I don’t have the exact exam language in front of me right now but the requirement would be something like “you have some process for learning about, assessing, and mitigating vulnerabilities in software dependencies that you use”.
Enabling an automated scan and version bump tool like dependabot is a common and easy way to prove your organization has those capabilities. But you could implement whatever process you want here and prove that you do it on the schedule you say you do in order to satisfy the audit requirement.
True on all counts. But the lowest effort is "just turn off dependabot", which is what I suspect most of the people trying to get past SOC2 will do (like myself).
I have not seen any serious OSS project to auto-merge version bumps. Most of them have manual approval for it afaik
Curious if auto-merge philosophy changes between libraries and applications. The library definitely has a larger user base to break and a wider matrix of use-cases. IMO, auto-merge is more palatable for an application – do you agree? Especially when you're under SOC2/FedRAMP/etc.
Work has been fiddling around with Dependabot and force-enabling it everywhere (not auto-merging, just having it generating the PRs)... my feedback was that it is built on the presumption that All Updates Are Automatically Good, but this is transparently a false statement. In fact, Dependabot, by being so fast and automatic, may actually raise the probability of some project getting malicious code injected into it! Consider the timeline of malicious code injection:
The faster you act after step 1, the more chance you'll have time to put it into your system before the world reaches step 3. Dependabot maximizes the speed of reaction after step one. If I'm doing things somewhat more manually then I'm much more likely to experience the world finding out about a corrupted dependency before I start incorporating it.Now, just typing it out it may sound like I'm more freaked out than I actually am. While supply-chain attacks are a problem, they are getting worse, and they will continue to get worse, they are also still an exotic situation bubbling on the fringe of my awareness, as opposed to something I'm encountering regularly. For a reasonable project the most likely outcome is that dependabot enhancing this exposure window will still not have any actual real-world impact, and I'm aware of that. However, where this is relevant is if you are thinking of Dependabot and its workflow as a way of managing security risk, because you imagine updates as likely carrying security improvements, and that's your primary purpose for using it (as opposed to other uses, such as, your system slowly falling behind in dependencies until it calcifies and can't be updated without a huge degree of effort, a perfectly reasonable threat and Dependabot is a sensible response to that), then you also need to consider the ways in which is may actually increase your vulnerability to threats like supply-chain attacks.
And of course, projects do not start out with all their vulnerabilities on day one and then monotonically remove them. Many vulnerabilities are introduced later. For each such vulnerability, there is a first release that includes them, and for which treating that update as if it was just a Good Thing was in fact not true, and anyone who pushed it in as quickly as possible made a mistake. Unfortunately, sometimes hard problems are just hard problems.
Though I have wondered about the idea of programming something like Dependabot, but telling it, hey, tell me about known CVEs and security releases, but otherwise, let things cook for 6 months before automatically building a PR for me to update. That would radically reduce this risk I'm outlining here.
(In fact, after pondering, I'm kind of reminded of how Debian and a lot of Linux distros work, with their staged Cutting Edge versus Testing versus Stable versus Long Term Support. Dependabot sort of builds in the presumption that you want that Cutting Edge level of updates... but in many cases, no, I really don't. I'd much rather build with Stable or Long Term Support for a lot of things, and dip into the riskier end of the pool for specific things if I need to.)
Dependabot already differentiates between version updates and security updates:
https://docs.github.com/en/code-security/dependabot/dependab...
https://docs.github.com/en/code-security/dependabot/dependab...
I have only skimmed the docs and I wouldn't be completely shocked if there's a "wait X weeks/months to notify about non-security updates" but I don't know about it if it's there. If it is there, hey, great! Won't be the first time I really wished that X did Y and found out that yes it already does.
The folks at Synaktiv had a nice detailed blog post on this same vector last year[1].
The bottom line with these kinds of things is that virtually nobody should be using `pull_request_target`, even with “trusted” machine actors like Dependabot. It’s a pretty terrible footgun.
[1]: https://www.synacktiv.com/en/publications/github-actions-exp...
Fixed link: https://www.synacktiv.com/en/publications/github-actions-exp...
Thanks, I've fixed my comment as well.
What a weird and distracting AI-generated header image.
Yeah, why is he saying "confused deputy" as he pulls the lever. Sounds like he knows he shouldn't even be in charge!
If people are this obsessed with making the little version number of their dependencies as big as possible why don't they just make downloading the dependencies part of the build system?? Or better yet make the user supply their own dependencies and link them in as shared libraries? This problem was solved at least 30 years ago...
Wait, how is it possible for anyone who opens a PR to issue dependabot commands for main repository? There should be some kind of authorization in place to avoid it, right? Should it not ignore any commands coming from outside users who do not have commit access?
This is explained here:
> Here's the trick: github.actor does not always refer to the actual creator of the Pull Request. It's the user who caused the latest event that triggered the workflow.
its a fork
Seems like a bit forced scenario to me. I have never seen anyone auto-merge Dependabot PR automatically using GitHub Action.
Also pull_request_target is a big red flag in any GHA and even highlighted in GHA docs. It’s like running untrusted code with all your secrets handed over to it.
> I have never seen anyone auto-merge Dependabot PR automatically using GitHub Action.
For better or worse, it's a pattern that GitHub explicitly documents[1].
(An earlier version of this page also recommended `pull_request_target`, hence the long tail of public repositories that use it.)
[1]: https://docs.github.com/en/code-security/dependabot/working-...
> So, they created workflows to auto-merge PRs if the creator was Dependabot. Seems safe, doesn't it?
No? In what world would it be safe to merge code, AI-generated or not, which you haven't reviewed, much less do it automatically without you even knowing it happened?
How do you know that you need the changes (whether bug or CVE)? How do you know the code isn't malicious? How do you know your systems are compatible with the change? How do you know you won't need to perform manual work during the migration?
I think this response is understandable. But also feels disingenuous. If a dependency update appears to be an automated version bump, and you trust your test base, and you trust the dependency author (did you read the whole dependency's dependency graph before importing it? You probably didn't.) it's only a small step to auto-merge. The horror here is that the bot isn't the author and the version update isn't legitimate. That's on Github's presentation and settings, in my opinion.
Relying on a human reviewer, regardless, is a weak guarantee. If your security posture is "Joe shouldn't make mistakes", you still have a weak security posture.
> ... and you trust the dependency author ... it's only a small step to auto-merge
I disagree for the reasons listed above, but we're talking about 3rd-party dependencies here, not trusted ones. Given the numerous scenarios I listed above, it's a huge step from "using a 3rd party library" to "trusting the author of it".
We'd have to start with "...and you don't trust the author", because for most 3rd-party dependencies, the author has given us neither sufficient evidence to do so, nor sufficient recourse if that trust is violated.
> Relying on a human reviewer, regardless, is a weak guarantee.
Relying on countless random 3rd party to not own you when they know people are pulling in their code as a dependency is a far, far worse guarantee. How would that strategy have protected someone against this supply-chain attack?:
https://www.wiz.io/blog/github-action-tj-actions-changed-fil...