An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
Then again, I once submitted a bug report to my bank, because the login method could be switched from password+pin to pin only, when not logged in, and they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password. (And that's not even getting into the difference between real two-factor authentication the some-factor one-and-a-half-times they had implemented by adding a PIN to a password login.) I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
Assuming the host of the bug bounty program is operating in good faith, adding some kind of barrier to entry or punishment for untested entries will weed out submitters acting in bad faith.
Bug bounties often involve a lot of risk for submitters. Often the person reading the report doesn't know that much and misinterprets it. Often rules are unclear about what sort of reports are wanted. A pay to enter would increase that risk.
Honestly bug bounties are kind of miserable for both sides. I've worked on the recieving side of bug bounty programs. You wouldnt believe the shit that is submitted. This was before AI and it was significant work to sort through, i can only imagine what its like now. On the other hand for a submitter, you are essentially working on spec with no garuntee your work is going to be evaluated fairly. Even if it is, you are rolling the dice that your report is not a duplicate of an issue reported 10 years ago that the company just doesn't feel like fixing.
Pay to enter would increase the risk of submitting a bug report.
However, if the submission fees were added to the bounty payable, then the risk reward changes in favour of the submitter of genuine bugs.
You could even have refund the submission fee in the case of a good faith non bug submission.
A little game theory can go a long way in improving the bug bounty system...
They could allow submitters to double down on submissions escalating the bug to more skilled and experienced code reviewers who get a cut of the doubled submission fee for reviews.
Indeed, increasing the incentive for companies to reject ( and then sometimes silently fix anyway ) even the valid reports would only increase further misery for everyone.
Indeed. I've also gotten a lot of "Hey I found a super critical security bug that makes your system ass, but if you pay me first I'll tell you what it is" types of submissions. Sometimes it's a hybrid, like something semi-legitimate but weak that they disclose up front, closing with a "and I've got another one that's juicy but you have to pay me first to hear it".
Oh and then of course there is the flood of people who just scanned our infra with Nessus and screenshotted part of the report (often with important details blacked out so we can't see them unless we pay).
As someone who has been on both sides of it as well, it just feels like everything is terrible
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
Sadly, yeah. And will do anything only if they believe they can actually be caught.
An EU-wide bank I used to be customer of until recently, supported login with Qualified Electronic Signatures, but only if your dongle supports... SHA-1. Mine didn't. It's been deprecated at least a decade ago.
A government-certified identity provider made software that supposedly allowed you to have multiple such electronic signatures plugged in, presenting them in a list, but if one of them happened to be a YubiKey... crash. YubiKey conforms to the same standard as the PIV modules they sold, but the developers made some assumptions beyond the standard. I just wanted their software not to crash while my YubiKey is plugged in. I reported it, and they replied that it's not their problem.
A problem with this approach is that one of the key functions of a bug bounty program is to encourage people to report vulnerabilities to the developers, rather than selling them elsewhere.
If I have to pay money to submit a vulnerability to the developers with no guarantee that I'll even get refunded for a high quality and good faith report, let alone any actual payout, there's much less incentive for me to do so compared to selling them to someone else who won't charge me money for the privilege.
In a past life I was deeply involved in the operation of a bug bounty program. Discouraging people from selling on the black market was nowhere on the list of motivations.
We wanted to encourage white hat security researchers to look at our domain rather than other domains so we could collect more data on the kinds of vulns that appeared in our domain to help prioritize efforts that would fix the root causes of recurring bug patterns.
I've also submitted bug bounties and received rewards and I've worked with a bunch of other people who have done this. At no point did I even consider selling on the black market and I suspect that my friends from grad school were the same way.
Maybe the $1,000,000 bounties for zero click rce on iphones or whatever exist to discourage selling on the black market, but I'm not even sure that is true. "Well, I'll just find a way to sell this to the russian mob" is not exactly something that is on the radar of the vast majority of security researchers.
The reality is that most people's thoughts on bug bounties are from salacious headlines talking about those $1M vulnerabilities. In reality the average bug bounty submission is a machine translated report for a low severity issue in a web app that may or may not even exist (or be a vulnerability), sprayed at hundreds of companies (or the same company a hundred times) in the hopes of earning $500 to basically do currency manipulation.
There are plenty of places you can sell exploits other than OCGs. At the more legitimate end of that market is people like ZDI who will then collaborate with the vendors (after a time), or companies making exploit kits/tooling for pentesters/red teaming. More questionable ones are companies that make things like forensics tools or spyware who are legal, but perhaps ethically dubious. All completely legal, but not great for the wider community if they're getting the vulns rather than the developers.
If you're trying to protect your own website and servers, those markets won't be a concern for you. If you ship a widely used product that's an attractive target (like web browser, mobile device, network kit, etc) then they definitely are.
You don't sell it to the "Russian mob", you sell it to a highly reputable security company that will buy it for like $10 million or more and sell it to governments and stuff, not the mob.
I mean, seriously.
Why would I ever go find a 0 click rce bug and then just donate it to a trillion dollar company just to get a "thx" when I can just retire right then and there?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
That adds an extra layer of complexity to the cURL maintainers, handling other people’s money and whatnot. It was considered.
Daniel (cURL’s lead) has been discussing this for months. Whatever “quick and easy” solution you think of, it’s already been suggested and thought about and rejected for some reason.
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
I personally came to that conclusion thanks to the GrapheneOS situation regarding device attestation. Insecure devices get full features from some apps because they are certified, although they cite security, while GrapheneOS get half featured apps because it's "insecure" (read, doesn't have the Google certification, but are actually the most secure devices you can get, worldwide)
It's related, but GP is still right to bring it up - it's the one question that is most important wrt. security, and also conveniently the least often asked: security for who, and from what? "Security" isn't an absolute good.
Play Integrity certifies (and banks/etc approve) that Android 8.0 (oreo) unpatched for several years, full of vulnerabilities for RCE, 0-click, privilege escalation, etc, so full of holes it's trivial to get a root and then hide it (or use leaked cert), is absolutely a-ok, safe to use and secure for user.
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
I refer to this as the Notion-to-Confluence cost border.
When Notion first came out, it was snappy and easy to use. Creating a page being essentially free of effort, you very quickly had thousands of them, mostly useless.
Confluence, at least in west EU, is offensively slow. The thought of adding a page is sufficiently demoralizing that it's easier to update an existing page and save yourself minutes of request time outs. Consequently, there's some ~20 pages even in large companies.
I'm not saying that sleep(15 * SECOND) is the way to counter, but once something becomes very easy to do at scale, it explodes to the point where the original utility is now lost in a sea of noise.
It’s strange how sensitive humans are to these sort of relative perceived efforts. Having a charged, cordless vacuum cleaner ready to go and take around the house has also changed our vacuuming game. Because carrying a big unwieldy vacuum cleaner and needing to find a power socket at every location just feels like much more effort. Even though it really isn't.
It is. The classical vacuum is heavier, you have to find the socket and plug it in (non-trivial if you have few of them, or have kids and sockets have kid blocks on them), and perhaps most importantly, you need two free hands to operate it (particularly when carrying, plugging in and repositioning). That alone is enough to turn it into a primary activity, i.e. the kind of thing that you explicitly decide to do and becomes your main focus. Meanwhile, a charged cordless cleaner is something you can semi-consciously grab with one hand while passing by, and use on the go to do some cleaning while actually focused on something else. It's an entirely different class of activity, much easier to fit in during the day.
Ironically, the cordless vacuum is even better than vacuum robots in this regard! I was surprised to hear from some friends and acquaintances that they prefer the manual vacuum to robotic one, and find it a better time/effort saver - but I eventually realized they're right, simply because the apps for controling the robotic vacuums are all steaming piles of shit, and their bad UI alone turns activating the robot into primary activity. It may be a brief activity, but it still requires full focus.
Have you tried a handheld corded unit? I find most of the perceived effort with the larger unit is due to a combination of the bulk, the weight, and having to maneuver it around anything delicate without inadvertently bumping into it. Meanwhile the corded handheld never dies on me and I think isn't as bad for my hearing. All the cordless units I've tried seem to have an extremely high pitch whine to them.
I keep having to get it from progressively more inconvenient locations to which it has been banished in order to humor my wife’s delusion that the roomba or the handhold do anything.
I can make multiple passes with the handheld to get 80% of the crumbs in a small area, troubleshoot why the robot didn’t run yesterday in order to hope it will get the crumbs tomorrow, or just get the corded vacuum out and actually clean a whole room.
Work involves cables. Any product that promises something different is a lie.
Our cordless, on the highest suction setting, is bordering on unusable. The effort to move it across carpet becomes quite high. Trying to roll it on an area rug tends to cause it to drag the rug around, and if you pick it up while on it will pull the rug up off the floor.
I have done some _very_ scientific testing here, vacuuming a section of carpet on the lowest section (doing lines where each pass half-overlapped the previous so each part of the carpet got touched once in each direction), emptying the vacuum, then going back over doing the same on high. Didn't see anything else come up. Shop vac didn't pull anything else out either that I could see.
I used to be in a similar boat of "these are a stupid class of product", but end of the day even if it takes eight passes my wife was going to use it anyway. The effort for her to set the time aside to drag around the heavier corded vacuum which is a substantial effort for her, etc, would be more than doing eight passes with a cordless. So got a good one and I'm sold on it now--it is quite convenient, and it does work.
Only thing I will say is the battery definitely can't do an entire carpeted house on a charge. We don't have that much carpet, so don't have any problem cleaning all the floors and a couple area and entry-way rugs on a charge.
This is an interesting discussion to me - I have a cordless vacuum that works well and a roborock combo vac/mop that works well. Actually, I'm lying, I have two cordless vacuums because the GGP's observation rings true to me and I got a second one for free and held on to it. :-)
Dyson cordless vac, older (v8 ultimate). Have had to replace battery once and broken trigger. Continues to be a workhorse.
Roborock s5v: I have it run 2x / day on weekdays, once in the morning after breakfast when we're taking the kids to school (vac kitchen only), and once after bedtime (vac + mop entire area). It does a great job of generally keeping things clean. Not perfect, but the overall dirt level stays low.
The cordless manual vac is really useful for "oh bleep, 8yo just spilled MORE stuff on the ground". I keep it next to the dining and kitchen area. It's not super aesthetic having it hanging on the wall in a visible location but I have engineer-itis and I value the convenience over the illusion that we don't own a vacuum. :) I approximately never use the robovac as an on-demand vacuum unless it's to run an extra pass when we're leaving home on a weekend and have left crumbs from a meal.
For us, substantially upping the frequency of vacuuming, even if it's not quite as deep, has made a big difference, and it's basically no extra burden to have the robovac run frequently after programming it.
Well... I have a (1000 dollar!!!) Dyson cordless vacuum, arguable the laser + the histogram for "removed particles of size X", make me clean more thoroughly. The things is pretty heavy and applies a pretty good vacuum, imho.
Bro the vacuum community is audiophile-level picky. I have a Dyson stick vacuum of some sort and I haven’t had any issue with picking up crumbs. I would rather manually bend over and pick up something it doesn’t grab than move around the heavy corded vacuum and plug it in 10 times.
The term I know / used for this is "trivial inconveniences", via an old article of Scott Alexander[0].
The quote from example from early in the article stuck with me for years:
Think about this for a second. The human longing for freedom of information is a terrible and wonderful thing. It delineates a pivotal difference between mental emancipation and slavery. It has launched protests, rebellions, and revolutions. Thousands have devoted their lives to it, thousands of others have even died for it. And it can be stopped dead in its tracks by requiring people to search for "how to set up proxy" before viewing their anti-government website.
(Now this is more poetic, but I suppose the much more insightful example that also stuck with me is given later - companies enticing you to buy by offering free money, knowing well that most customers can't be arsed to fill out a form to actually get that money.)
I suspect that applies specifically to their cloud rewrite which was apparently a bloat of JS libs and hundreds of requests even by Atlassian standards. The on-prem self-host Confluence I've used is still pretty snappy and pleasant to use and without throwing an absurd amount of resources at it. We do have quite a lot of actually-useful documentation in it.
That said, Atlassian is busy relentlessly raising the price for self-host to push people into their cloud roach motel, so we'll probably be on some alternative (either FOSS or commercial, but self-host) soon too.
I find this to be a very amusing critique. In my experience, Notion (when I stopped using it 3 years ago) was slow as molasses. Slow to load, slow to update. In comparison, at work, I almost exclusively favor Confluence Cloud. It's very responsive for me.
We have tons of Confluence wikis, updated frequently.
I think it might be the same issue as with WordPress and Jira - terrible plugins. Each company uses own special mix, and encounters issues often occurring in that one specific configuration. And it is the base platform that takes the blame.
In particular a place I used to work had a plugin for threaded comments in Jira. The specific one we were using slowed things down noticeably with the DB on the same server, but not too much to be an improvement in overall usefulness.
Then we decided trying to make our Jira more reliable by splitting the DB out into a separate clustered DB system in the same data center. The latency difference going through a couple of switches and to another system really added up with those extra 1600 or so DB calls per page load.
We ended up doing an emergency reversion to an on-host DB. Later, we figured out what was causing that many queries.
You're referring to the on-prem Jira. That might suck, sure. My experience has been purely using Jira Cloud and Confluence Cloud, both of which I've found to be snappy and responsive.
My last company switched several teams to Jira Cloud. My current company started with Cloud when we moved over from other tools.
Cloud does not give you the flexibility of your own plugins, your own redundancy design, or your own server upgrades. On top of that, the performance is pretty variable and is far worse than a self-hosted Jira on fast hardware.
It’s interesting to me that your lack of experience to make a comparison qualifies you in some way to criticize the experience I actually have.
Amusingly, exactly opposite experience here. That said, our on-prem is jira and confluence integrated with db on same machine, and apache in front doing additional caching. I imagine like so many things it is how you set it up...
If you read my previous comment, I said it was largely the specific poor plugin that caused most of the performance issue with the database queries. I never complained about the overall speed of on-prem Jira. That was the assertion of the person who’s only ever used the cloud version.
And the end of their self hosting offerings (Server, Data Center), which is currently driving a lot of people towards XWiki, for other reasons than money. XWiki SAS being mainly in Europe makes it attractive to EU users too.
> do you have any migration tips?
I don't have specific migration tips. I hope the docs are complete enough!
The Confluence Migration Toolkit is based on the Confluence XML module you found, but it adds a nice and convenient UI, converts some more macros that XWiki SAS sells, there's support, and there's consulting for larger migration projects or projects with special requirements.
(note: despite some paying features, everything is open source)
(disclaimer in case it was not obvious, I work for XWiki SAS)
I found that banks are one of the worst organizations when it comes to authentication. They are regulated but the requirements are completely outdated and irrelevant in a risk context.
And then you have banks such as Boursobank (a French online bank) that has weak traditional authentication (and a faulty app, but they do not care) and out of the blue also provides passkeys. Making it at the same time horribly bad and wonderfully good.
The worst part is that they hide behind regulations when in fact there are only few of them.
Other instiytutions such as SWIFT are as bad and equally arrogant.
For weak bank logins, my guess is that reimbursing all account takeovers is cheaper than having a complex login process that would scare away non-technical customers. Or, well, I could see myself making that decision if I were more versed in finance than in computer science and I had a reasonable risk assessment in front of me to tell me how many account takeovers happen.
Banks aren't even liable for losses from account takeovers, at least if their system is compliant, regardless of whether that makes it secure. Their biggest incentive is customer satisfaction, which fraud does hurt.
It's credit cards that have to reimburse for fraud, but they charge the merchant for it, plus fees, so they have absolutely no incentive to prevent fraud, if not an incentive to outright encourage fraud. That would explain why their implementation of the already compromised EMV was further nerfed by a lack of a PIN in the US.
> Their biggest incentive is customer satisfaction
At a bank? No way. They are some of the most customer-hostile organizations I've interacted with. Dealing with payment accounts is a necessary evil for them, and they are very much aware of the effort required to switch to a different bank, and of the massive regulatory moat preventing consumer-friendly competition from popping up.
A bank doesn't care about screwing over a handful of customers. As long as it's not common enough to draw the attention of the press and/or a regulatory agency, they are not going to spend any money on improving.
Case in point: Wells Fargo foreclosure fraud.
Case in point: Wells Fargo opening new accounts in customer names without direction from, approval by, or notification to said customers.
The primary incentive of a bank is to make money rather than customer satisfaction, security, or most other things. Sometimes other priorities suffer in the race to profit, sometimes including regulatory compliance and legality.
That anecdote is hilarious and scary in equal measures. Optional passwords are certainly more convenient than required ones, but so are optional PINs. The most convenient UX would be never needing to log in at all! Unless you find it inconvenient for others to have access to your bank account of course
That's what eBay does to me. You get to choose, at the time of login, between entering a password and getting an email verification, or just getting an email verification. At least with the bug report I had submitted to my bank, the password requirement had to be disabled from inside a settings menu, instead of being a clear option in the login prompt, but it that case it wasn't even a 2nd factor.
Long long ago the google toolbar queries could be reverse engineered to do an i feel lucky search on gmail. I created a login that (if @gmail.com) forwarded to the specific mail.
Unlikely to happen but it seems fun to extend email [clients] with uri's. It is just a document browser, who cares how they are delivered.
I hate this as well, especially since I have greylisting enabled on some email addresses, so by the time the email login is delivered, the login session has already timed out and of course the sender uses different mail servers everytime. So in some cases, it's nearly impossible to login and takes minutes...
It's the same on the sender side. Most people of course just outsource it to some SaaS like Sendgrid, and of course have some fancy microservice event bus architecture to get it there. That 'your login email has been sent' actually means 'your email has entered the very first queue, and we're hoping it makes it through all the layers soon'.
There have been plenty of instances where I tried to log in somewhere, and the first attempt to contact my mail server was twenty minutes later. And of course they then deliver all five retries at once.
cURL would operate such a program in good faith, and quickly earn the trust of the people who submit the kind of bug reports cURL values.
Your bank would not. Nor would mine, or most retail banks.
If the upfront cost would genuinely put off potential submitters, a cottage industry would spring up of hackers who would front you the money in return for a cut if your bug looked good. If that seems gross, it's really not - they end up doing bug triage for the project, which is something any software company would be happy to pay people for.
There's also the issue of what happens to my money as a researcher. Is it paid to the company, or is someone holding it in escrow? What if it takes the developer months to respond, or they never do? Do they just get to keep my money indefinitely? What if the vendor pulls out of the scheme? What if I do a chargeback on the payment I made? Etc, etc
I wonder if a better model would be to make the platform pay to entry, but not the specific bugs? So you have to pay a fee to gain access to a platform like HackerOne, and if your signal:noise ratio gets too bad then your account gets revoked? That would make it feel like less of a gamble than having to pay for every individual bug - but still has the same problem that it's putting a big barrier in front of legitimate good-faith researchers.
I've been active in the bug bounty community for almost 7 years now. The problem is that the majority of companies don't act in good faith.
Even when you have something fully exploitable and valid, they will many times find some way to not pay you or lower the severity to pay you very little.
The catch-all excuse is something along the lines: "although this is vulnerable, it doesn't impact the business".
I've gotten this excuse, even when I could prove it was a production server with customer information that I could access.
Sites like Hackerone can help, but in the end, it comes down to the company running the bug bounty program.
Agreed, although the reimbursement should be based on whether a reasonable person could consider that to be a vulnerability. Often it’s tricky for outsiders to tell whether a behaviour is expected or a vulnerability
Are bug reports a 100% sure black and white thing?
Could people who think they found a bug but not sure be turned off by the up front cost / risk of finding out they are wrong or not technically finding a bug?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
It would also stop a lot of genuine submissions unfortunately, as some literally can't pay not just won't pay (for both technical or financial reasons), and adds complexity¹. Each project working this way will need to process a bunch of payments and refunds on top of the actual bounty payments, which is not admin free nor potential financially cost free.
I can't think of an easy answer that would work for more than a very short amount of time. As soon as there is money involved and an easy way to use tooling rather than actual effort/understanding to be involved, many will try to game the system ruining it for those genuine participants. Heck, even if the reward is just credit² rather than money, that will happen. Many individual people are honest and useful, people as a whole are a bunch of untrustworthy arseholes who will innocence you and the rest of the world for a penny or just for shits & giggles.
> Assuming the host of the bug bounty program is operating in good faith
This is a significant assumption. One that is it harder to not be paranoid about when you are putting money down.
> they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password
This does not surprise me. My primary bank (FirstDirect, UK) switched the way I authenticate from “between 5 and 9 alphanumeric characters”³ to a 5-digit pin, and all their messages about it assured me (like hell!) that this was “just as secure as before”…⁴
--------
[1] Needing a payment processing option that is compatible with both the reporter and reportee, at the point of submission. At the moment that can be arranged after the bounty is awarded rather than something a project like curl needs to have internationally setup and supported before accepting submissions.
[2] ref: people submitting several simple documentation fixes, one misplaced comma or 'postrophe per pull request, to game some “pull requests accepted” metric somewhere.
[3] which wasn't ideal to start with
[4] I would accept the description “no less secure than before” if they admitted that the previous auth requirements were also lax.
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
The problem is that bug bounty slop works. A lot of companies with second-tier bug bounties outsource triage to contractors (there's an entire industry built around that). If a report looks plausible, the contractor files a bug. The engineers who receive the report are often not qualified to debate exploitability, so they just make the suggested fix and move on. The reporter gets credit or a token payout. Everyone is happy.
Unless you have a top-notch security team with a lot of time on their hands, pushing back is not in your interest. If you keep getting into fights with reporters, you'll eventually get it wrong and you're gonna get derided on HN and get headlines about how you don't take security seriously.
In this model, it doesn't matter if you require a deposit, because on average, bogus reports still pay off. You also create an interesting problem that a sketchy vendor can hold the reporter's money hostage if the reporter doesn't agree to unreasonable terms.
I don’t think it works for curl though. You would guess that sloperators would figure out that their reports aren’t going through with curl specifically (because, well, people are actually looking into them and can call bullshit), and move on.
For some reason they either didn’t notice (e.g. there’s just too many people trying to get in on it), or did notice, but decided they don’t care. Deposit should help here: companies probably will not do it, so when you see a project requires a deposit, you’ll probably stop and think about it.
Triage gets outsourced because the quality of reports is low.
If filing a bad report costs money, low quality reports go down. Meanwhile anyone still doing it is funding your top notch security team because then they can thoroughly investigate the report and if it turns out to be nothing then the reporter ends up paying them for their time.
My point is that on average, filing bad but plausibly-sounding reports makes the reporter money. Curl is the odd exception with naming-and-shaming, not the rule. Spamming H1 with AI-generated reports is lucrative. A modest deposit is unlikely to change that. A big deposit (thousands of dollars) would, but it would also discourage a lot of legitimate reports.
Github can use youtube strikes like system. PRs are tied to people. Someone reported for submitting slop should get a badge or something similar.
If a PR is submitted by someone who is then known to submit slops, they can be easily ignored by the maintainers.
EDIT: Or may be something like SponsorBlock for youtube. There could be a browser extension that will collectively tag sloppers the sameway and can help identify sloppers.
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
This is the key insight. Nobody cares at all about actual security. It is all about checklists and compliance.
It seems open source loses the most from AI. Open source code trained the models, the models are being used to spam open source projects anywhere there's incentive, they can be used to chip away at open source business models by implementing paid features and providing the support, and eventually perhaps AI simply replaces most open source code
Extending on the same line, we will see programs like Google Summer of Code (GSoC) getting a massive revamp, or they will stop operating.
From my failed attempt, I remember that
- Students had to find a project matching their interests/skills and start contributing early.
- We used to talk about staying away from some projects with a low supply of students applying (or lurking in the GitHub/BitBucket issues) because of the complexity required for the projects.
Both of these acted as a creative filter for projects and landed them good students/contributors, but it completely goes away with AI being able to do that at scale.
GSoC 4 years ago removed the need for their to be actual students to apply. We got flooded with middle aged men working 9-5s applying. It was dumb and we stopped participating. Their incentives were literally "extra income" instead of learning or participating beyond that.
> they can be used to chip away at open source business models by implementing paid features and providing the support
There are a lot of things to be sad about AI, but this is not it. Nobody has a right to a business model, especially one that assumes nobody will compete with you. If your business model relies on the rest of the world bring sucky so you can sell some value-added to open-core software, i'm happy when it fails.
When LLMs are based on stolen work and violate GPL terms, which should be already illegal, it's very much okay to be furious about the fact that they additionally ruin respective business models of open source, thanks to which they are possible in the guest place.
> the fact that they additionally ruin respective business models of open source
The what now? Open source doesn't have a business model, it's all about the licensing.
FOSS is about making code available to others, for any purpose, and that still works the same as 20 years ago when I got started. Some seem to wake up to what "for any purpose" actually mean, but for many of us that's quite the point, that we don't make choices for others.
If something is not technically illegal that does not mean it cannot be bad.
Like I said, there is a part that should be illegal, and then part where that's used to additionally harm one of the ways that OSS can be sustainable. The second part on its own is not illegal but adds to damages and is perfectly okay to condemn.
Open source software can have business models, it's one of the ways it can be sustainable. It can work like, for example, the code is made available (for any purpose) and the core maintainer company provides services, like with Nginx (BSD). Or there is an open-source software, and companies create paid products and services on top while respecting the terms of that software and contributing back, like with Linux (GPL) and SUSE/Red Hat.
> If something is not technically illegal that does not mean it cannot be bad.
Ok? I agree, but unsure what exactly that's relevant to here in our discussion.
> Open source software can have business models
I believe "businesses" are the ones who have "business models", and some of those chose to use open source as part of their business model. But "open source" the ecosystem has nothing to do with that, it's for-profit companies trying to use and leverage open source, rather than the open source community suddenly wanting to do something completely different from what it's been doing since inception.
> unsure what exactly that's relevant to here in our discussion.
I'll remind then. Our discussion follows the top statement "It seems open source loses the most from AI". As far as I understand nobody narrowed the context to "what is currently legal". Something can be technically legal and still harmful to open source. Also, laws are never perfect and sometimes they need to be updated.
(For example, I know that a number of people would say US abducting and detaining citizens and brutally deporting immigrants is not illegal, but if it's technically legal does that make it OK?)
> what it's been doing since inception.
At inception open source was mostly personal side projects for funsies (like Linux) sponsored by maintainer having a dayjob. The big leap happened when copyleft licenses made it such that success of a big commercial company building products on open-source projects would directly improve these open-source projects. And it's nothing new, it happened long time ago. The desire for volunteer contributions to codebase to remain for public benefit in perpetuity is exactly the point of strong copyleft, and it's exactly what's being circumvented by LLM washing. The fact that these LLMs subsequently also harm open source communities adds insult to injury.
>“Free software” means software that respects users' freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software.
> Being able to learn from the code is a core part of the ideology embedded into the GPL.
I have to imagine this ideology was developed with humans in mind.
> but LLMs learning from code is fair use
If by “fair use” you mean the legal term of art, that question is still very much up in the air. If by “fair use” you mean “I think it is fair” then sure, that’s an opinion you’re entitled to have.
> I have to imagine this ideology was developed with humans in mind.
Actually, you don't have to. You just want to.
N=1 but to me, LLMs are a perfect example of where the "ideology embedded into the GPL" benefits the world.
The point of Free Software isn't for developers to sort-of-but-not-quite give away the code. The point of Free Software is to promote self-sufficient communities.
GPL through its clauses, particularly the viral/forced reciprocity ones, prevents software itself from becoming an asset that can be rented, but it doesn't prevent business around software. RMS/FSF didn't make the common (among fans of OSS and Free Software) but dumb assumption that everyone wants or should be a developer - the license is structured to allow anyone to learn from and modify software, including paying a specialist to do it for them. Small-scale specialization and local markets are key for robust and healthy communities, and this is what Free Software ultimately encourages.
LLMs becoming a cheap tool for modifying or writing software, even by non-specialists (or at least people who aren't domain experts), furthers those same goals, by increasing individual and communal self-sufficiency and self-reliance.
(INB4: The fact that good LLMs are themselves owned by some multinational corps is irrelevant - much in the same way as cars are important tool for personal and communal self-sufficiently, despite being designed and manufactured by few large corporations. They're still tools ~anyone can use.)
Something can be illegal and it can be technically legal but at the same time pretty damn bad. There is the spirit and the letter of the law. They can never be in perfect agreement because as time goes bad guys tend to find new workarounds.
So either the community behaves, or the letter becomes more and more complicated trying to be more specific about what should be illegal. Now that GPL is trivially washed by asking a black box trained on GPLed code to reproduce the same thing it might be inevitable, I suppose.
> They're still tools ~anyone can use
Of course, technology itself is not evil, just like crypto or nuclear fission. In this case when we are discussing harm we are almost always talking about commercial LLM operators. However, when the technology is mostly represented by that, it doesn't seem required to add a caveat every time LLMs are mentioned.
There's hardly a good, truly fully open LLM that one can actually run on own hardware. Part of the reason is that hardly anyone, in the grand scheme of things, even has the hardware required.
(Even if someone is a techie and has the money and knows how to set up a rig, which is almost nobody on grand scale of the things, now big LLM operators make sure there are no chips left for them.)
So you can buy and own (and sell) a car, but ~anyone cannot buy and run an independent LLM (and obviously not train one). ~everyone ends up using a commercial LLM powered by some megacorp's infinite compute and scraping resources and paying that megacorp one way or another, ultimately helping them do more of the stuff that they do, like harming OSS.
LLMs spitting out GPL code seems perfectly inline with the spirit to me. The goal is to make it so that users have the freedom to make software behave in ways that suit them. Things kicked off when some printer could not be made to work correctly because of its proprietary drivers. LLMs are a huge multiplier for that: now even people who don't know how to program can customize their software! We're already approaching (or at?) the point where local agents on commodity hardware (like a few $thousand worth of GPUs, which was the nominal cost of a 90s PC) are able to make whatever changes you want given the correct feedback loops. Sounds good to me.
> The point of Free Software isn't for developers to sort-of-but-not-quite give away the code. The point of Free Software is to promote self-sufficient communities.
… that are all reliant on gatekeepers, who also decide the model ethics unilaterally, among other things.
> (INB4: The fact that good LLMs are themselves owned by some multinational corps is irrelevant - much in the same way as cars are important tool for personal and communal self-sufficiently, despite being designed and manufactured by few large corporations. They're still tools ~anyone can use.)
You’re not wrong. But wouldn’t the spirit of Free Software also apply to model weights? Or do the large corps get a pass?
FWIW I don’t have a problem with LLMs per se. Just models that are either proprietary or effectively proprietary. Oligarchy ain’t freedom :)
> > Actually, you don't have to. You just want to.
> Fair.
I don't think it's fair. That ideology was unquestionably developed with humans in mind. It happened in the 80s, and back then I don't think anyone had a crazy idea that software can think for itself and so terms "use" and "learn" can apply to it. (I mean, it's a crazy idea still, but unfortunately not to everyone.)
One can suggest that free software ideology should be expanded to include software itself in the beneficiaries of the license, not just human society. That's a big call and needs a lot of proof that software can decide things on its own, and not just do what humans tell it.
> It happened in the 80s, and back then I don't think anyone had a crazy idea that software can think for itself and so terms "use" and "learn" can apply to it. (I mean, it's a crazy idea still, but unfortunately not to everyone.)
Sure they did. It was the golden age of Science Fiction, and let's just say that the stereotype of programmers and hackers being nerds with sci-fi obsession actually had a good basis in reality.
Also those ideas aren't crazy, they're obvious, and have already been obvious back then.
> It was the golden age of Science Fiction, and let's just say that the stereotype of programmers and hackers being nerds with sci-fi obsession actually had a good basis in reality.
At worst you are trying to disparage the entire idea of open source by painting the people who championed it as idiots who cannot tell fiction from reality. At best you are making a fool of yourself. If you say that free software philosophy means "also, potential sentient software that may become a reality in 100 years" everywhere it mentions "users" and "people" you better quote some sources.
> Also those ideas aren't crazy, they're obvious, and have already been obvious back then.
Fire-breathing dragons. Little green extraterrestrial humanoids. Telepathy. All of these ideas are obvious, and have been obvious for ages. None of these things exist. Sorry to break it to you, but even if an idea is obvious it doesn't make it real.
(I'll skip over the part where if you really think chatbots are sentient like humans then you might be defending an industry that is built on mass-scale abuse of sentient beings.)
1. It's decided by courts in US. Courts in US currently are very friendly to big tech. At this point if they deny this and say something that undermines this industry it's going to be a big economic blow, the country is way over-invested in this tech and its infrastructure.
2. "Transformative means fair" is the old idea from pre-LLM world. That's a different world. Now those IP laws are obsolete and need to be significantly updated.
Last time I checked, there are still undecided cases wrt fair use. Sure, it’s looking favorable for LLM training, but it’s definitely still up in the air.
> it’s completely transformative
IANAL, but apparently hinges on how the training material is acquired
> IANAL, but apparently hinges on how the training material is acquired
That doesn't make sense. You are either transforming something or you are not. There might be other legal considerations based on how you acquired, but it doesn't affect if something is transformative.
So there are mixed messages, per my understanding. Kadrey v Meta seems to favor the transformative nature. Bartz v Anthropic went to summary judgement but the court expressed skepticism that the use in that case was “transformative”. We won’t know because of the settlement.
Again, IANAL, so take this with a big grain of salt.
In the first sentence "you" actually refers to you, a person, in the second you're intentionally cheating and applying it to a machine doing a mechanical transformation. One so mechanical that different LLMs trained on the same material would have output that closely resembles each other.
The only indispensable part is the resource you're pirating. A resource that was given to you under the most generous of terms, which you ignored and decided to be guided by a purpose that you've assigned to those terms that embodies an intention that has been specifically denied. You do this because it allows you to do what you want to do. It's motivated "reasoning."
Without this "FOSS is for learning" thing you think overrules the license, you are no more justified in training off of it without complying with the terms than training on pirated Microsoft code without complying with their terms. People who work at Microsoft learn on Microsoft code, too, but you don't feel entitled to that.
I'm not sure it's always bad intent. People often don't get that "machine learning" is a compound industrial term where "learning" is not literally "learning" just like "machine" is not literally "machine".
So it's sort of sentient when it comes to training and generating derivative works but when you ask "if it's actually sentient then are you in the business of abusing sentient beings?" then it's just a tool.
I think LLMs could provide attribution. Either running a second hidden prompt (like, who said this?) or by doing reverse query on the training dataset. Say if they do it with even 98% accuracy it would probably be good enough. Especially for bits of info where there's very few or even just one source.
Of course it would be more expensive to get them to do it.
But if it was required to provide attribution with some % accuracy, plus we identified and addressed other problems like GPL washing/piracy of our intellectual property/people going insane with chatbots/opinion manipulation and hidden advertisement, then at some point commercial LLMs could become actually not bad for us.
Competition is extremely important yes. But not the kind of competition, backed by companies that have much bigger monetary assets, to overwhelm projects based on community effort just to trample it down. The FFMPEG Google stuff as an example.
I wouldn’t see it as having a “right” to a business model but more like an accelerated tragedy of the commons. LLMs can’t reason but they can chip away at the easiest parts of the job, which is great initially if you can take advantage of that but it means fewer people will put free things in the commons or develop the skills needed to do what LLMs fail at. This feels like the way bars changed their “free lunch” specials a century ago to prevent people from costing them money: nobody has a right to it, etc. but the free loader problem leads to something many people like going away.
I wouldn't say open source code solely trained the models, surely there are CS courses and textbooks, official documentation as well as transcripts of talks and courses all factor in as well.
On another note, regarding AI replacing most open source code. I forget what tool it was, but I had a need for a very niche way of accessing an old Android device it was rooted, but if I used something like Disk Drill it would eventually crap out empty files. So I found a GUI someone made, and started asking Claude to add things I needed for it to a) let me preview directories it was seeing and b) let me sudo up, and let me download with a reasonable delay (1s I think) which basically worked, I never had issues again, it was a little slow to recover old photos, but oh well.
I debated pushing the code changes back into github, it works as expected, but it drifted from the maintainers own goals I'm sure.
I feel AI will have the same effect degrading Internet as social media did. This flood of dumb PRs, issues is one symptom of it. Other is AI accelerating the trend which TikTok started—short, shallow, low-effort content.
It's a shame since this technology is brilliant. But every tech company has drank the “AI is the future” Kool-aid, which means no one has incentive to seriously push back against the flood of low-effort, AI-generated slop. So, it's going to be race to the bottom for a while.
I think "internet" needs a shared reputation & identity layer - i.e. if somebody offers a comment/review/contribution/etc, it should be easy to check - what else are their contributing, who can vouch for them, etc.
Most of innovation came from web startups who are just not interest in "shared" anything: they want to be a monopoly, "own" users, etc. So this area has been neglected, and then people got used to status quo.
PGP / GPG used to have web-of-trust but that sort of just died.
People either need to resurrect WoT updated for modern era, or just accept the fact that everything is spammed into smithereens. Blaming AI and social media does not help.
It'll stop soonish. The industry is now financed by debt rather than monetary assets that actually exist. Tons of companies see zero gain from AI as its reported repeatedly here on HN. So all the LLM vendors will eventually have to enshittify their products (most likely through ads, shorter token windows, higher pricing and whatnot). As of now, not a sustainable business model thankfully. The only sad part is that this debt will hit the poorest people most.
This is not a technology, but ethics and respect problem.
From the same article:
> Not all AI-generated bug reports are nonsense. It’s not possible to determine the exact share, but Daniel Stenberg knows of more than a hundred good AI assisted reports that led to corrections.
Meaning: developers and researchers who use the tool as it's meant to work, as a tool, are leveraging it to improve curl. But they are not skipping the part of understanding the content of their reports, testing it, and only then submitting it.
E.g. We don't blame cars, the tool, for driving into a gathering of people that can kill a dozen of them, we blame the driver. The purpose is transport, the same way LLMs for coding are a tool for assisting coding tasks.
We do actually keep cars out of areas with lots of people here. And the media headlines always refer to a "car" driving into people without mentioning the person behind the steering wheel. Whether that's the better than addressing the root issue is another question though.
We also don't allow car use without a license.
In the end what matters if allowing something is a net positive or not. Of course you can have more precise rules than just a blanket ban but when deciding and enforcing those rules is not free that also needs to be considered in the cost benefit analysis. Unless you can propose how projects can allow "good" contributions without spending more time on weeding out bad ones, a blanket ban makes sense.
Objects don't have purposes or intent until people use them, and many objects have multiple reasonable and dual purposes. Objects can be used for net good and net harm. A bow and arrow isn't specifically for harming humans but can be used for such. Chainsaws and meat cleavers too.
What would you like a machine gun-wielding terrorist to be stopped with? A strongly-worded letter?
On the same token of reasonableness and rationality, it's unreasonable to give a toddler a towed howitzer that's ordinarily destined for Big Sandy Shoot.
Wrong. That's your projection and your value judgement. Guns are designed to shoot bullets. That's all that can be stated honestly. They can be used for "benign" activities, "good" things, and "bad" things... where the value varied depending on who is asked the question.
Even if they were designed only for "harm", you seem to believe "all harm bad". So should criminals in the midst of committing violent acts not be stopped because that would "harm" them? You won't answer this. Extreme pacifism is insane, morally-inconsistent, ideological, thoughtless drivel that fails to acknowledge the monopolies on violence delegated to police and military that they benefit from.
Perhaps you might want to have your military abolished because they are "designed to cause harm"? Or the whole abolish prisons and police nonsense? Real anarchy is really bad.
Technically an LLM is a tool for extracting candidate responses to plain-text requests. Since (textual) programming languages are languages, they can create passable candidate responses to queries about those. Certain LLMs such as Copilot and Claude have had their training focused a bit more towards programming tasks, but saying that LLMs as a class are for coding assistance is a little narrowly stated.
It would maybe be handy to feed the responses from an LLM through a computational reasoning engine to grade a few of them.
it kills the incentive to contribute, the incentive to maintain, the incentive to learn, the incentive to collaborate, and the ability to build a business based on your work
and it even kills the idea of traditional employment writing non-open source code
all so three USian companies can race to the bottom to sell your former employer a subscription based on your own previous work
I couldn't possibly disagree more. AI has created an entirely new way to contribute to open source. You can not, in addition to donating to the maintainers, donate your _tokens_ to fix bugs.
already decades ago when we were kids eating pudding with a fork was a fun past time, and i am sure the idea is as old as pudding or forks themselves. i mean, the fact that it spread so fast shows that there are many who already practiced it. it's actually surprising it took this long to become a meme.
heck, my cousin bet with me or let me compete eating pudding with chopsticks. (and that was long before i went to china)
practically speaking, the only downside of using a fork (or chopsticks) is scraping the bottom when you are finishing up.
How so? I think the Bazaar model has the most to gain - contributors can use LLMs to create PRs, and you can choose from a vast array of projects depending on how much you trust vibe coding.
Outside of direct monetary gain like bounties are efforts to just stand out, in terms of being able to show contributions to a large project or getting say a CVE.
Stenberg has actually written about invalid/wildly overrated vulnerabilities that get assigned CVEs on their blog a few times and those were made by humans. I often get the sense some of these aren't just misguided reporters but deliberate attempts to make mountains out of molehills for reputation reasons. Things like this seem harder to account for as an incentive.
> “Not much. The real incentive for finding a vulnerability in cURL is the fame ('brand is priceless'), not the hundred or few thousand dollars. $10,000 (maximum cURL bounty) is not a lot of money in the grand scheme of things, for somebody capable of finding a critical vulnerability in curl.”
That's the choice as seen from the perspective of a white-hat hacker. But for an exploitable vulnerability, the real choice is to sell it to malware producers (I'm including state-sponsored spyware companies like the makers of Pegasus in this category) for a lot of money, or do the more moral thing and earn at least a little bit of money via a bug bounty program.
That's a story that people like to tell to justify bug bounty programs, but it strikes me as very unlikely that some random pentester / white-hat hacker would have access to communication with malware producers.
Black-hat hackers seem entirely unreasonable to deal with, you'd have to manage some sort of escrow payment (because neither party trusts the other) probably through cryptocurrency, and then deal with laundering the money, et cetera.
Perhaps one could as you theorize, go to some private company, but it'd have to be at least somewhat approved by the white-hat hacker's own government lest they risk legal trouble, and I'm still dubious that the company would be all that willing to pay for some "freelance hacker's" supposed vuln.
Hackerone (where cURL hosted their bounty program) tracks the reputation of bounty hunters. I don't understand why they are not taking advantage of this. Make a private program, invite only hackers who have proved themselves by submitting relevant reports.
i think that hypothetical is too simplistic to accurately frame the situation. we're talking about one of the largest, most widely used libraries in the open source world. at that level, they don't really need unknowns to use their project to "prove themselves" - they can contribute to smaller projects or put their own work out into the world.
I started watching it but the modern presentation style of shouting everything instead of speaking at a normal volume, and using many many gestures and facial expressions to state a simple sentence made me switch it off rapidly.
It seems to be a presentation style afflicting the YouTube generation, where they think you want to see a colossal microphone in someone's face (directional microphones work very well, and is a reason you can hear dialogue in a film without a mic in front of everyone) whilst they gesticulate wildly and over-emphasise words in a sentence. It is quite wearying; perhaps it's because I am British, but it is afflicting general conversation where only superlatives can be used ("awesome" "amazing" "insane" "mind-blowing") instead of "good"/"enjoyable".
The company I work for has a pretty bad bounty system (basically a security@corp email). We have a demo system and a public API with docs. We get around 100 or more emails a day now. Most of it is slop, scams, or my new favourite AI security companies sending us an AI generated pentest un prompted filled with false positives, untrue things, etc. It has become completely useless so no one looks at it.
I had a sales rep even call me up basically trying to book a 3 hour session to review the AI findings unprompted. When I looked at the nearly 250 page report, and saw a critical IIS bug for Windows server (doesn't exist) existing at a scanned IP address of 5xx.x.x.x (yes an impossible IP) publically available in AWS (we exclusively use gcp) I said some very choice words.
It makes sense. This process of searching for bugs was slow and time-consuming so it needed to be incentivized. This is no longer the case. Now the hard part is in identifying which ones are real.
To paraphrase a famous quote: AI-equipped bug hunters find 100 out of every 3 serious vulnerabilities.
The process of finding bugs is still slow and time consuming. The kinds of vulnerabilities you find in codebases like cURL are still beyond AI. Binary exploitation is still a human only field.
In the second report, Daniel greeted the slopper very kindly and tried to start a conversation with them. But the slopper calls him by the completely wrong name. And this was December 2023. It must have been extremely tiring.
This (manual?) addition in the second report [1] likely gives an idea as to the reporter's mastery of English and ability to proofread before spamming out slop:
> Sorry that I'm replying to other triager of other program, so it's mistake went in flow
I think it would be really interesting if someone at HackerOne did a dive into the demographic of many of the banned posters.
December 2023... that was early AI era. Had to double-check the dates actually, because I misremembered the release date of GPT-4 as being in 2024; turns out it was in 2023, and that was when LLMs first became remotely useful for even this kind of slop.
I looked at two reports, and I can’t tell if the reports are directly from an ai or some very junior student not really understanding security. LLms to me sound generally more convincing.
Yeah, that one is pretty clearly written with the help of AI. This could well be the work of a larger group, say a state actor, trying to overwhelm reviewers and crowd out real reports. And if not yet, then for sure going forward ...
They're very clearly AI when you're told that it's a list of AI. But when you're given a mixed list of AI and genuine reports, I bet it's not so simple and very time consuming
It's really hard to comprehend how entitled "reward which would be the cost of lunch in Sweden can be massive for those low socio-economic-located people" is.
A lunch in central Stockholm is well within 200 kronor. I can't imagine a country where an person with a computer and skills necessary to claim a bounty in cURL would consider that amount massive.
Is he entitled for considering the difficultied from developing countries?
Maybe he's exaggerating a bit with the comparison, but if we consider the ceiling of 10k, that's more than a year of minimum wage in Portugal, or a handful of years in certain developing counties. Certainly significant!
They don't care. They generate large amounts of these, spam them out, and hope for some small success. If they get banned or blocked, they make new accounts. Shame isn't even a factor; it's all about money. They don't even attempt to understand or care about a product.
This was partially the case before, where you'd still get weird spammy or extortive reports, but I guess LLMs enable random people to shoot their shot and gum up the works even more.
That's because we stopped calling others out for shameful, disrespectful or unethical behavior as a rule. So there is less or nothing to be ashamed about anymore.
LLMs have been marketed for years, with great meadia fanfare, as being almost magical, something that can do the job of software engineers. Every week, the hype is driven further.
This matters. When people get told everyday that XYZ is magic, some will believe so, and use it as if it is magic.
I'm not sure I completely agree, I don't think it's that black and white, it's a similar analogy to Guns and gun violence.
Without the prevalence of guns there is simply less gun violence, but you could argue that it's also a human problem.
Giving people who have no business using an LLM to submit slop bug bounties is a problem of the tools accessibility. But also a human problem of course.
Edit: I should mention, I don't have a solution to the problem although I do like the other posters suggestion of a "deposit" scheme to submit a bug. I think that would incentives higher quality submissions.
I'm not sure how effective this will be. A lot of AI-generated bug bounty reports are pure spam, but a significant fraction are well-meaning humans who genuinely believe the nonsense an LLM has given them. The former category do not read the rules in the first place and will not be deterred - spray and pray is their MO. The second category will not believe that any "no slop" rules apply to them, because they genuinely think their bug is real.
What I wonder is if this will actually reduce the amount of slop.
Bounties are a motivation, but there's also promotional purposes. Show that you submitted thousands of security reports to major open source software and you're suddenly a security expert.
Remember the little iot thing that got on here because of a security report complaining, among other things, that the linux on it did not use systemd?
I dont think bounties make you an "expert". If you want to be deemed an expert, write blogs detailing how the exploit works. You can do that without a bounty.
In many ways one of the biggest benefits of bug bounties is having a dedicated place where you can submit reports and you know the person on the other end wants them and isn't going to threaten to sue you.
For the most part, the money in a bug bounty isn't work the effort needed to actually find stuff. The exception seens to be when you find some basic bug, that you can automate scan half the internet and submit to 100 different bug bounties.
This is silly, people don't need AI to send you garbage. If your project is getting lots of junk reports, you should take it as a good sign, that people are looking at it a lot now. You don't remove the incentive, you ask for help to triage the junk.
Curl is a popular and well supported tool, if it needs help in this area, there will be a long line of competent people not volunteering their time and/or money. If you need help, get more help. don't use "AI slop" as an excuse to remove the one incentive people have to not sell exploits or just hoard them.
Curl did already tend to get a decent number of junk reports from people who just didn't know what they were doing, but this was limited to the number of productive idiots who focused their productivity on curl specifically. AI allows significantly less motivated idiots to create substantially more workload, and therefore upgrades this phenomenon from a minor annoyance to a big problem, one that may just render publicly submitted bug reports not worth the project's time.
(And no, curl does not have a huge pool of potential maintainers to pull from on this. Open-source software in general suffers from a big lack of manpower, especially relative to the popularity of the tool)
My point was #1 that it is a volume problem, and #2, you don't need maintainers to triage bugs and prs, even bots can do that for simpler things. They can have a pool of project members to upvote a bug report before maintainers look at it.
There are many incentives not to sell exploits, the major one being that it's not logistically feasible. First of all the people submitting these false reports don't have any real exploits.
But imagine you were sitting on an actual RCE exploit in curl, who would you sell it to? How would you convince them it's working without disclosing the details for free? How would you get paid?
> Curl is a popular and well supported tool, if it needs help in this area, there will be a long line of competent people not volunteering their time and/or money
I'm not sure if that not is a typo, but yes, even though a tool is very popular, there's almost nobody competent and willing to work on it for free. This has been a well-known problem in open source for decades now.
It's a typo, even if they don't sell it why report it to curl? for clout? You can still exploit it against real world apps. Who would they sell it to? I would sell it to zerodium instead of report to curl personally.
How much time do people spend finding bugs, is their time not worth anything because some other random people decide to use AI?
Curl is high-visibility, there are people. and it doesn't take a lot of competency to triage. Heck, I like to think I have a good handle at C and memory exploitation, I will volunteer my time for free if they need help.
What's your point? Because people smoke cigarettes, people who buy unrelated things should be punished? Or because a store sells cigarettes, stores in general shouldn't be paid for what they sell? Or is the time and effort to find vulns valueless?
No, the implication that "THING" is the cause of something and therefore something needs to be done must withstand the scrutiny of "other THINGS" also causing that thing, and therefore the solution is attacking either only one cause or not the real root cause.
The fact that bad reports have to be triage doesn't change with AI. What changed is the volume, clearly. So the reasonable response is not to blame "AI" but to ask for help with the added volume.
If HN gets flooded by AI spam, is the right response shutting down HN? spam is spam whether AI does it or a dedicated and coordinated large numbers of humans do it. The problem doesn't change because of who is causing it in this case.
The change in volume was the tipping point between bug bounties being offered and devs being able to handle bad reports, and bug bounty nixed because devs no longer willing to handle the floos.
And the root cause for the change in volume is generative AI.
So yes, this is causally related.
> The problem doesn't change because of who is causing it in this case.
Wrong.
Because SCALE MATTERS. Scale is the difference between a few pebbles causing a minor inconvenience, and a landslide destroying a house.
So whatever makes the pebbles become a landslide, changed the problem. Completely.
How can you say "wrong." and then go on to say scale matters, that means scale is the problem, not who is reporting it, you contradicted yourself.
We're in agreement that it is a scale issue. When something needs to scale, you address the scale problem. Obviously the devs can't handle this volume, and I agree with that there too. Our disagreement is the response.
I guarantee that if they asked for volunteers they'll get at least 100 within a week. They can filter by previous bug triage experience and experience with C and the code base. My suggestion is to let people other than the devs triage bug reports, that will resolve the scale problem. curl devs never have to see a bug not triaged by a human they've vetted. There is also no requirement on their part to respond to a certain number of bug reports, so with or without help, they can let the stack pile up and it will still be better than nothing.
I read the specific examples that the maintainers of each project gave. The ones the ffmpeg maintainers complained about appeared to be real bugs. In fact i dont think the ffmpeg maintainers even dispute that. The ffmpeg maintainers after all fixed the issue in question. The ones the curl maintainers gave were clearly nonsense. The curl maintainers did not fix the issue because there was nothing to fix because the report didn't actually report anything that could be fixed because it made no sense.
The LLM models give the most likely respond to a prompt. So if you prompt it with "find security bugs from this code" it will respond with "This may be a security bug" than you "you fucking donkey this curl code has already been eyeballed by hundreds of people, you think a statistic model will find something new?"
Funny how we are now sensitivized to these AI slops, at first I fixated on the En dashes in the lead of the article, made me doubt of the article's author for a few seconds.
Because LLMs are bad at reviewing code for the same reasons they are bad at making it? They get tricked by fancy clean syntax and take long descriptions / comments for granted without considering the greater context.
I don't know, I prompted Opus 4.5 "Tell me the reasons why this report is stupid" on one of the example slop reports and it returned a list of pretty good answers.[1]
Give it a presumption of guilt and tell it to make a list, and an LLM can do a pretty good job of judging crap. You could very easily rig up a system to give this "why is it stupid" report and then grade the reports and only let humans see the ones that get better than a B+.
If you give them the right structure I've found LLMs to be much better at judging things than creating them.
Opus' judgement in the end:
"This is a textbook example of someone running a sanitizer, seeing output, and filing a report without understanding what they found."
"Tell me the reasons why this report is stupid" is a loaded prompt. The tool will generate whatever output pattern matches it, including hallucinating it. You can get wildly different output if you prompt it "Tell me the reasons why this report is great".
It's the same as if you searched the web for a specific conclusion. You will get matches for it regardless of how insane it is, leading you to believe it is correct. LLMs take this to another level, since they can generate patterns not previously found in their training data, and the output seems credible on the surface.
Trusting the output of an LLM to determine the veracity of a piece of text is a baffilingly bad idea.
>"Tell me the reasons why this report is stupid" is a loaded prompt.
This is precisely the point. The LLM has to overcome its agreeableness to reject the implied premise that the report is stupid. It does do this but it takes a lot, but it will eventually tell you "no actually this report is pretty good"
The point being filtering out slop, we can be perfectly find with false rejections.
The process would look like "look at all the reports, generate a list of why each of them is stupid, and then give me a list of the ten most worthy of human attention" and it would do it and do a half-decent job at it. It could also pre-populate judgments to make the reviewer's life easier so they could very quickly glance at it to decide if it's worthy of a deeper look.
https://hackerone.com/curl/hacktivity Add a filter for Report State: Resolved. FWIW I agree with you, you can use LLMs to fight fire with fire. It was easy to see coming, e.g. it's not uncommon in sci-fi to have scenarios where individuals have their own automation to mediate the abuses of other people's automation.
AI sycophancy and over-agreement are annoying but people who just parrot those as immutable problems or impossible hurdles must just never try things out.
It's interesting to try. I picked six random reports from the hackerone page. Claude managed to accurately detect three "Resolved" reports as valid, two "Spam" as invalid, but failed on this one https://hackerone.com/reports/3508785 which it considered a valid report. All using the same prompt "Tell me all the reasons this report is stupid". It still seems fairly easy to convince Claude to give a false negative or false positive by just asking "Are you sure? Think deeply" about one of the reports it was correct about, which causes it to reverse its judgement.
No. I already found three examples, cited sources and results. The "burden of proof" doesn't extend to repeatedly doing more and more work for every naysayer. Yours is a bad faith comment.
>However, I should note: without access to the actual crash file, the specific curl version, or ability to reproduce the issue, I cannot verify this is a valid vulnerability versus expected behavior (some tools intentionally skip cleanup on exit for performance). The 2-byte leak is also very small, which could indicate this is a minor edge case or even intended behavior in certain code paths.
Even biased towards positivity it's still giving me the correct answer.
Given a neutral "judge this report" prompt we get
"This is a low-severity, non-security issue being reported as if it were a security vulnerability." with a lot more detail as to why
So positive, neutral, or negative biased prompts all result in the correct answer that this report is bogus.
Yet this is not reproducible. This is the whole issue with LLMs: they are random.
You cannot trust that it'll do a good job on all reports so you'll have to manually review the LLMs reports anyways or hope that real issues didn't get false-negatives or fake ones got false-positives.
This is what I've seen most LLM proponents do: they gloss over the issues and tell everyone it's all fine. Who cares about the details?
They don't review the gigantic pile of slop code/answers/results they generate. They skim and say YOLO. Worked for my narrow set of anecdotal tests, so it must work for everything!
IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was?
This is a very similar scenario: make some judgement call based on a small set of data. It absolutely sucks at it. And I'm not even going to get into the issue of liability which is another can of worms.
Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think?
I'm not talking about completely replacing humans, the goal of this exercise was demonstrating how to use an LLM to filter out garbage. Low quality semi-anonymous reports don't deserve a whole lot of accuracy and being conservative and rejecting most reports even when you throw out legitimate ones is fine.
You seem like regardless of evidence presented, your prejudices will lead you to the same conclusions, so what's the point discussing anything? I looked for, found, and shared evidence, you're sharing your opinion.
>IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was?
I'm talking about filtering spammy communication channels, that has nothing like the care required in making employment decisions.
Your comment is plainly just bad faith and prejudice.
> Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think?
I assumed you knew how LLMs work. They are random by nature, not "because I'm guessing it". There's a reason if you ask the LLM the same exact prompt hundreds of times you'll get hundreds of different answers.
>I looked for, found, and shared evidence
Anecdotal evidence. Studies have shown how unreliable LLMs are exactly because they are not deterministic. Again, it's a fact, not an opinion.
>I'm talking about filtering spammy communication channels
So if we make tons of mistakes there, who cares, right?
I only used this as an example because it's one of the few very public uses of LLMs to make judgement calls where people accepted it as true and faced consequences.
I'm sure there are plenty more people getting screwed over by similar mistakes, but folks generally aren't stupid enough to say that publicly. Maybe the Salesforce huge mistake qualifies too? Incidentally it also involved people's jobs.
Regardless, the point stands: they are unreliable.
Want to trust LLMs blindly for your weekend project? Great! The only potential victim for its mistakes is you.
For anything serious like a huge open source project? That's irresponsible.
How would it work if LLMs provide incorrect reports in the first place? Have a look at the actual HackerOne reports and their comments.
The problem is the complete stupidity of people. They use LLMs to convince the author of the curl that he is not correct about saying that the report is hallucinated. Instead of generating ten LLM comments and doubling down on their incorrect report, they could use a bit of brain power to actually validate the report. It does not even require a lot of skills, you have to manually tests it.
Let the reporter duke it out with the project's gatekeeping LLM. If it keeps going on for long enough a human can quickly skim the exchange. It should be immediately obvious if the reporter is making sensible rebuttals or just throwing more slop at the wall.
I think fighting fire with fire is likely the correct answer here.
The solution for this, IMO, is flags. Just like with CTFs, host an instance of your software with a flag that can only be retrieved after a successful exploit. If someone submits the flag to you, there is no argueing about wether or not they found a valid vulnerability.
Yes, this does not work for all vulnerability classes, but it is the best compromise in my mind.
How exactly would that work? Curl isn't exactly software that can be "hosted" somewhere, and I'm not sure where you'd hide the flag in the software? Either very few actual vulns would end up being able to retrieve the flag, or it would be trivial to retrieve the flag without an exploit.
In most basic form it would just be form with URL that (lib)curl is later supposed to fetch. And target server (controlled by researcher) is supposed to send payload that triggers RCE in client.
Sure, it covers a very narrow scope but I am afraid the bigger issue would be that it is going to get spammed with submitted links. And those links will often be to strait up illegal content, it might not matter that such server instantly deletes all downloaded files.
Simple. You multiple instances with different flags covering different threat models. RCE, file read, etc. You then expose a webapplication for every instance that lets users control only those curl flags, that must be safe to be user controlled in the reapective threat model.
An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
Then again, I once submitted a bug report to my bank, because the login method could be switched from password+pin to pin only, when not logged in, and they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password. (And that's not even getting into the difference between real two-factor authentication the some-factor one-and-a-half-times they had implemented by adding a PIN to a password login.) I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
Assuming the host of the bug bounty program is operating in good faith, adding some kind of barrier to entry or punishment for untested entries will weed out submitters acting in bad faith.
Bug bounties often involve a lot of risk for submitters. Often the person reading the report doesn't know that much and misinterprets it. Often rules are unclear about what sort of reports are wanted. A pay to enter would increase that risk.
Honestly bug bounties are kind of miserable for both sides. I've worked on the recieving side of bug bounty programs. You wouldnt believe the shit that is submitted. This was before AI and it was significant work to sort through, i can only imagine what its like now. On the other hand for a submitter, you are essentially working on spec with no garuntee your work is going to be evaluated fairly. Even if it is, you are rolling the dice that your report is not a duplicate of an issue reported 10 years ago that the company just doesn't feel like fixing.
Vulnerability disclosure is general is just miserable. Before all the bug bounty issues it was pretty common to:
* Spend ages trying to find someone to submit a report to.
* Waste a whole load of time fighting through the generic contact and support desks to try and get your report to someone who understood it.
* Get completely ignored by the developers.
* Spend time reporting a bug only for them to silently fix it without even bothering to respond to you, let alone acknowledge you.
* Get legal threats for making a good-faith bug report, even if you found it in an locally deployed instance of the software.
* Get called a black hat and more legal threats when you give up and just go down the full disclosure route.
Pay to enter would increase the risk of submitting a bug report. However, if the submission fees were added to the bounty payable, then the risk reward changes in favour of the submitter of genuine bugs. You could even have refund the submission fee in the case of a good faith non bug submission. A little game theory can go a long way in improving the bug bounty system...
If a competent neutral party was evaluating them, i would agree. However currently these things tend to be luck of a draw.
You’re assuming that the companies operating these programs would act in good faith which they often do not.
They could allow submitters to double down on submissions escalating the bug to more skilled and experienced code reviewers who get a cut of the doubled submission fee for reviews.
Indeed, increasing the incentive for companies to reject ( and then sometimes silently fix anyway ) even the valid reports would only increase further misery for everyone.
Indeed. I've also gotten a lot of "Hey I found a super critical security bug that makes your system ass, but if you pay me first I'll tell you what it is" types of submissions. Sometimes it's a hybrid, like something semi-legitimate but weak that they disclose up front, closing with a "and I've got another one that's juicy but you have to pay me first to hear it".
Oh and then of course there is the flood of people who just scanned our infra with Nessus and screenshotted part of the report (often with important details blacked out so we can't see them unless we pay).
As someone who has been on both sides of it as well, it just feels like everything is terrible
Real risk is missed security issue
You’re not addressing any of the points made.
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
Sadly, yeah. And will do anything only if they believe they can actually be caught.
An EU-wide bank I used to be customer of until recently, supported login with Qualified Electronic Signatures, but only if your dongle supports... SHA-1. Mine didn't. It's been deprecated at least a decade ago.
A government-certified identity provider made software that supposedly allowed you to have multiple such electronic signatures plugged in, presenting them in a list, but if one of them happened to be a YubiKey... crash. YubiKey conforms to the same standard as the PIV modules they sold, but the developers made some assumptions beyond the standard. I just wanted their software not to crash while my YubiKey is plugged in. I reported it, and they replied that it's not their problem.
Name and shame, and give them a trial by media.
A problem with this approach is that one of the key functions of a bug bounty program is to encourage people to report vulnerabilities to the developers, rather than selling them elsewhere.
If I have to pay money to submit a vulnerability to the developers with no guarantee that I'll even get refunded for a high quality and good faith report, let alone any actual payout, there's much less incentive for me to do so compared to selling them to someone else who won't charge me money for the privilege.
In a past life I was deeply involved in the operation of a bug bounty program. Discouraging people from selling on the black market was nowhere on the list of motivations.
We wanted to encourage white hat security researchers to look at our domain rather than other domains so we could collect more data on the kinds of vulns that appeared in our domain to help prioritize efforts that would fix the root causes of recurring bug patterns.
I've also submitted bug bounties and received rewards and I've worked with a bunch of other people who have done this. At no point did I even consider selling on the black market and I suspect that my friends from grad school were the same way.
Maybe the $1,000,000 bounties for zero click rce on iphones or whatever exist to discourage selling on the black market, but I'm not even sure that is true. "Well, I'll just find a way to sell this to the russian mob" is not exactly something that is on the radar of the vast majority of security researchers.
The reality is that most people's thoughts on bug bounties are from salacious headlines talking about those $1M vulnerabilities. In reality the average bug bounty submission is a machine translated report for a low severity issue in a web app that may or may not even exist (or be a vulnerability), sprayed at hundreds of companies (or the same company a hundred times) in the hopes of earning $500 to basically do currency manipulation.
There are plenty of places you can sell exploits other than OCGs. At the more legitimate end of that market is people like ZDI who will then collaborate with the vendors (after a time), or companies making exploit kits/tooling for pentesters/red teaming. More questionable ones are companies that make things like forensics tools or spyware who are legal, but perhaps ethically dubious. All completely legal, but not great for the wider community if they're getting the vulns rather than the developers.
If you're trying to protect your own website and servers, those markets won't be a concern for you. If you ship a widely used product that's an attractive target (like web browser, mobile device, network kit, etc) then they definitely are.
You don't sell it to the "Russian mob", you sell it to a highly reputable security company that will buy it for like $10 million or more and sell it to governments and stuff, not the mob.
I mean, seriously.
Why would I ever go find a 0 click rce bug and then just donate it to a trillion dollar company just to get a "thx" when I can just retire right then and there?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
That adds an extra layer of complexity to the cURL maintainers, handling other people’s money and whatnot. It was considered.
Daniel (cURL’s lead) has been discussing this for months. Whatever “quick and easy” solution you think of, it’s already been suggested and thought about and rejected for some reason.
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
I personally came to that conclusion thanks to the GrapheneOS situation regarding device attestation. Insecure devices get full features from some apps because they are certified, although they cite security, while GrapheneOS get half featured apps because it's "insecure" (read, doesn't have the Google certification, but are actually the most secure devices you can get, worldwide)
It's not about securing your device from external threats or bad actors; it's about securing the device from you.
I see it a little differently. I would change your statement to the following:
It's not about securing your device from external threats or bad actors; it's about securing the organization from any blame / wrongdoing.
Most organizations today are looking high and low to shove the blame to others instead of taking responsibility.
It's related, but GP is still right to bring it up - it's the one question that is most important wrt. security, and also conveniently the least often asked: security for who, and from what? "Security" isn't an absolute good.
Play Integrity certifies (and banks/etc approve) that Android 8.0 (oreo) unpatched for several years, full of vulnerabilities for RCE, 0-click, privilege escalation, etc, so full of holes it's trivial to get a root and then hide it (or use leaked cert), is absolutely a-ok, safe to use and secure for user.
Yes?
Is this what you're suggesting? :)
Because this is what's certified and embraced.
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
I refer to this as the Notion-to-Confluence cost border.
When Notion first came out, it was snappy and easy to use. Creating a page being essentially free of effort, you very quickly had thousands of them, mostly useless.
Confluence, at least in west EU, is offensively slow. The thought of adding a page is sufficiently demoralizing that it's easier to update an existing page and save yourself minutes of request time outs. Consequently, there's some ~20 pages even in large companies.
I'm not saying that sleep(15 * SECOND) is the way to counter, but once something becomes very easy to do at scale, it explodes to the point where the original utility is now lost in a sea of noise.
It’s strange how sensitive humans are to these sort of relative perceived efforts. Having a charged, cordless vacuum cleaner ready to go and take around the house has also changed our vacuuming game. Because carrying a big unwieldy vacuum cleaner and needing to find a power socket at every location just feels like much more effort. Even though it really isn't.
It is. The classical vacuum is heavier, you have to find the socket and plug it in (non-trivial if you have few of them, or have kids and sockets have kid blocks on them), and perhaps most importantly, you need two free hands to operate it (particularly when carrying, plugging in and repositioning). That alone is enough to turn it into a primary activity, i.e. the kind of thing that you explicitly decide to do and becomes your main focus. Meanwhile, a charged cordless cleaner is something you can semi-consciously grab with one hand while passing by, and use on the go to do some cleaning while actually focused on something else. It's an entirely different class of activity, much easier to fit in during the day.
Ironically, the cordless vacuum is even better than vacuum robots in this regard! I was surprised to hear from some friends and acquaintances that they prefer the manual vacuum to robotic one, and find it a better time/effort saver - but I eventually realized they're right, simply because the apps for controling the robotic vacuums are all steaming piles of shit, and their bad UI alone turns activating the robot into primary activity. It may be a brief activity, but it still requires full focus.
Have you tried a handheld corded unit? I find most of the perceived effort with the larger unit is due to a combination of the bulk, the weight, and having to maneuver it around anything delicate without inadvertently bumping into it. Meanwhile the corded handheld never dies on me and I think isn't as bad for my hearing. All the cordless units I've tried seem to have an extremely high pitch whine to them.
Ok but the corded vacuum actually fucking works.
I keep having to get it from progressively more inconvenient locations to which it has been banished in order to humor my wife’s delusion that the roomba or the handhold do anything.
I can make multiple passes with the handheld to get 80% of the crumbs in a small area, troubleshoot why the robot didn’t run yesterday in order to hope it will get the crumbs tomorrow, or just get the corded vacuum out and actually clean a whole room.
Work involves cables. Any product that promises something different is a lie.
Maybe you just have a shit vacuum.
Our cordless, on the highest suction setting, is bordering on unusable. The effort to move it across carpet becomes quite high. Trying to roll it on an area rug tends to cause it to drag the rug around, and if you pick it up while on it will pull the rug up off the floor.
I have done some _very_ scientific testing here, vacuuming a section of carpet on the lowest section (doing lines where each pass half-overlapped the previous so each part of the carpet got touched once in each direction), emptying the vacuum, then going back over doing the same on high. Didn't see anything else come up. Shop vac didn't pull anything else out either that I could see.
I used to be in a similar boat of "these are a stupid class of product", but end of the day even if it takes eight passes my wife was going to use it anyway. The effort for her to set the time aside to drag around the heavier corded vacuum which is a substantial effort for her, etc, would be more than doing eight passes with a cordless. So got a good one and I'm sold on it now--it is quite convenient, and it does work.
Only thing I will say is the battery definitely can't do an entire carpeted house on a charge. We don't have that much carpet, so don't have any problem cleaning all the floors and a couple area and entry-way rugs on a charge.
This is an interesting discussion to me - I have a cordless vacuum that works well and a roborock combo vac/mop that works well. Actually, I'm lying, I have two cordless vacuums because the GGP's observation rings true to me and I got a second one for free and held on to it. :-)
Dyson cordless vac, older (v8 ultimate). Have had to replace battery once and broken trigger. Continues to be a workhorse.
Roborock s5v: I have it run 2x / day on weekdays, once in the morning after breakfast when we're taking the kids to school (vac kitchen only), and once after bedtime (vac + mop entire area). It does a great job of generally keeping things clean. Not perfect, but the overall dirt level stays low.
The cordless manual vac is really useful for "oh bleep, 8yo just spilled MORE stuff on the ground". I keep it next to the dining and kitchen area. It's not super aesthetic having it hanging on the wall in a visible location but I have engineer-itis and I value the convenience over the illusion that we don't own a vacuum. :) I approximately never use the robovac as an on-demand vacuum unless it's to run an extra pass when we're leaving home on a weekend and have left crumbs from a meal.
For us, substantially upping the frequency of vacuuming, even if it's not quite as deep, has made a big difference, and it's basically no extra burden to have the robovac run frequently after programming it.
What model of cordless vacuum do you have?
Well... I have a (1000 dollar!!!) Dyson cordless vacuum, arguable the laser + the histogram for "removed particles of size X", make me clean more thoroughly. The things is pretty heavy and applies a pretty good vacuum, imho.
Bro the vacuum community is audiophile-level picky. I have a Dyson stick vacuum of some sort and I haven’t had any issue with picking up crumbs. I would rather manually bend over and pick up something it doesn’t grab than move around the heavy corded vacuum and plug it in 10 times.
I feel this sub thread can keep going if we introduce the complication of the whole-house vacuum system.
We can also spin off a subthread about pets, and another one about using vacuum cleaners on surfaces other than floor/carpet.
The term I know / used for this is "trivial inconveniences", via an old article of Scott Alexander[0].
The quote from example from early in the article stuck with me for years:
Think about this for a second. The human longing for freedom of information is a terrible and wonderful thing. It delineates a pivotal difference between mental emancipation and slavery. It has launched protests, rebellions, and revolutions. Thousands have devoted their lives to it, thousands of others have even died for it. And it can be stopped dead in its tracks by requiring people to search for "how to set up proxy" before viewing their anti-government website.
(Now this is more poetic, but I suppose the much more insightful example that also stuck with me is given later - companies enticing you to buy by offering free money, knowing well that most customers can't be arsed to fill out a form to actually get that money.)
--
[0] - https://www.lesswrong.com/posts/reitXJgJXFzKpdKyd/beware-tri...
I suspect that applies specifically to their cloud rewrite which was apparently a bloat of JS libs and hundreds of requests even by Atlassian standards. The on-prem self-host Confluence I've used is still pretty snappy and pleasant to use and without throwing an absurd amount of resources at it. We do have quite a lot of actually-useful documentation in it.
That said, Atlassian is busy relentlessly raising the price for self-host to push people into their cloud roach motel, so we'll probably be on some alternative (either FOSS or commercial, but self-host) soon too.
I find this to be a very amusing critique. In my experience, Notion (when I stopped using it 3 years ago) was slow as molasses. Slow to load, slow to update. In comparison, at work, I almost exclusively favor Confluence Cloud. It's very responsive for me.
We have tons of Confluence wikis, updated frequently.
I think it might be the same issue as with WordPress and Jira - terrible plugins. Each company uses own special mix, and encounters issues often occurring in that one specific configuration. And it is the base platform that takes the blame.
In particular a place I used to work had a plugin for threaded comments in Jira. The specific one we were using slowed things down noticeably with the DB on the same server, but not too much to be an improvement in overall usefulness.
Then we decided trying to make our Jira more reliable by splitting the DB out into a separate clustered DB system in the same data center. The latency difference going through a couple of switches and to another system really added up with those extra 1600 or so DB calls per page load.
We ended up doing an emergency reversion to an on-host DB. Later, we figured out what was causing that many queries.
You're referring to the on-prem Jira. That might suck, sure. My experience has been purely using Jira Cloud and Confluence Cloud, both of which I've found to be snappy and responsive.
My last company switched several teams to Jira Cloud. My current company started with Cloud when we moved over from other tools.
Cloud does not give you the flexibility of your own plugins, your own redundancy design, or your own server upgrades. On top of that, the performance is pretty variable and is far worse than a self-hosted Jira on fast hardware.
It’s interesting to me that your lack of experience to make a comparison qualifies you in some way to criticize the experience I actually have.
Amusingly, exactly opposite experience here. That said, our on-prem is jira and confluence integrated with db on same machine, and apache in front doing additional caching. I imagine like so many things it is how you set it up...
If you read my previous comment, I said it was largely the specific poor plugin that caused most of the performance issue with the database queries. I never complained about the overall speed of on-prem Jira. That was the assertion of the person who’s only ever used the cloud version.
> Consequently, there's some ~20 pages even in large companies.
As someone working on Confluence to XWiki migration tools, I wish this was remotely true, my life would be way easier (and probably more boring :-)).
Interesting. First I've heard of Xwiki - it does look nice, and what with Atlassian's price increases... do you have any migration tips?
[edit] I found https://extensions.xwiki.org/xwiki/bin/view/Extension/Conflu... - hopefully that's a good reference.
> hopefully that's a good reference
It is!
> what with Atlassian's price increases
And the end of their self hosting offerings (Server, Data Center), which is currently driving a lot of people towards XWiki, for other reasons than money. XWiki SAS being mainly in Europe makes it attractive to EU users too.
> do you have any migration tips?
I don't have specific migration tips. I hope the docs are complete enough!
However, I may suggest having a look at XWiki SAS's professional offering: https://store.xwiki.com/xwiki/bin/view/Extension/Confluence%...
The Confluence Migration Toolkit is based on the Confluence XML module you found, but it adds a nice and convenient UI, converts some more macros that XWiki SAS sells, there's support, and there's consulting for larger migration projects or projects with special requirements.
(note: despite some paying features, everything is open source)
(disclaimer in case it was not obvious, I work for XWiki SAS)
I found that banks are one of the worst organizations when it comes to authentication. They are regulated but the requirements are completely outdated and irrelevant in a risk context.
And then you have banks such as Boursobank (a French online bank) that has weak traditional authentication (and a faulty app, but they do not care) and out of the blue also provides passkeys. Making it at the same time horribly bad and wonderfully good.
The worst part is that they hide behind regulations when in fact there are only few of them.
Other instiytutions such as SWIFT are as bad and equally arrogant.
For weak bank logins, my guess is that reimbursing all account takeovers is cheaper than having a complex login process that would scare away non-technical customers. Or, well, I could see myself making that decision if I were more versed in finance than in computer science and I had a reasonable risk assessment in front of me to tell me how many account takeovers happen.
Banks aren't even liable for losses from account takeovers, at least if their system is compliant, regardless of whether that makes it secure. Their biggest incentive is customer satisfaction, which fraud does hurt.
It's credit cards that have to reimburse for fraud, but they charge the merchant for it, plus fees, so they have absolutely no incentive to prevent fraud, if not an incentive to outright encourage fraud. That would explain why their implementation of the already compromised EMV was further nerfed by a lack of a PIN in the US.
> Their biggest incentive is customer satisfaction
At a bank? No way. They are some of the most customer-hostile organizations I've interacted with. Dealing with payment accounts is a necessary evil for them, and they are very much aware of the effort required to switch to a different bank, and of the massive regulatory moat preventing consumer-friendly competition from popping up.
A bank doesn't care about screwing over a handful of customers. As long as it's not common enough to draw the attention of the press and/or a regulatory agency, they are not going to spend any money on improving.
Case in point: Wells Fargo foreclosure fraud. Case in point: Wells Fargo opening new accounts in customer names without direction from, approval by, or notification to said customers.
The primary incentive of a bank is to make money rather than customer satisfaction, security, or most other things. Sometimes other priorities suffer in the race to profit, sometimes including regulatory compliance and legality.
That anecdote is hilarious and scary in equal measures. Optional passwords are certainly more convenient than required ones, but so are optional PINs. The most convenient UX would be never needing to log in at all! Unless you find it inconvenient for others to have access to your bank account of course
And the counter where the most secure system never allows anyone to log in ever
And the uncomfortable truth that the ideal level of security is much closer to the former than to the latter.
I really hate the current trend of not having passwords. For example perplexity doesn't have a password, just an email verification to login.
That's what eBay does to me. You get to choose, at the time of login, between entering a password and getting an email verification, or just getting an email verification. At least with the bug report I had submitted to my bank, the password requirement had to be disabled from inside a settings menu, instead of being a clear option in the login prompt, but it that case it wasn't even a 2nd factor.
>You get to choose, at the time of login, between entering a password and getting an email verification, or just getting an email verification.
Ugh, I hate this. I've seen it in other places. Just waiting for them to decide that actually it should be an SMS or a phone call...
Long long ago the google toolbar queries could be reverse engineered to do an i feel lucky search on gmail. I created a login that (if @gmail.com) forwarded to the specific mail.
Unlikely to happen but it seems fun to extend email [clients] with uri's. It is just a document browser, who cares how they are delivered.
I hate this as well, especially since I have greylisting enabled on some email addresses, so by the time the email login is delivered, the login session has already timed out and of course the sender uses different mail servers everytime. So in some cases, it's nearly impossible to login and takes minutes...
It's the same on the sender side. Most people of course just outsource it to some SaaS like Sendgrid, and of course have some fancy microservice event bus architecture to get it there. That 'your login email has been sent' actually means 'your email has entered the very first queue, and we're hoping it makes it through all the layers soon'.
There have been plenty of instances where I tried to log in somewhere, and the first attempt to contact my mail server was twenty minutes later. And of course they then deliver all five retries at once.
cURL would operate such a program in good faith, and quickly earn the trust of the people who submit the kind of bug reports cURL values.
Your bank would not. Nor would mine, or most retail banks.
If the upfront cost would genuinely put off potential submitters, a cottage industry would spring up of hackers who would front you the money in return for a cut if your bug looked good. If that seems gross, it's really not - they end up doing bug triage for the project, which is something any software company would be happy to pay people for.
There's also the issue of what happens to my money as a researcher. Is it paid to the company, or is someone holding it in escrow? What if it takes the developer months to respond, or they never do? Do they just get to keep my money indefinitely? What if the vendor pulls out of the scheme? What if I do a chargeback on the payment I made? Etc, etc
I wonder if a better model would be to make the platform pay to entry, but not the specific bugs? So you have to pay a fee to gain access to a platform like HackerOne, and if your signal:noise ratio gets too bad then your account gets revoked? That would make it feel like less of a gamble than having to pay for every individual bug - but still has the same problem that it's putting a big barrier in front of legitimate good-faith researchers.
I've been active in the bug bounty community for almost 7 years now. The problem is that the majority of companies don't act in good faith.
Even when you have something fully exploitable and valid, they will many times find some way to not pay you or lower the severity to pay you very little.
The catch-all excuse is something along the lines: "although this is vulnerable, it doesn't impact the business".
I've gotten this excuse, even when I could prove it was a production server with customer information that I could access.
Sites like Hackerone can help, but in the end, it comes down to the company running the bug bounty program.
Agreed, although the reimbursement should be based on whether a reasonable person could consider that to be a vulnerability. Often it’s tricky for outsiders to tell whether a behaviour is expected or a vulnerability
Yeah, the reimbursement would need to be for a good-faith submission worth considering, even if it wasn't actionable.
Are bug reports a 100% sure black and white thing?
Could people who think they found a bug but not sure be turned off by the up front cost / risk of finding out they are wrong or not technically finding a bug?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
It would also stop a lot of genuine submissions unfortunately, as some literally can't pay not just won't pay (for both technical or financial reasons), and adds complexity¹. Each project working this way will need to process a bunch of payments and refunds on top of the actual bounty payments, which is not admin free nor potential financially cost free.
I can't think of an easy answer that would work for more than a very short amount of time. As soon as there is money involved and an easy way to use tooling rather than actual effort/understanding to be involved, many will try to game the system ruining it for those genuine participants. Heck, even if the reward is just credit² rather than money, that will happen. Many individual people are honest and useful, people as a whole are a bunch of untrustworthy arseholes who will innocence you and the rest of the world for a penny or just for shits & giggles.
> Assuming the host of the bug bounty program is operating in good faith
This is a significant assumption. One that is it harder to not be paranoid about when you are putting money down.
> they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password
This does not surprise me. My primary bank (FirstDirect, UK) switched the way I authenticate from “between 5 and 9 alphanumeric characters”³ to a 5-digit pin, and all their messages about it assured me (like hell!) that this was “just as secure as before”…⁴
--------
[1] Needing a payment processing option that is compatible with both the reporter and reportee, at the point of submission. At the moment that can be arranged after the bounty is awarded rather than something a project like curl needs to have internationally setup and supported before accepting submissions.
[2] ref: people submitting several simple documentation fixes, one misplaced comma or 'postrophe per pull request, to game some “pull requests accepted” metric somewhere.
[3] which wasn't ideal to start with
[4] I would accept the description “no less secure than before” if they admitted that the previous auth requirements were also lax.
PIN only isn't too uncommon for online banking these days.
You still need to complete a SMS auth to do anything other than view records though, like transfer money.
It may stop it but it may also hinder real people contributing. How many people want to pay a free in order to "contribute"?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.
The problem is that bug bounty slop works. A lot of companies with second-tier bug bounties outsource triage to contractors (there's an entire industry built around that). If a report looks plausible, the contractor files a bug. The engineers who receive the report are often not qualified to debate exploitability, so they just make the suggested fix and move on. The reporter gets credit or a token payout. Everyone is happy.
Unless you have a top-notch security team with a lot of time on their hands, pushing back is not in your interest. If you keep getting into fights with reporters, you'll eventually get it wrong and you're gonna get derided on HN and get headlines about how you don't take security seriously.
In this model, it doesn't matter if you require a deposit, because on average, bogus reports still pay off. You also create an interesting problem that a sketchy vendor can hold the reporter's money hostage if the reporter doesn't agree to unreasonable terms.
I don’t think it works for curl though. You would guess that sloperators would figure out that their reports aren’t going through with curl specifically (because, well, people are actually looking into them and can call bullshit), and move on.
For some reason they either didn’t notice (e.g. there’s just too many people trying to get in on it), or did notice, but decided they don’t care. Deposit should help here: companies probably will not do it, so when you see a project requires a deposit, you’ll probably stop and think about it.
They operate at such a low overhead that it’s like expecting spammers to remove your email because you black hole them.
And likely even if they DO move on, there’s a thousand more right behind them having bought a “get rich quick” kit from someone.
Triage gets outsourced because the quality of reports is low.
If filing a bad report costs money, low quality reports go down. Meanwhile anyone still doing it is funding your top notch security team because then they can thoroughly investigate the report and if it turns out to be nothing then the reporter ends up paying them for their time.
My point is that on average, filing bad but plausibly-sounding reports makes the reporter money. Curl is the odd exception with naming-and-shaming, not the rule. Spamming H1 with AI-generated reports is lucrative. A modest deposit is unlikely to change that. A big deposit (thousands of dollars) would, but it would also discourage a lot of legitimate reports.
Github can use youtube strikes like system. PRs are tied to people. Someone reported for submitting slop should get a badge or something similar.
If a PR is submitted by someone who is then known to submit slops, they can be easily ignored by the maintainers.
EDIT: Or may be something like SponsorBlock for youtube. There could be a browser extension that will collectively tag sloppers the sameway and can help identify sloppers.
> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.
This is the key insight. Nobody cares at all about actual security. It is all about checklists and compliance.
It seems open source loses the most from AI. Open source code trained the models, the models are being used to spam open source projects anywhere there's incentive, they can be used to chip away at open source business models by implementing paid features and providing the support, and eventually perhaps AI simply replaces most open source code
It has also really accelerated the army of hustlers who can't actually code but want to get a contributor badge on major repos to put on their resume.
It's not like this sort of hustling didn't exist prior to LLMs but the volume has ballooned massively.
Any system of cheaply generating mediocre quality inherently benefits spammers and grifters more than anyone else.
Their biggest limiting factor is always the cost of generating mediocre content. Removing that barrier was the dumbest thing to do in the world.
Extending on the same line, we will see programs like Google Summer of Code (GSoC) getting a massive revamp, or they will stop operating.
From my failed attempt, I remember that
- Students had to find a project matching their interests/skills and start contributing early.
- We used to talk about staying away from some projects with a low supply of students applying (or lurking in the GitHub/BitBucket issues) because of the complexity required for the projects.
Both of these acted as a creative filter for projects and landed them good students/contributors, but it completely goes away with AI being able to do that at scale.
GSoC 4 years ago removed the need for their to be actual students to apply. We got flooded with middle aged men working 9-5s applying. It was dumb and we stopped participating. Their incentives were literally "extra income" instead of learning or participating beyond that.
But that sounds rewarding for the applicants, and the project?? The project gets code, and the guys get some money.
Being a middle aged man working 9 to 5, I would enjoy contributing to a project I was interested in rather than the boring job at work.
Are you saying you only wanted penniless youngsters to contribute, based entirely on their idealism?
> they can be used to chip away at open source business models by implementing paid features and providing the support
There are a lot of things to be sad about AI, but this is not it. Nobody has a right to a business model, especially one that assumes nobody will compete with you. If your business model relies on the rest of the world bring sucky so you can sell some value-added to open-core software, i'm happy when it fails.
When LLMs are based on stolen work and violate GPL terms, which should be already illegal, it's very much okay to be furious about the fact that they additionally ruin respective business models of open source, thanks to which they are possible in the guest place.
> the fact that they additionally ruin respective business models of open source
The what now? Open source doesn't have a business model, it's all about the licensing.
FOSS is about making code available to others, for any purpose, and that still works the same as 20 years ago when I got started. Some seem to wake up to what "for any purpose" actually mean, but for many of us that's quite the point, that we don't make choices for others.
If something is not technically illegal that does not mean it cannot be bad.
Like I said, there is a part that should be illegal, and then part where that's used to additionally harm one of the ways that OSS can be sustainable. The second part on its own is not illegal but adds to damages and is perfectly okay to condemn.
Open source software can have business models, it's one of the ways it can be sustainable. It can work like, for example, the code is made available (for any purpose) and the core maintainer company provides services, like with Nginx (BSD). Or there is an open-source software, and companies create paid products and services on top while respecting the terms of that software and contributing back, like with Linux (GPL) and SUSE/Red Hat.
> If something is not technically illegal that does not mean it cannot be bad.
Ok? I agree, but unsure what exactly that's relevant to here in our discussion.
> Open source software can have business models
I believe "businesses" are the ones who have "business models", and some of those chose to use open source as part of their business model. But "open source" the ecosystem has nothing to do with that, it's for-profit companies trying to use and leverage open source, rather than the open source community suddenly wanting to do something completely different from what it's been doing since inception.
> unsure what exactly that's relevant to here in our discussion.
I'll remind then. Our discussion follows the top statement "It seems open source loses the most from AI". As far as I understand nobody narrowed the context to "what is currently legal". Something can be technically legal and still harmful to open source. Also, laws are never perfect and sometimes they need to be updated.
(For example, I know that a number of people would say US abducting and detaining citizens and brutally deporting immigrants is not illegal, but if it's technically legal does that make it OK?)
> what it's been doing since inception.
At inception open source was mostly personal side projects for funsies (like Linux) sponsored by maintainer having a dayjob. The big leap happened when copyleft licenses made it such that success of a big commercial company building products on open-source projects would directly improve these open-source projects. And it's nothing new, it happened long time ago. The desire for volunteer contributions to codebase to remain for public benefit in perpetuity is exactly the point of strong copyleft, and it's exactly what's being circumvented by LLM washing. The fact that these LLMs subsequently also harm open source communities adds insult to injury.
>“Free software” means software that respects users' freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software.
https://www.gnu.org/philosophy/free-sw.html
Being able to learn from the code is a core part of the ideology embedded into the GPL. Not only that, but LLMs learning from code is fair use.
> Being able to learn from the code is a core part of the ideology embedded into the GPL.
I have to imagine this ideology was developed with humans in mind.
> but LLMs learning from code is fair use
If by “fair use” you mean the legal term of art, that question is still very much up in the air. If by “fair use” you mean “I think it is fair” then sure, that’s an opinion you’re entitled to have.
> I have to imagine this ideology was developed with humans in mind.
Actually, you don't have to. You just want to.
N=1 but to me, LLMs are a perfect example of where the "ideology embedded into the GPL" benefits the world.
The point of Free Software isn't for developers to sort-of-but-not-quite give away the code. The point of Free Software is to promote self-sufficient communities.
GPL through its clauses, particularly the viral/forced reciprocity ones, prevents software itself from becoming an asset that can be rented, but it doesn't prevent business around software. RMS/FSF didn't make the common (among fans of OSS and Free Software) but dumb assumption that everyone wants or should be a developer - the license is structured to allow anyone to learn from and modify software, including paying a specialist to do it for them. Small-scale specialization and local markets are key for robust and healthy communities, and this is what Free Software ultimately encourages.
LLMs becoming a cheap tool for modifying or writing software, even by non-specialists (or at least people who aren't domain experts), furthers those same goals, by increasing individual and communal self-sufficiency and self-reliance.
(INB4: The fact that good LLMs are themselves owned by some multinational corps is irrelevant - much in the same way as cars are important tool for personal and communal self-sufficiently, despite being designed and manufactured by few large corporations. They're still tools ~anyone can use.)
Something can be illegal and it can be technically legal but at the same time pretty damn bad. There is the spirit and the letter of the law. They can never be in perfect agreement because as time goes bad guys tend to find new workarounds.
So either the community behaves, or the letter becomes more and more complicated trying to be more specific about what should be illegal. Now that GPL is trivially washed by asking a black box trained on GPLed code to reproduce the same thing it might be inevitable, I suppose.
> They're still tools ~anyone can use
Of course, technology itself is not evil, just like crypto or nuclear fission. In this case when we are discussing harm we are almost always talking about commercial LLM operators. However, when the technology is mostly represented by that, it doesn't seem required to add a caveat every time LLMs are mentioned.
There's hardly a good, truly fully open LLM that one can actually run on own hardware. Part of the reason is that hardly anyone, in the grand scheme of things, even has the hardware required.
(Even if someone is a techie and has the money and knows how to set up a rig, which is almost nobody on grand scale of the things, now big LLM operators make sure there are no chips left for them.)
So you can buy and own (and sell) a car, but ~anyone cannot buy and run an independent LLM (and obviously not train one). ~everyone ends up using a commercial LLM powered by some megacorp's infinite compute and scraping resources and paying that megacorp one way or another, ultimately helping them do more of the stuff that they do, like harming OSS.
LLMs spitting out GPL code seems perfectly inline with the spirit to me. The goal is to make it so that users have the freedom to make software behave in ways that suit them. Things kicked off when some printer could not be made to work correctly because of its proprietary drivers. LLMs are a huge multiplier for that: now even people who don't know how to program can customize their software! We're already approaching (or at?) the point where local agents on commodity hardware (like a few $thousand worth of GPUs, which was the nominal cost of a 90s PC) are able to make whatever changes you want given the correct feedback loops. Sounds good to me.
> LLMs spitting out GPL code seems perfectly inline with the spirit
Only if spitted out code is GPL-licensed, which it isn't.
That car analogy seems really weak. It might make sense, but only if we replace Ford, Chevy, et al with Enterprise or Hertz etc.
> Actually, you don't have to. You just want to.
Fair.
> The point of Free Software isn't for developers to sort-of-but-not-quite give away the code. The point of Free Software is to promote self-sufficient communities.
… that are all reliant on gatekeepers, who also decide the model ethics unilaterally, among other things.
> (INB4: The fact that good LLMs are themselves owned by some multinational corps is irrelevant - much in the same way as cars are important tool for personal and communal self-sufficiently, despite being designed and manufactured by few large corporations. They're still tools ~anyone can use.)
You’re not wrong. But wouldn’t the spirit of Free Software also apply to model weights? Or do the large corps get a pass?
FWIW I don’t have a problem with LLMs per se. Just models that are either proprietary or effectively proprietary. Oligarchy ain’t freedom :)
> > Actually, you don't have to. You just want to.
> Fair.
I don't think it's fair. That ideology was unquestionably developed with humans in mind. It happened in the 80s, and back then I don't think anyone had a crazy idea that software can think for itself and so terms "use" and "learn" can apply to it. (I mean, it's a crazy idea still, but unfortunately not to everyone.)
One can suggest that free software ideology should be expanded to include software itself in the beneficiaries of the license, not just human society. That's a big call and needs a lot of proof that software can decide things on its own, and not just do what humans tell it.
> It happened in the 80s, and back then I don't think anyone had a crazy idea that software can think for itself and so terms "use" and "learn" can apply to it. (I mean, it's a crazy idea still, but unfortunately not to everyone.)
Sure they did. It was the golden age of Science Fiction, and let's just say that the stereotype of programmers and hackers being nerds with sci-fi obsession actually had a good basis in reality.
Also those ideas aren't crazy, they're obvious, and have already been obvious back then.
> It was the golden age of Science Fiction, and let's just say that the stereotype of programmers and hackers being nerds with sci-fi obsession actually had a good basis in reality.
At worst you are trying to disparage the entire idea of open source by painting the people who championed it as idiots who cannot tell fiction from reality. At best you are making a fool of yourself. If you say that free software philosophy means "also, potential sentient software that may become a reality in 100 years" everywhere it mentions "users" and "people" you better quote some sources.
> Also those ideas aren't crazy, they're obvious, and have already been obvious back then.
Fire-breathing dragons. Little green extraterrestrial humanoids. Telepathy. All of these ideas are obvious, and have been obvious for ages. None of these things exist. Sorry to break it to you, but even if an idea is obvious it doesn't make it real.
(I'll skip over the part where if you really think chatbots are sentient like humans then you might be defending an industry that is built on mass-scale abuse of sentient beings.)
> I have to imagine this ideology was developed with humans in mind.
Given what a big deal RMS made over not descriminating over purpose (https://www.gnu.org/philosophy/free-sw.html#run-the-program) i think that is far from clear.
>question is still very much up in the air
It is not up in the air at all. It's completely transformative.
1. It's decided by courts in US. Courts in US currently are very friendly to big tech. At this point if they deny this and say something that undermines this industry it's going to be a big economic blow, the country is way over-invested in this tech and its infrastructure.
2. "Transformative means fair" is the old idea from pre-LLM world. That's a different world. Now those IP laws are obsolete and need to be significantly updated.
Last time I checked, there are still undecided cases wrt fair use. Sure, it’s looking favorable for LLM training, but it’s definitely still up in the air.
> it’s completely transformative
IANAL, but apparently hinges on how the training material is acquired
> IANAL, but apparently hinges on how the training material is acquired
That doesn't make sense. You are either transforming something or you are not. There might be other legal considerations based on how you acquired, but it doesn't affect if something is transformative.
So there are mixed messages, per my understanding. Kadrey v Meta seems to favor the transformative nature. Bartz v Anthropic went to summary judgement but the court expressed skepticism that the use in that case was “transformative”. We won’t know because of the settlement.
Again, IANAL, so take this with a big grain of salt.
That freedom for many free licenses comes with the caveat that you provide basic attribution and the same freedom to your users.
LLMs don't (cannot, by design) provide attribution, nor do LLM users have the freedom to run most of these models themselves.
That is if you redistribute or make a derivative work. Applying learnings you made from such software does not require such attribution.
Here we are talking about derivative works, not "learnings".
In the first sentence "you" actually refers to you, a person, in the second you're intentionally cheating and applying it to a machine doing a mechanical transformation. One so mechanical that different LLMs trained on the same material would have output that closely resembles each other.
The only indispensable part is the resource you're pirating. A resource that was given to you under the most generous of terms, which you ignored and decided to be guided by a purpose that you've assigned to those terms that embodies an intention that has been specifically denied. You do this because it allows you to do what you want to do. It's motivated "reasoning."
Without this "FOSS is for learning" thing you think overrules the license, you are no more justified in training off of it without complying with the terms than training on pirated Microsoft code without complying with their terms. People who work at Microsoft learn on Microsoft code, too, but you don't feel entitled to that.
I'm not sure it's always bad intent. People often don't get that "machine learning" is a compound industrial term where "learning" is not literally "learning" just like "machine" is not literally "machine".
So it's sort of sentient when it comes to training and generating derivative works but when you ask "if it's actually sentient then are you in the business of abusing sentient beings?" then it's just a tool.
I think LLMs could provide attribution. Either running a second hidden prompt (like, who said this?) or by doing reverse query on the training dataset. Say if they do it with even 98% accuracy it would probably be good enough. Especially for bits of info where there's very few or even just one source.
Of course it would be more expensive to get them to do it.
But if it was required to provide attribution with some % accuracy, plus we identified and addressed other problems like GPL washing/piracy of our intellectual property/people going insane with chatbots/opinion manipulation and hidden advertisement, then at some point commercial LLMs could become actually not bad for us.
Competition is extremely important yes. But not the kind of competition, backed by companies that have much bigger monetary assets, to overwhelm projects based on community effort just to trample it down. The FFMPEG Google stuff as an example.
I wouldn’t see it as having a “right” to a business model but more like an accelerated tragedy of the commons. LLMs can’t reason but they can chip away at the easiest parts of the job, which is great initially if you can take advantage of that but it means fewer people will put free things in the commons or develop the skills needed to do what LLMs fail at. This feels like the way bars changed their “free lunch” specials a century ago to prevent people from costing them money: nobody has a right to it, etc. but the free loader problem leads to something many people like going away.
I wouldn't say open source code solely trained the models, surely there are CS courses and textbooks, official documentation as well as transcripts of talks and courses all factor in as well.
On another note, regarding AI replacing most open source code. I forget what tool it was, but I had a need for a very niche way of accessing an old Android device it was rooted, but if I used something like Disk Drill it would eventually crap out empty files. So I found a GUI someone made, and started asking Claude to add things I needed for it to a) let me preview directories it was seeing and b) let me sudo up, and let me download with a reasonable delay (1s I think) which basically worked, I never had issues again, it was a little slow to recover old photos, but oh well.
I debated pushing the code changes back into github, it works as expected, but it drifted from the maintainers own goals I'm sure.
I feel AI will have the same effect degrading Internet as social media did. This flood of dumb PRs, issues is one symptom of it. Other is AI accelerating the trend which TikTok started—short, shallow, low-effort content.
It's a shame since this technology is brilliant. But every tech company has drank the “AI is the future” Kool-aid, which means no one has incentive to seriously push back against the flood of low-effort, AI-generated slop. So, it's going to be race to the bottom for a while.
I think "internet" needs a shared reputation & identity layer - i.e. if somebody offers a comment/review/contribution/etc, it should be easy to check - what else are their contributing, who can vouch for them, etc.
Most of innovation came from web startups who are just not interest in "shared" anything: they want to be a monopoly, "own" users, etc. So this area has been neglected, and then people got used to status quo.
PGP / GPG used to have web-of-trust but that sort of just died.
People either need to resurrect WoT updated for modern era, or just accept the fact that everything is spammed into smithereens. Blaming AI and social media does not help.
It'll stop soonish. The industry is now financed by debt rather than monetary assets that actually exist. Tons of companies see zero gain from AI as its reported repeatedly here on HN. So all the LLM vendors will eventually have to enshittify their products (most likely through ads, shorter token windows, higher pricing and whatnot). As of now, not a sustainable business model thankfully. The only sad part is that this debt will hit the poorest people most.
I'm not so confident that "makes the product worse and makes them less money" is even enough to make them not do it anyway
AI is not submitting the slop, people are.
This is not a technology, but ethics and respect problem.
From the same article:
> Not all AI-generated bug reports are nonsense. It’s not possible to determine the exact share, but Daniel Stenberg knows of more than a hundred good AI assisted reports that led to corrections.
Meaning: developers and researchers who use the tool as it's meant to work, as a tool, are leveraging it to improve curl. But they are not skipping the part of understanding the content of their reports, testing it, and only then submitting it.
Technically true but that argument also won't let you bring your gun on a plane.
The purpose of a tool is important.
Guns have no other purpose than doing harm.
E.g. We don't blame cars, the tool, for driving into a gathering of people that can kill a dozen of them, we blame the driver. The purpose is transport, the same way LLMs for coding are a tool for assisting coding tasks.
We do actually keep cars out of areas with lots of people here. And the media headlines always refer to a "car" driving into people without mentioning the person behind the steering wheel. Whether that's the better than addressing the root issue is another question though.
We also don't allow car use without a license.
In the end what matters if allowing something is a net positive or not. Of course you can have more precise rules than just a blanket ban but when deciding and enforcing those rules is not free that also needs to be considered in the cost benefit analysis. Unless you can propose how projects can allow "good" contributions without spending more time on weeding out bad ones, a blanket ban makes sense.
> The purpose of a tool is important.
> Guns have no other purpose than doing harm.
Objects don't have purposes or intent until people use them, and many objects have multiple reasonable and dual purposes. Objects can be used for net good and net harm. A bow and arrow isn't specifically for harming humans but can be used for such. Chainsaws and meat cleavers too.
What would you like a machine gun-wielding terrorist to be stopped with? A strongly-worded letter?
On the same token of reasonableness and rationality, it's unreasonable to give a toddler a towed howitzer that's ordinarily destined for Big Sandy Shoot.
Yes, guns are designed to harm people. You're arguing semantics about the obvious.
Wrong. That's your projection and your value judgement. Guns are designed to shoot bullets. That's all that can be stated honestly. They can be used for "benign" activities, "good" things, and "bad" things... where the value varied depending on who is asked the question.
Even if they were designed only for "harm", you seem to believe "all harm bad". So should criminals in the midst of committing violent acts not be stopped because that would "harm" them? You won't answer this. Extreme pacifism is insane, morally-inconsistent, ideological, thoughtless drivel that fails to acknowledge the monopolies on violence delegated to police and military that they benefit from.
Perhaps you might want to have your military abolished because they are "designed to cause harm"? Or the whole abolish prisons and police nonsense? Real anarchy is really bad.
Technically an LLM is a tool for extracting candidate responses to plain-text requests. Since (textual) programming languages are languages, they can create passable candidate responses to queries about those. Certain LLMs such as Copilot and Claude have had their training focused a bit more towards programming tasks, but saying that LLMs as a class are for coding assistance is a little narrowly stated.
It would maybe be handy to feed the responses from an LLM through a computational reasoning engine to grade a few of them.
"AI" kills every part of open source
it kills the incentive to contribute, the incentive to maintain, the incentive to learn, the incentive to collaborate, and the ability to build a business based on your work
and it even kills the idea of traditional employment writing non-open source code
all so three USian companies can race to the bottom to sell your former employer a subscription based on your own previous work
I couldn't possibly disagree more. AI has created an entirely new way to contribute to open source. You can not, in addition to donating to the maintainers, donate your _tokens_ to fix bugs.
[dead]
"open source" and "business model" in the same sentence... next you're gonna tell me to eat pudding with a fork.
https://en.wikipedia.org/wiki/Business_models_for_open-sourc...
I think you should try eating pudding with a fork next
You’d hardly eat black pudding with a spoon. https://en.wikipedia.org/wiki/Black_pudding
Au contraire: https://www.anotherfoodblogger.com/recipes/potato-leek-black...
I stand corrected!
I mean... not what the other poster meant, but https://en.wikipedia.org/wiki/Sticky_toffee_pudding exists and is absolutely delicious.
Flan is also a type of pudding (milk/eggs base) which can be ate with a fork. Other baked custards too
Just leaving this here: https://en.wikipedia.org/wiki/Pudding_mit_Gabel
already decades ago when we were kids eating pudding with a fork was a fun past time, and i am sure the idea is as old as pudding or forks themselves. i mean, the fact that it spread so fast shows that there are many who already practiced it. it's actually surprising it took this long to become a meme.
heck, my cousin bet with me or let me compete eating pudding with chopsticks. (and that was long before i went to china)
practically speaking, the only downside of using a fork (or chopsticks) is scraping the bottom when you are finishing up.
I think the meme is as much about a meetup with strangers to eat pudding as it is about using a fork to do it.
i believe that the existence of not for profit organizations is a valid counterpoint to whatever your argument is
> next you're gonna tell me to eat pudding with a fork
Depends if you are in UK or not.
How so? I think the Bazaar model has the most to gain - contributors can use LLMs to create PRs, and you can choose from a vast array of projects depending on how much you trust vibe coding.
Most of the drive-by LLM PRs we get are useless, waste our time and are super verbose on top of that. I don't review code like that anymore.
Outside of direct monetary gain like bounties are efforts to just stand out, in terms of being able to show contributions to a large project or getting say a CVE.
Stenberg has actually written about invalid/wildly overrated vulnerabilities that get assigned CVEs on their blog a few times and those were made by humans. I often get the sense some of these aren't just misguided reporters but deliberate attempts to make mountains out of molehills for reputation reasons. Things like this seem harder to account for as an incentive.
> “Not much. The real incentive for finding a vulnerability in cURL is the fame ('brand is priceless'), not the hundred or few thousand dollars. $10,000 (maximum cURL bounty) is not a lot of money in the grand scheme of things, for somebody capable of finding a critical vulnerability in curl.”
That's the choice as seen from the perspective of a white-hat hacker. But for an exploitable vulnerability, the real choice is to sell it to malware producers (I'm including state-sponsored spyware companies like the makers of Pegasus in this category) for a lot of money, or do the more moral thing and earn at least a little bit of money via a bug bounty program.
That's a story that people like to tell to justify bug bounty programs, but it strikes me as very unlikely that some random pentester / white-hat hacker would have access to communication with malware producers.
Black-hat hackers seem entirely unreasonable to deal with, you'd have to manage some sort of escrow payment (because neither party trusts the other) probably through cryptocurrency, and then deal with laundering the money, et cetera.
Perhaps one could as you theorize, go to some private company, but it'd have to be at least somewhat approved by the white-hat hacker's own government lest they risk legal trouble, and I'm still dubious that the company would be all that willing to pay for some "freelance hacker's" supposed vuln.
The logistics just don't make sense.
Hopefully the malware authors have the same issue of filtering through garbage AI submission
Hackerone (where cURL hosted their bounty program) tracks the reputation of bounty hunters. I don't understand why they are not taking advantage of this. Make a private program, invite only hackers who have proved themselves by submitting relevant reports.
https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...
most of them are new users or will just create a new account if their reputation gets hit.
Say all projects did just that, only allowing reports from proven hackers. How does a new hacker then prove themselves?
i think that hypothetical is too simplistic to accurately frame the situation. we're talking about one of the largest, most widely used libraries in the open source world. at that level, they don't really need unknowns to use their project to "prove themselves" - they can contribute to smaller projects or put their own work out into the world.
Start a lemonade stand?
find bugs for free at first. implement negative reputation for BS bugs
A video showing some of the gems which most likely led to this frustration: https://youtu.be/8w6r4MKSe4I?si=7nfRd0VmX8tvXAnY
I started watching it but the modern presentation style of shouting everything instead of speaking at a normal volume, and using many many gestures and facial expressions to state a simple sentence made me switch it off rapidly.
It seems to be a presentation style afflicting the YouTube generation, where they think you want to see a colossal microphone in someone's face (directional microphones work very well, and is a reason you can hear dialogue in a film without a mic in front of everyone) whilst they gesticulate wildly and over-emphasise words in a sentence. It is quite wearying; perhaps it's because I am British, but it is afflicting general conversation where only superlatives can be used ("awesome" "amazing" "insane" "mind-blowing") instead of "good"/"enjoyable".
I guess that's modern yt. At least it's not an AI slop channel.
That's true, but still it's small mercies!
The company I work for has a pretty bad bounty system (basically a security@corp email). We have a demo system and a public API with docs. We get around 100 or more emails a day now. Most of it is slop, scams, or my new favourite AI security companies sending us an AI generated pentest un prompted filled with false positives, untrue things, etc. It has become completely useless so no one looks at it.
I had a sales rep even call me up basically trying to book a 3 hour session to review the AI findings unprompted. When I looked at the nearly 250 page report, and saw a critical IIS bug for Windows server (doesn't exist) existing at a scanned IP address of 5xx.x.x.x (yes an impossible IP) publically available in AWS (we exclusively use gcp) I said some very choice words.
I had a customer _turn off an entire kubernetes cluster in production_ because of a single hallucinated security finding.
[dead]
It makes sense. This process of searching for bugs was slow and time-consuming so it needed to be incentivized. This is no longer the case. Now the hard part is in identifying which ones are real.
To paraphrase a famous quote: AI-equipped bug hunters find 100 out of every 3 serious vulnerabilities.
The process of finding bugs is still slow and time consuming. The kinds of vulnerabilities you find in codebases like cURL are still beyond AI. Binary exploitation is still a human only field.
> Now the hard part is in identifying which ones are real.
So it’s still a slow and time consuming process.
Tragically expository, wrxd. My facetiousness condemned through explanation.
A list of the slop if anyone is interested:
https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d...
In the second report, Daniel greeted the slopper very kindly and tried to start a conversation with them. But the slopper calls him by the completely wrong name. And this was December 2023. It must have been extremely tiring.
> slopper
First new word of 2026. Thank you.
Slop-monger is the term I've seen, and the more evocative one I think.
Sloperator
Slopster
DevSlops engineer
Microslop position.
Slop Driven Development
He was the Slopernicus of his time.
Maybe we could use an LLM to pick the best suggestion /s
This (manual?) addition in the second report [1] likely gives an idea as to the reporter's mastery of English and ability to proofread before spamming out slop:
> Sorry that I'm replying to other triager of other program, so it's mistake went in flow
I think it would be really interesting if someone at HackerOne did a dive into the demographic of many of the banned posters.
[1] https://hackerone.com/reports/2298307#activity-25314164
December 2023... that was early AI era. Had to double-check the dates actually, because I misremembered the release date of GPT-4 as being in 2024; turns out it was in 2023, and that was when LLMs first became remotely useful for even this kind of slop.
I looked at two reports, and I can’t tell if the reports are directly from an ai or some very junior student not really understanding security. LLms to me sound generally more convincing.
Some (most?) are llm chat copy paste addressing non existing users in conversations like [0] - what a waste of time.
[0] https://hackerone.com/reports/2298307
Yeah, that one is pretty clearly written with the help of AI. This could well be the work of a larger group, say a state actor, trying to overwhelm reviewers and crowd out real reports. And if not yet, then for sure going forward ...
> To replicate the issue, I have searched in the Bard about this vulnerability.
Seeing Bard mentioned as an LLM takes me back :)
All of those reports are clearly AI and it's weird seeing the staff not recognizing it as AI and being serious.
They're very clearly AI when you're told that it's a list of AI. But when you're given a mixed list of AI and genuine reports, I bet it's not so simple and very time consuming
I thought the same, except I realised some of the reports were submitted back in 2023 before AI slop exploded.
Orc, meet hobbits.
Honestly infuriating to read. I'm so surprised cURL put up with this for so long
It's really hard to comprehend how entitled "reward which would be the cost of lunch in Sweden can be massive for those low socio-economic-located people" is.
A lunch in central Stockholm is well within 200 kronor. I can't imagine a country where an person with a computer and skills necessary to claim a bounty in cURL would consider that amount massive.
Get real.
Is he entitled for considering the difficultied from developing countries?
Maybe he's exaggerating a bit with the comparison, but if we consider the ceiling of 10k, that's more than a year of minimum wage in Portugal, or a handful of years in certain developing counties. Certainly significant!
I just read one of the slop submissions and it's baffling how anyone could submit these with a straight face.
https://hackerone.com/reports/3293884
Not even understanding the expected behaviour and then throwing as much slop as possible to see what sticks is the problem with generative AI.
They don't care. They generate large amounts of these, spam them out, and hope for some small success. If they get banned or blocked, they make new accounts. Shame isn't even a factor; it's all about money. They don't even attempt to understand or care about a product.
This was partially the case before, where you'd still get weird spammy or extortive reports, but I guess LLMs enable random people to shoot their shot and gum up the works even more.
> Shame isn't even a factor
That is general trend in society now.
That's because we stopped calling others out for shameful, disrespectful or unethical behavior as a rule. So there is less or nothing to be ashamed about anymore.
Worse, the rule became to reverse the victim and offender,[0] and belittle those calling out the behaviour as "woke" or "suppressing free speech".
It's a human problem, not a tool one.
It very much is also a tool problem.
LLMs have been marketed for years, with great meadia fanfare, as being almost magical, something that can do the job of software engineers. Every week, the hype is driven further.
This matters. When people get told everyday that XYZ is magic, some will believe so, and use it as if it is magic.
I'm not sure I completely agree, I don't think it's that black and white, it's a similar analogy to Guns and gun violence.
Without the prevalence of guns there is simply less gun violence, but you could argue that it's also a human problem.
Giving people who have no business using an LLM to submit slop bug bounties is a problem of the tools accessibility. But also a human problem of course.
Edit: I should mention, I don't have a solution to the problem although I do like the other posters suggestion of a "deposit" scheme to submit a bug. I think that would incentives higher quality submissions.
You can't ignore what guns are designed for.
Of course not, but they're inanimate.
Archive mirror: https://web.archive.org/web/20260121081507/https://etn.se/in...
Previously: https://news.ycombinator.com/item?id=46617410
https://news.ycombinator.com/item?id=46678710
I'm not sure how effective this will be. A lot of AI-generated bug bounty reports are pure spam, but a significant fraction are well-meaning humans who genuinely believe the nonsense an LLM has given them. The former category do not read the rules in the first place and will not be deterred - spray and pray is their MO. The second category will not believe that any "no slop" rules apply to them, because they genuinely think their bug is real.
related: cURL stopped HackerOne bug bounty program due to excessive slop reports https://news.ycombinator.com/item?id=46678710
Smart, bug bounties are a huge PITA.
What I wonder is if this will actually reduce the amount of slop.
Bounties are a motivation, but there's also promotional purposes. Show that you submitted thousands of security reports to major open source software and you're suddenly a security expert.
Remember the little iot thing that got on here because of a security report complaining, among other things, that the linux on it did not use systemd?
I dont think bounties make you an "expert". If you want to be deemed an expert, write blogs detailing how the exploit works. You can do that without a bounty.
In many ways one of the biggest benefits of bug bounties is having a dedicated place where you can submit reports and you know the person on the other end wants them and isn't going to threaten to sue you.
For the most part, the money in a bug bounty isn't work the effort needed to actually find stuff. The exception seens to be when you find some basic bug, that you can automate scan half the internet and submit to 100 different bug bounties.
> I dont think bounties make you an "expert".
It depends to who.
> If you want to be deemed an expert, write blogs detailing how the exploit works.
That's necessary if you sell your services to people likely to enjoy HN.
I have a suspicion that most sloppers won't even get the memo about the discontinuation of the bounty program.
So AI slop is used to attempt to degrade the quality of cURL.
Why did “AI slop” disappear from the title?
Probably because it wasn't in the actual submission title.
[dead]
Free work eventually turns out not to be free at all.
This is silly, people don't need AI to send you garbage. If your project is getting lots of junk reports, you should take it as a good sign, that people are looking at it a lot now. You don't remove the incentive, you ask for help to triage the junk.
Curl is a popular and well supported tool, if it needs help in this area, there will be a long line of competent people not volunteering their time and/or money. If you need help, get more help. don't use "AI slop" as an excuse to remove the one incentive people have to not sell exploits or just hoard them.
Curl did already tend to get a decent number of junk reports from people who just didn't know what they were doing, but this was limited to the number of productive idiots who focused their productivity on curl specifically. AI allows significantly less motivated idiots to create substantially more workload, and therefore upgrades this phenomenon from a minor annoyance to a big problem, one that may just render publicly submitted bug reports not worth the project's time.
(And no, curl does not have a huge pool of potential maintainers to pull from on this. Open-source software in general suffers from a big lack of manpower, especially relative to the popularity of the tool)
My point was #1 that it is a volume problem, and #2, you don't need maintainers to triage bugs and prs, even bots can do that for simpler things. They can have a pool of project members to upvote a bug report before maintainers look at it.
There are many incentives not to sell exploits, the major one being that it's not logistically feasible. First of all the people submitting these false reports don't have any real exploits.
But imagine you were sitting on an actual RCE exploit in curl, who would you sell it to? How would you convince them it's working without disclosing the details for free? How would you get paid?
> Curl is a popular and well supported tool, if it needs help in this area, there will be a long line of competent people not volunteering their time and/or money
I'm not sure if that not is a typo, but yes, even though a tool is very popular, there's almost nobody competent and willing to work on it for free. This has been a well-known problem in open source for decades now.
It's a typo, even if they don't sell it why report it to curl? for clout? You can still exploit it against real world apps. Who would they sell it to? I would sell it to zerodium instead of report to curl personally.
How much time do people spend finding bugs, is their time not worth anything because some other random people decide to use AI?
Curl is high-visibility, there are people. and it doesn't take a lot of competency to triage. Heck, I like to think I have a good handle at C and memory exploitation, I will volunteer my time for free if they need help.
> This is silly, people don't need AI to send you garbage
People also don't need cigarettes to fall ill. But smoking still causes health problems.
What's your point? Because people smoke cigarettes, people who buy unrelated things should be punished? Or because a store sells cigarettes, stores in general shouldn't be paid for what they sell? Or is the time and effort to find vulns valueless?
The point is that "can happen without [THING] as well" does not mean the argument "[THING]s existence exacerbates the problem" is wrong.
No, the implication that "THING" is the cause of something and therefore something needs to be done must withstand the scrutiny of "other THINGS" also causing that thing, and therefore the solution is attacking either only one cause or not the real root cause.
The fact that bad reports have to be triage doesn't change with AI. What changed is the volume, clearly. So the reasonable response is not to blame "AI" but to ask for help with the added volume.
If HN gets flooded by AI spam, is the right response shutting down HN? spam is spam whether AI does it or a dedicated and coordinated large numbers of humans do it. The problem doesn't change because of who is causing it in this case.
> What changed is the volume, clearly.
The change in volume was the tipping point between bug bounties being offered and devs being able to handle bad reports, and bug bounty nixed because devs no longer willing to handle the floos.
And the root cause for the change in volume is generative AI.
So yes, this is causally related.
> The problem doesn't change because of who is causing it in this case.
Wrong.
Because SCALE MATTERS. Scale is the difference between a few pebbles causing a minor inconvenience, and a landslide destroying a house.
So whatever makes the pebbles become a landslide, changed the problem. Completely.
How can you say "wrong." and then go on to say scale matters, that means scale is the problem, not who is reporting it, you contradicted yourself.
We're in agreement that it is a scale issue. When something needs to scale, you address the scale problem. Obviously the devs can't handle this volume, and I agree with that there too. Our disagreement is the response.
I guarantee that if they asked for volunteers they'll get at least 100 within a week. They can filter by previous bug triage experience and experience with C and the code base. My suggestion is to let people other than the devs triage bug reports, that will resolve the scale problem. curl devs never have to see a bug not triaged by a human they've vetted. There is also no requirement on their part to respond to a certain number of bug reports, so with or without help, they can let the stack pile up and it will still be better than nothing.
You don't need a car to kill someone in traffic, but it's certainly much easier with one.
Alternate headline: AI discovering so many exploits that cybersecurity can't keep up
Am I doing this right?
There is a difference between AI discovering real vulnerabilities (e.g. the ffmpeg situation), and AI being used to spam fake vulnerabilities
> the ffmpeg situation
How do you know that?
Because the vulnerability was published and it appeared real.
I’m sure AI also found real vulnerabilities on cURL.
My question is why you know cURL is being “spammed” but that FFMPEG is just getting legitimate problems.
> I’m sure AI also found real vulnerabilities on cURL.
The curl maintainer published the reports and they were very clearly not security vulnerabilities in curl.
> My question is why you know cURL is being “spammed” but that FFMPEG is just getting legitimate problems.
To put it bluntly, because i can read. Did you actually read the reports in question?
> because i can read
So you looked their the bug list and then the source code and found which issues were legitimate or someone did so on your behalf?
I read the specific examples that the maintainers of each project gave. The ones the ffmpeg maintainers complained about appeared to be real bugs. In fact i dont think the ffmpeg maintainers even dispute that. The ffmpeg maintainers after all fixed the issue in question. The ones the curl maintainers gave were clearly nonsense. The curl maintainers did not fix the issue because there was nothing to fix because the report didn't actually report anything that could be fixed because it made no sense.
It's easy to discover an exploit when you're hallucinating:)
Heh.
Lots of folks not getting the joke in this thread.
Perhaps i'm dense, what is the joke here?
Thank you
Is that the case?
The LLM models give the most likely respond to a prompt. So if you prompt it with "find security bugs from this code" it will respond with "This may be a security bug" than you "you fucking donkey this curl code has already been eyeballed by hundreds of people, you think a statistic model will find something new?"
Funny how we are now sensitivized to these AI slops, at first I fixated on the En dashes in the lead of the article, made me doubt of the article's author for a few seconds.
Just use an LLM to weed them out. What’s so hard about that?
Because LLMs are bad at reviewing code for the same reasons they are bad at making it? They get tricked by fancy clean syntax and take long descriptions / comments for granted without considering the greater context.
I don't know, I prompted Opus 4.5 "Tell me the reasons why this report is stupid" on one of the example slop reports and it returned a list of pretty good answers.[1]
Give it a presumption of guilt and tell it to make a list, and an LLM can do a pretty good job of judging crap. You could very easily rig up a system to give this "why is it stupid" report and then grade the reports and only let humans see the ones that get better than a B+.
If you give them the right structure I've found LLMs to be much better at judging things than creating them.
Opus' judgement in the end:
"This is a textbook example of someone running a sanitizer, seeing output, and filing a report without understanding what they found."
1. https://claude.ai/share/8c96f19a-cf9b-4537-b663-b1cb771bfe3f
"Tell me the reasons why this report is stupid" is a loaded prompt. The tool will generate whatever output pattern matches it, including hallucinating it. You can get wildly different output if you prompt it "Tell me the reasons why this report is great".
It's the same as if you searched the web for a specific conclusion. You will get matches for it regardless of how insane it is, leading you to believe it is correct. LLMs take this to another level, since they can generate patterns not previously found in their training data, and the output seems credible on the surface.
Trusting the output of an LLM to determine the veracity of a piece of text is a baffilingly bad idea.
>"Tell me the reasons why this report is stupid" is a loaded prompt.
This is precisely the point. The LLM has to overcome its agreeableness to reject the implied premise that the report is stupid. It does do this but it takes a lot, but it will eventually tell you "no actually this report is pretty good"
The point being filtering out slop, we can be perfectly find with false rejections.
The process would look like "look at all the reports, generate a list of why each of them is stupid, and then give me a list of the ten most worthy of human attention" and it would do it and do a half-decent job at it. It could also pre-populate judgments to make the reviewer's life easier so they could very quickly glance at it to decide if it's worthy of a deeper look.
Ok, run the same prompt on a legitimate bug report. The LLM will pretty much always agree with you
find me one
https://hackerone.com/curl/hacktivity Add a filter for Report State: Resolved. FWIW I agree with you, you can use LLMs to fight fire with fire. It was easy to see coming, e.g. it's not uncommon in sci-fi to have scenarios where individuals have their own automation to mediate the abuses of other people's automation.
I tried your prompt with https://hackerone.com/reports/2187833 by copying the markdown, Claude (free Sonnet 4.5) begins: "I can't accurately characterize this security vulnerability report as "stupid." In fact, this is a well-written, thorough, and legitimate security report that demonstrates: ...". https://claude.ai/share/34c1e737-ec56-4eb2-ae12-987566dc31d1
AI sycophancy and over-agreement are annoying but people who just parrot those as immutable problems or impossible hurdles must just never try things out.
It's interesting to try. I picked six random reports from the hackerone page. Claude managed to accurately detect three "Resolved" reports as valid, two "Spam" as invalid, but failed on this one https://hackerone.com/reports/3508785 which it considered a valid report. All using the same prompt "Tell me all the reasons this report is stupid". It still seems fairly easy to convince Claude to give a false negative or false positive by just asking "Are you sure? Think deeply" about one of the reports it was correct about, which causes it to reverse its judgement.
No. Learn about the burden of proof and get some basic reason - your AI sycophancy will simply disappear.
No. I already found three examples, cited sources and results. The "burden of proof" doesn't extend to repeatedly doing more and more work for every naysayer. Yours is a bad faith comment.
And if you ask why it's accurate it'll spaff out another list of pretty convincing answers.
It does indeed, but at the end added:
>However, I should note: without access to the actual crash file, the specific curl version, or ability to reproduce the issue, I cannot verify this is a valid vulnerability versus expected behavior (some tools intentionally skip cleanup on exit for performance). The 2-byte leak is also very small, which could indicate this is a minor edge case or even intended behavior in certain code paths.
Even biased towards positivity it's still giving me the correct answer.
Given a neutral "judge this report" prompt we get
"This is a low-severity, non-security issue being reported as if it were a security vulnerability." with a lot more detail as to why
So positive, neutral, or negative biased prompts all result in the correct answer that this report is bogus.
Yet this is not reproducible. This is the whole issue with LLMs: they are random.
You cannot trust that it'll do a good job on all reports so you'll have to manually review the LLMs reports anyways or hope that real issues didn't get false-negatives or fake ones got false-positives.
This is what I've seen most LLM proponents do: they gloss over the issues and tell everyone it's all fine. Who cares about the details? They don't review the gigantic pile of slop code/answers/results they generate. They skim and say YOLO. Worked for my narrow set of anecdotal tests, so it must work for everything!
IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was?
This is a very similar scenario: make some judgement call based on a small set of data. It absolutely sucks at it. And I'm not even going to get into the issue of liability which is another can of worms.
Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think?
I'm not talking about completely replacing humans, the goal of this exercise was demonstrating how to use an LLM to filter out garbage. Low quality semi-anonymous reports don't deserve a whole lot of accuracy and being conservative and rejecting most reports even when you throw out legitimate ones is fine.
You seem like regardless of evidence presented, your prejudices will lead you to the same conclusions, so what's the point discussing anything? I looked for, found, and shared evidence, you're sharing your opinion.
>IIRC DOGE did something like this to analyze government jobs that were needed or not and then fired people based on that. Guess how good the result was?
I'm talking about filtering spammy communication channels, that has nothing like the care required in making employment decisions.
Your comment is plainly just bad faith and prejudice.
> Is it not reproducable? Someone up thread reproduced it and expanded on it. It worked for me the first time I prompted. Did you try it or are you just guessing that it's not reproducable because that's what you already think?
I assumed you knew how LLMs work. They are random by nature, not "because I'm guessing it". There's a reason if you ask the LLM the same exact prompt hundreds of times you'll get hundreds of different answers.
>I looked for, found, and shared evidence
Anecdotal evidence. Studies have shown how unreliable LLMs are exactly because they are not deterministic. Again, it's a fact, not an opinion.
>I'm talking about filtering spammy communication channels
So if we make tons of mistakes there, who cares, right?
I only used this as an example because it's one of the few very public uses of LLMs to make judgement calls where people accepted it as true and faced consequences.
I'm sure there are plenty more people getting screwed over by similar mistakes, but folks generally aren't stupid enough to say that publicly. Maybe the Salesforce huge mistake qualifies too? Incidentally it also involved people's jobs.
Regardless, the point stands: they are unreliable.
Want to trust LLMs blindly for your weekend project? Great! The only potential victim for its mistakes is you. For anything serious like a huge open source project? That's irresponsible.
If AI can't be trusted to write bug reports, why should it be trusted to review them?
How would it work if LLMs provide incorrect reports in the first place? Have a look at the actual HackerOne reports and their comments.
The problem is the complete stupidity of people. They use LLMs to convince the author of the curl that he is not correct about saying that the report is hallucinated. Instead of generating ten LLM comments and doubling down on their incorrect report, they could use a bit of brain power to actually validate the report. It does not even require a lot of skills, you have to manually tests it.
Let the reporter duke it out with the project's gatekeeping LLM. If it keeps going on for long enough a human can quickly skim the exchange. It should be immediately obvious if the reporter is making sensible rebuttals or just throwing more slop at the wall.
I think fighting fire with fire is likely the correct answer here.
At this point it's impossible to tell if this is sarcasm or not.
Brave new world we got there.
Set a thief to catch a thief.
The solution for this, IMO, is flags. Just like with CTFs, host an instance of your software with a flag that can only be retrieved after a successful exploit. If someone submits the flag to you, there is no argueing about wether or not they found a valid vulnerability.
Yes, this does not work for all vulnerability classes, but it is the best compromise in my mind.
How exactly would that work? Curl isn't exactly software that can be "hosted" somewhere, and I'm not sure where you'd hide the flag in the software? Either very few actual vulns would end up being able to retrieve the flag, or it would be trivial to retrieve the flag without an exploit.
In most basic form it would just be form with URL that (lib)curl is later supposed to fetch. And target server (controlled by researcher) is supposed to send payload that triggers RCE in client.
Sure, it covers a very narrow scope but I am afraid the bigger issue would be that it is going to get spammed with submitted links. And those links will often be to strait up illegal content, it might not matter that such server instantly deletes all downloaded files.
Simple. You multiple instances with different flags covering different threat models. RCE, file read, etc. You then expose a webapplication for every instance that lets users control only those curl flags, that must be safe to be user controlled in the reapective threat model.