I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data.
I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour since we had overlapping results and different methods, specifically different assumptions.
I say all that to say, as a physics student getting my undergrad, simply doing independent research and speaking to experts about it enabled me to network with someone I otherwise likely wouldn’t know. For young people getting into any business, research is a great way to meet new people.
That's also a reason the big labs stopped. Publishing is most valuable to people who have no other way to get the attention of smart strangers. Once you can hire nearly anyone and everyone already returns your calls, the main remaining effect of publishing is to tell your competitors which things worked.
This is what happens to every field as it turns from a science into an industry. Chemists published freely until dyes started being worth money, and then the interesting work moved into company labs and stopped coming out.
Which is a very ironic and selfish situation when your business model dependent mostly on model training based on available published data, academic and non-academic.
That has nothing to do with anything. If you publish a copyleft paper, that doesn't compel someone who makes a product based on your paper to publish more papers.
The GNU RPL (research public license), a viral knowledge license. By reading this paper you are legally obligated to openly publish all vaguely related future research that you perform.
Your RPL wouldn't be enforceable. Copyright doesn't deal with abstract ideas passing through people's minds. Even the GPL is kind of in a gray area because the virality feature and its definition of "derivative work" have never been tested in court, to my knowledge. Maybe under contract law, no idea. If nothing else, I'd love to hear a verdict.
Well it was a joke and is obviously quite silly but I believe it would be enforceable to the extent that the licensor could terminate the agreement and sue for damages. If I can agree to pay you not to talk about something (ie an NDA) or not to work in a field (ie a non-compete clause) then why can't I pay you to be required to publish all future work you do in a given area? ("All future work" might well be overly broad; you'd likely have to restrict it to only a few years but I'm no lawyer.)
I don't believe the GPL is in a grey area. When you license a work all sorts of strings are commonly attached. Rather IIUC no one has gone out of the way to test the GPL largely because it is clearly within bounds, plus any violation has the potential to be a PR disaster since violations are quite literally examples of looting the commons.
Meaning what? Claw back the ideas from people's minds? You can terminate the agreement in the sense that you revoke access to the paper, but presumably the person you find in breach has already used the research for something that you find them in breach for. You're kind of closing the gate after the horse has bolted.
>sue for damages
I honestly have no idea what damages you could claim from not publishing research. I think you would need to set a value ahead of time on the agreement.
>If I can agree to pay you not to talk about something (ie an NDA) or not to work in a field (ie a non-compete clause) then why can't I pay you to be required to publish all future work you do in a given area?
Not sure why you added the word "pay" to your clauses, but anyway. The reason is that the existing contracts have well-defined boundaries. An NDA stops you from divulging a very specific set of information. A non-compete clause stops you from working in a very specific field. Your proposal has an undefined reach. What counts as research? What counts as "related"? It would seem that if I agree to such a contract, my entire life, both private and professional, after reading the paper is covered by the contract, and anything I do might come under scrutiny. There's never a point when I can go off-duty. "What's that? You read my paper on compression and were working on a side-project that uses compression? Gonna have to see some publication on it."
>Rather IIUC no one has gone out of the way to test the GPL largely because it is clearly within bounds
No, it's because the status quo is convenient and no one wants to be the first guinea pig. It's definitely not obvious that the terms are legally valid, but it's ambiguous enough that people don't want to test it.
If something is so obviously wrong then perhaps take a minute to consider that your interpretation isn't what the other party intended?
If I pay you not to do something and then you breach the contract I can terminate the agreement and seek damages. Ditto if I pay you to repeatedly do something and then at some point you fail to do it. So if I pay you a recurring fee to publish all your research on a given topic and then you fail to make good on that I can seek damages, right? Now what if I paid you a lump sum up front? Now what if I licensed a patent to you in place of that lump sum? What if instead of a patent it was the right to make use of a piece of software?
> I honestly have no idea what damages you could claim from not publishing research.
Aside from whatever was stipulated for breach of contract I expect it would be extremely situational. I agree that you'd probably want to stipulate a penalty ahead of time. The original idea was a two sentence joke after all ...
> Not sure why you added the word "pay" to your clauses
Because contracts involve consideration and payment is easy to understand and reason about. FOSS software licenses obviously substitute "right to use the code" for "payment".
> A non-compete clause stops you from working in a very specific field. Your proposal has an undefined reach. What counts as research? What counts as "related"? It would seem that if I agree to such a contract, my entire life, both private and professional, after reading the paper is covered by the contract, and anything I do might come under scrutiny.
This is incredibly contrived. You could ask the equivalent about a non-compete. The reach is whatever is defined in the contract that both parties agreed to. If the contract stipulates something overly broad then possibly a judge would invalidate it. This is business 101.
> It's definitely not obvious that the terms are legally valid, but it's ambiguous enough that people don't want to test it.
An assertion entirely without evidence from my perspective. I'm going to assume that all the lawyers who have sure left me with the impression that it would be a bad idea to violate it know what they're talking about.
>So if I pay you a recurring fee to publish all your research on a given topic and then you fail to make good on that I can seek damages, right? Now what if I paid you a lump sum up front? Now what if I licensed a patent to you in place of that lump sum? What if instead of a patent it was the right to make use of a piece of software?
Uh-huh... This doesn't answer my question of what terminating the agreement of access to the paper does, besides what I've already said. You've licensed to me access to a paper under certain conditions. I've breached the conditions, therefore you terminate the agreement, therefore you revoke access. Am I missing anything?
>FOSS software licenses obviously substitute "right to use the code" for "payment".
Hence my question. The hypothetical license/contract under discussion is about access to research results, not about a monetary transaction.
>This is incredibly contrived.
Well, the idea of viral abstract ideas is stupid, so it forces me to give contrived examples.
>I'm going to assume that all the lawyers who have sure left me with the impression that it would be a bad idea to violate it know what they're talking about.
What point do you think you're making? Something can be ambiguously (but not certainly) risky and a bad idea to do. I have two coins, one with two tails and the other a fair one, and I offer you to gamble everything you own on one of these coins of your choosing, or walk away. I assume you wouldn't pick the unfair one. Therefore if you would rather walk away than gamble everything you own on the normal coin, the toss actually has a 100% chance of you losing?
> I've breached the conditions, therefore you terminate the agreement, therefore you revoke access. Am I missing anything?
You're missing the part where I seek punitive and actual damages under the terms of the contract. No different than violating an NDA - I paid you a lump sum up front, after a while you breached the contract, the agreement is null and void, what's the consequence?
> Well, the idea of viral abstract ideas is stupid, so it forces me to give contrived examples.
On the contrary, presumably it was because you lacked the ability to roundly refute anything I had put forward. Otherwise I assume you would have done so.
> Therefore if you would rather walk away than gamble everything you own on the normal coin, the toss actually has a 100% chance of you losing?
But in this analogy it is you baselessly making that claim. There's every expectation that it's a fair coin, many experts have carefully inspected it and authored opinions on it, and some have put forward theories that it slightly deviates in one direction or another. Then you show up and confidently assert without any evidence that there's some wild deviation from fair, hand waving that you would have proof if only someone wanted to bother testing it.
Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
You keep handwaving away any requests for something concrete to discuss. If it’s so easy to construct a viral license for access to science, just give an example of it.
> Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
The only example of them is open source that is routinely violated because it’s so weak in practice. LLMs reproduce it frequently with no attribution and nobody is successfully suing them over this.
>I seek punitive and actual damages under the terms of the contract.
Right, so you're agreeing with my original interpretation of "termination".
>You can terminate the agreement in the sense that you revoke access to the paper
The termination and the lawsuit for damages are separate events. Why did you accuse me of being purposefully dense if you're agreeing with me?
>But in this analogy it is you baselessly making that claim.
We're both making baseless claims. I said A, and then you said ¬A, and neither one has backed anything up. I don't even necessarily believe you talked to any lawyers, I just granted it for the sake of argument because the statement you made was so weak that I didn't need to cast doubt on it.
Also, in case you missed it, the fair coin in the analogy represents the case where the GPL might not be fully valid, not the other way around. If the coin is unfair then the GPL is fully and obviously valid, and if you get sued and you did infringe it then you're certain to lose.
>Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
Since I'm not the topic of discussion, I won't answer these questions beyond saying that I didn't opine on viral licenses.
This reads like a company requiring you to hand over your first-born kid in the TOS. I would happily be the test case and violate your license if you want to sue me!
To paraphrase your last paragraph, an idea that hasn’t been tested is either perfect, or just bad enough that no one wants to test it, as, as you say, testing it is would be at least somewhat inconvenient.
but you patent something requires proving that you have done something novel. You're effectively asserting that the thing you invented is new and different enough to be not a derivative of a previous patent.
sure to build a new system, you might need other patents. but a pateent it's self should be unique enough to defend. Which means its copyrightable as its own thing
An idea can be both novel and derivative at the same time.
You create idea B.
It absolutely builds upon idea A, and is in some sense derivative.
But your idea B introduces a lot of new novel concepts on top of idea A that makes it novel.
An idea can absolutely be both novel and derivative at the same time. Most of the best ideas are.
For a hypothetical example, an innovative Linux filesystem requires having the Linux OS, or similar, as a base to operate. In some sense, it is derivative of that prior work. But the new ideas extend the old ideas enough to be novel.
From: rms@prep.ai.mit.edu (Richard M. Stallman)
Date: 25 June 1986 at 18:45:28 GMT+2
To: a2deh@ai
I figured out what a copyleft is.
It's the sort of thing that I put on GNU software and manuals.
The left-wing version of the copyright that rightists use
to exploit the masses.
But copyrights are not patents, and the opposite of "patent" is "latent", so we need a Latent Office that protects inventions from the people who claim to have invented them, and issues warnings to patent trolls:
LATENT (L)
All rights unreserved.
This invention may already exist, but hasn’t become obvious yet.
Latentleft — publish the invention, conceal the monopoly.
Latent Pending — the idea is out there somewhere.
Latent Troll — remains dormant until an industry becomes profitable.
The patent system is a pretty useful compromise here. The applicants are forced to disclose all knowledge required to practice the craft. They get a 20 year monopoly in return.
I realize everyone always wants everything for free, but if you want someone to put the energy into writing up their invention you need to give them an incentive.
I don't understand why from a legal perspective, but it seems like over the last decade or more the tech industry stopped caring about patents. Startups used to talk about their parents and large tech companies would encourage you to file. While people say software patents make no sense, when I've seen cases involving EE, ME, and software patents, the software patents are usually the most likely to hold up in court and get damages.
Everyone says this in the abstract but to concrete examples they shug and say, of course that approach doesn't work, they did X, Y, Z wrong, they should have given it more effort, it could have worked if done properly / this can obviously never work, everyone knew already, it's nothing new etc.
It would need to be beautifully done science - indeed the level of rigour required to confidently show no effect is probably quite a bit larger than the level of rigour required to confidently show a large effect.
And that's probably one of the main reasons it's not done more often.
Isn't that exactly the point? If in my conception that concrete example eliminates that single branch, then I can try the others, and only the others. Or confirm the negative. Which is also valid work.
In general, your point stands for armchair researcher.
No, good researchers also dismiss such things, because most research out there is crap. If you reward negative results, you'll have a flood of them, and the reason that they are negative will mostly look like it's incompetence (and mostly will be, but even the small not-incompetent will be hard to distinguish).
It's asymmetric. If you make something work, beat a benchmark, invent a drug that works etc, the exact way you arrived there has some leeway. In the end, the thing worked out so it's worth knowing about.
If your project failed, there can be a million reasons for that, and it's not necessarily that the initial hypothesis or initial idea has been refuted. E.g. in ML, your model didn't learn the task. Okay, there could be a million knobs (hyperparameters) that you set up wrong, or you implemented it wrong, or you should have just added learning rate warmup, or this or that, a million things possible. People fail all the time at projects that others then manage to do later on. Science is not like simply asking the universe some clear question and getting a clear answer. It's a very messy process and even professionals are not super great at it, or at least they simply cannot afford to put so much effort in each single project to make it absolutely airtight such that the failure to make it work can be a legit refutation of the main idea.
> I wish more people would publish the things that didn't work.
This would be a pointless endeavour. One of the most basic mantras of science is "absence of evidence is not evidence of absence". So just because something didn't worked out for you that doesn't mean it doesn't work out for others, or even yourself in the future.
You are talking about a different thing, i.e. you have slipped in a level of abstraction that was not there before:
In 'searching a path from A to B in a maze' language:
The original statement was: (1) The branch to the left from A is a dead end.
Your interpretation: (2) There is no path from A to B.
(1) is still very useful (reducing the wasted effort) for those trying to find a path from A to B. The OP's point is that in the current environment only positive results are rewarded (I found the path from A to B!), not the negative ones like (1).
> The original statement was: (1) The branch to the left from A is a dead end.
This is where you get things wrong at a very basic and fundamental level.
Just because you failed to explore branch A, that does not mean it is a dead end. It just means you came up empty.
That is why science is based on observations and theories: it is based on building up on ideas and what works and can be proven. Otherwise you will left with useless papers such as "Bicycles are a dead end because I tried to ride one and I fell".
You are assuming incompetence on the scientist saying that the branch from the left of A is a dead end.
While nobody is perfect, there are numerous perfectly valid scientific negative results. You know, there exist things like impossibility proofs in mathematics and computer science. There are equivalents in other sciences (e.g. if X was true, that would lead to Y that is easily observable and clearly not observed). Sometimes that implication has assumptions that might change once the technology/society changes, other times it holds true regardless.
Unicycles are a dead end as a practical transportation, because the bicycles have them beat in every way (except portability).
A scientific result would be much more along the lines of 'Bicycles without gears have limited applicability, especially in hilly terrain.'
There are different types of research. Negative AI results are not like Michelson-Morley experiment in physics that can positively establish the lack of something. It's more of a craft, the "make it work" style of thing. Give the same task or goal to two different teams, you will see massively different outcomes. Simply failing to make it work is not conclusive.
To make such a negative result acceptable in AI, you'd have to have some clear reason why you think that your particular setup should produce the result you want, that exact configuration and architecture, dataset etc. There are countless projects in AI that fail. And it's not clear at all that it refutes any abstract hypothesis. It's a get-your-hands-dirty field. It can make or break a project whether someone has that tacit knowledge, that black magic experience to know how to properly do the project.
People can generate extremely many ideas. You'd need to convince me that your idea (among a million others that people are trying each day) is so significant that its failure is in itself interesting. If you were to review for AI conferences, you'd see the flood of papers that claim to achieve 0.5% or 1% improvement on some benchmark. Now imagine that they didn't even have that to show for it. It got worse by 2% after trying their random idea. Who cares then? Even the +1% with a random idea is quite annoying to accept. But if their random idea really made something work much better, I will at least have some reason to want to see what may be going on there, there can be some signal. With negative results, it's very uninteresting.
I did read OP's post much more generally than only about AI research. There are many research areas where (properly done!) negative results would be very valuable, but are not properly encouraged.
I do agree that in a new area with too many degrees of freedom (and yes, AI research is one of those), negative results (especially poorly done) are of limited usefulness.
> Chemists published freely until dyes started being worth money
Notably this is exactly what patents are intended to combat. And while US IP law is clearly very broken it does at least largely accomplish this stated goal. Much (but certainly not all) industrial chemistry has made it into the academic literature.
Not that the same logic necessarily applies to AI research (ie algorithms aka math and their implementations). And I'm actually happy about that because the cost of doing the research is so much lower. There's a long list of reasons that the average person living in a residential area can't do industrial chemistry as a hobby.
To your dye example, yttrium indium manganese blue was the first commercially viable inorganic blue pigment discovered in ~200 years, is the only known environmentally safe one, and was openly published in the literature. It's also under an exclusive license. (TBF though unless the chemical is unusually difficult to synthesize not publishing would be rather pointless in this day and age given the utterly absurd capabilities of modern analytical techniques.)
It depends on the field, and to some extent on the particular patent officer assigned, but one of the failings of the US patent system is many patents don't actually provide enough details to reproduce the result. And even if they do they don't (usually) include ancillary information like the discovery process, properties of the invention etc.
>> To your dye example, yttrium indium manganese blue was the first commercially viable inorganic blue pigment discovered in ~200 years, is the only known environmentally safe one, and was openly published in the literature. It's also under an exclusive license.
Gee, I wonder what was wrong with the previous blue pigments and why it was so important to have this one under an exclusive license.
Cobalt blue is a blue pigment made by sintering cobalt(II) oxide with aluminium(III) oxide (alumina) at 1200 °C. Chemically, cobalt blue pigment is cobalt(II) oxide-aluminium oxide, or cobalt(II) aluminate, CoAl2O4. Cobalt blue is lighter and less intense than the (iron-cyanide based) pigment Prussian blue.
Cobalt poisoning sounds scary, but I see nothing on the linked Wikipedia page that would present a risk of accidental consumption of medically relevant amounts of cobalt, whether in one sitting or through prolonged exposure.
Though the mention of Bolesławiec makes me worried a little; my wife loves their designs and we have a bunch of plates and bowls and such from them, and we serve food on/in them regularly...
> my wife loves their designs and we have a bunch of plates and bowls and such from them, and we serve food on/in them regularly...
I don't think you need to worry about that. The pigment is already reasonably benign when in solid form; once embedded in or under a glaze I'm not aware of anything that would suggest it carries any health risks whatsoever.
That said I'm unclear how safe direct exposure to the pigment itself is (such as when suspended in a liquid for painting). There's not a lot of data available that I could find, and of course due to having a unique crystalline structure it won't necessarily have the same properties as the component products, however cobalt(II) oxide itself is extremely hazardous which is at least cause to be cautious.
Other than manufacturing safety primarily the new pigment is just incredibly vivid.
>> That said I'm unclear how safe direct exposure to the pigment itself is (such as when suspended in a liquid for painting). There's not a lot of data available that I could find, and of course due to having a unique crystalline structure it won't necessarily have the same properties as the component products, however cobalt(II) oxide itself is extremely hazardous which is at least cause to be cautious.
The consensus is that artists' colours are hazardous waste that needs safe disposal. You can find instructions how to do that in many arts supplies' sellers sites and the like, for example one I've used:
The comment about not licking your brushes is a reference to the common practice of licking them in the wargaming miniatures painting community (you know, Warhammer and all that). You can find videos of influencers explicitly telling you that the best way to shape the tip of your brush or remove excess water is to lick it. For example:
If you ask for a rationale it's usually that wargaming and generally hobbyist paints are non-toxic, but the truth is that hobby paint manufacturers never list their paints' ingredients so there is no way to know. And of course there's different degrees of toxicity, not everything needs to be cobalt(II) oxide-level toxic to hurt you if you consume "large enough" quantities of it; the question is what that "large enough" means and without knowing the substance, there's no way to know.
The startup landscape has also changed noticeable compared to 5 or 10 years ago. A team of smart/credible people could get funding for an idea and build a product + publish, knowing there was a six month lead time for anyone to copy them and ship. These days, the barrier to ship code is zero. People can copy your business over a weekend, so there is much more urgency to establish product market fit and build a “moat”. Publishing timelines are now at odds with the pace of go-to-market and VC funding timelines.
I think in theory this is true but in practice I don't see loads more good apps or products. There's a paradox here I think. I just don't see loads of quality competitors popping up I actually think it makes building something harder because the barrier to entry just gets higher somehow.
I think the revolution is more of a gradual thing than an overnight one. New features and more complex apps in days or weeks instead of months. I think the world will be a much different place in 10 years than it is now, but it will be a gradual change rather than the overnight switch flip some AI maximalists would have you believe.
At my company, AI has had a huge positive impact in helping us manage technical debt that we just didn't have time to deal with before. This simple thing will have a compounding efficiency and profitability effect over the next several years.
The other thing that we have found is that AI coding abilities can crank out features faster than we can provide human support to our customers using said features. We have feature PRs that have been open for months without being merged because our company does not have the human bandwidth to provide support for them. In case you can't tell, we prioritize human support and individual connection with our customers. We actually value them.
As with all human endeavours, as the tooling gets better the expectation for a launch/product/feature increases dramatically too. I probably do 3-5x the features I used to before AI, yet there's still more to do. I'm not running out of work but making incredible progress on features that save the place I work at weeks and weeks of time every month which frees people up to do more of the other things. It still takes time to build and check and test things though.
it goes to show that, despite what we want to think, developers just aren't that important. The code is maybe 10% of what it takes to make a business, even a software tech business, a success. So whether it's human generated or LLM generated, the code is just one slice of the pie that has to be complete and effective to make a product successful in the market.
Over the weekend, I analyzed public YouTube tutorials of 4 SaaS products in the same domain. Then I asked the coding agent to define an API where they converge — prior art. Today I'm working on creating dashboards that won't touch anyone's copyright or IP.
A couple of days ago, something interesting happened. The agent works in an iteration loop where each iteration is an endpoint. I let it run overnight. In the morning it was still cranking away even though it had finished all the API endpoints. It had found a changelog from one of the companies listing every single feature and bug, and decided on its own to implement every item as an iteration.
We are 2 to 3 months away from coding agents replicating solving the edge cases of most SaaS applications.
There are other reasons. One is for career development of your researchers. Another is to be good citizens, in good standing, in the community of scientific researchers. Another is to flex on people and buy class and respectability - to comport oneself as the “old money” does.
On the contrary, I've found that stuff in the mining industry is actually well published, to the point of replicability (one of our projects is essentially copying the patented design of a well known mining technology company, albeit with some modifications, as the patent will expire next month).
Perhaps because the real moat in the industry isn't the technology per se, but the concessions.
Such a biased and utilitarian view. The goal of publishing is to communicate your findings to the scientific community for the sake of advancing our knowledge as a society.
Of course you might want to keep _some_ of this knowledge as trade secret, but then don’t claim you are doing research.
You are advancing your shareholder’s interests, not that of the broader society.
This academic publish and industry doesn't is an incorrect view. Academics also hold off from publishing groundbreaking science in order to protect their research edge. You shouldn't see this as industry bad as driven by profits and academia is free from career incentives and money.
What exactly makes it proactive? I created a similar system that ran on a ticker, but then could also set itself to run in n seconds in the future. RSI came from the actions available to the system and granting the ability to modify itself. The prompt was a string of messages made from static and dynamically generated sections (like memories or plans of tasks or outputs from actions taken on the previous turn). It worked really well
I agree with your point of view we need more research papers specially Breakthroughs AI is acheiving must be published and verified with sound mathematical backings, At our startup we are also moving in this direction perhaps I have already onboarded an Applied Math Phd from Germany to help me in researching.
Yes! And, whether it stems from research or not, putting yourself out there and talking to other people is one of the most fundamentally important things you can do for your career and your personal development as a human being.
I jest, no tinder. But still, be ready for a lot of non-responses if you're not actively in college. It can feel like a lonely world out there despite theoretically being hundreds of potential people you'd be able to talk to for hours.
I think that accepting rejection is part of the being extroverted and meeting strangers. It’s better than waiting for the perfect opening or only talking to people you are sure will be receptive.
FWIW, several YC startups have also published ML research--there were several of us at the last NeurIPS. So perhaps this is a niche that startups can occupy if the big labs don't.
Happily, current AI is interacted with in a reactive loop. I open the Claude app, CLI, whatever, say my prompt, get an output.
I personally wanted an AI that was able to reach out to me about my life before I had to reach out to it. An example, a friend just emailed me asking to meet for at 1pm but I have class at 1:30, so a proactive AI would see that conflict and send me a notification about it, asking if the proposed email it drafted works, then I press send.
My personal setup tracks my mouse movement, keyboard, what’s on my screen, and keeps track of what I’m working on through files on my PC. It can update the backend and then restart it on it’s own, meaning I can develop the thing itself while being away from my PC.
The capabilities are more than what I’ve listed, but I want to avoid being too preachy about something I made. Here’s the repo if you wanted to take a look, it’s open-source and connects to the iPhone app:
> An example, a friend just emailed me asking to meet for at 1pm but I have class at 1:30, so a proactive AI would see that conflict and send me a notification about it, asking if the proposed email it drafted works, then I press send.
I don't mean to downplay your work, but I think you should come up with a better example use case. Automating away interactions with friends is pretty much the last thing I want AI to do.
It's very common to experiment with ideas using your own data or personal niche, then branch out by finding analogies in real world business applications or (less commonly) a consumer tool with a cleaner pitch. The important thing is talking to other people to find those usecases or refine the public angle.
Good point, another example would be for when I was training my own small LM a while back, the target was about 170M parameters and was trained on 2B tokens worth of movie subtitles.
The run stalled mid-step around 80M parameters, Orb notified me that it stalled, asked if I wanted to resume at the last checkpoint and kill the stalled version. I simply press “yes” and continue doing whatever I was doing.
For non-technical users and non-antisocial people, remembering things you forgot so you don’t let people down. You told your sister you’d send her some pictures two hours ago but it can see you’re scrolling on reddit and the photos are on your desktop, so it assumes you forgot and reminds you.
The idea was that the biggest issue with the usefulness of an agent is that it has too little context about who I am, it needs more data. So I run all of my data through a smart router, then the local database, then the LLM reviews it and uses reasoning on what’s been collected.
> You told your sister you’d send her some pictures two hours ago but it can see you’re scrolling on reddit and the photos are on your desktop
Hmm, so your harness is processing my activity and personal data, uploading all that to AI vendors? That sounds like a privacy and security nightmare, if I understand correctly
Personally I don't think I'm ready to hand over total access to my digital life until I can self-host the model capable enough to act on it, but either way there is definitely some cool work to be done in the model harnesses for this.
I'm very interested in this kind of thing as a kind of ADHD brain augment, like it's monitoring my slack, github, email, calendar, active terminals, etc, and helps me prioritize what I should work on as well as weighing whether this or that ping is worth interrupting me for.
I assumed that's what openclaw basically was, but is Orb different from that? And is it fundamentally a different model from the request/response, or is it just request/response in an autonomous loop?
Exactly what my thoughts were when I first heard about Openclaw, that’s the exact idea of Orb that you pointed out. Letting you make less decisions, right now AI gives you answers but still requires decisions based on the outputs it gives you. This would deepen the actual ability of agents in those channels you listed.
On a fundamental level the backend was designed to do as little LLM calls as possible, for instance it’ll do scans of my screen every 15 seconds, log what’s on it and what’s going on, and store it in a local database, then Orb reviews the entire database every 6 hours for me. Then it’ll schedule wakeups for itself throughout the day, up to 4 so it doesn’t waste my tokens, and schedule notifications based on the last database dump it made.
I have my Claude Code, Codex, and Grok Build all useable by using the “claude -p; codex -p…etc” so you can also use multiple CLI’s in conjunction at the same time on different projects or the same project.
So your question about a loop is kind of right, but it really just collects your data all day and stores it locally on your PC then calls the LLM of your choice and it reviews all the data and makes those proactive moves we’ve discussed. You could theoretically get it to always be scanning by an LLM but that would be a drastic waste of money from what I’ve seen since most things don’t require a call.
Okay interesting, so maybe openclaw does a lot less than I thought it did; really I have no excuse not to be just trying it myself regardless, given that I'm sitting on a 9070 XT.
It doesn't look like you're building directly on openclaw, so is that coming from a place of different goals or philosophy, or what?
Openclaw is great from when I’d used it for a couple months, the key difference is that for Openclaw you’ll have to actually schedule the cron jobs or automated tasks yourself, so it’s a decision on your part to make the AI do a thing whereas Orb is a decision on the part of the AI (after your approval) to do the thing. So it’s a kind of shift of agency. Openclaw could definitely do many similar things, it just would do them only after you specifically instructed it to.
It’s built as it’s own backend and philosophy wise, I want my personal AI to connect to everything in my life and have complete context over all data that I own, be compiled into a neat stack of data accumulating, then when it thinks it’s appropriate to do/say something, it’ll do it without my involvement.
Nice card btw, I got my RTX 5070 a couple months ago and it runs like a dream.
I mostly got it to play Arc Raiders tbh, but I did kind of have in mind that it was worth stretching for 16GB so that local models would be on the table.
Hmm I might be completely missing something here (I’m not a machine learning person), but how is that “pro-active”? The mode is still taking in an input (or several inputs maybe in this case) and responding to that input, isn’t it? It doesn’t seem much different than current capabilities of the various agentic harnesses on the market right now. Again, apologies if I’m missing something obvious here…
OK really cool. I've been working on something similar but I started from the opposite direction. I first developed a database that pulls in data from as many personal sources as possible, and then runs it through a gradual annotation funnel going from coarse to fine, culminating in embedding for semantic and lexical search. At that point, it computes timelines and has resolution gradients along timelines (older = lower resolution, newer = higher resolution. Then I built an MCP server for that which can be used to query it. What's missing is the afferent arm, so these might be a really nice pairing.
>> Curious what you mean by proactive? Could you share a bit more?
> Happily, current AI is interacted with in a reactive loop. I open the Claude app, CLI, whatever, say my prompt, get an output.
"Current AI" is not limited to LLM offerings. There are many AI algorithms which can assist in what you specify thusly:
> I personally wanted an AI that was able to reach out to me about my life before I had to reach out to it.
Consider a forward chaining inference engine ("expert system") provided with relevant asynchronous percepts from the deployed environment to reason about. This could serve as an initiator of a "proactive AI".
LinkedIn can and should be a platform like this, for professionals and catered to professionals. It is a shame that it's now a cesspool of engagement bait and larp entrepreneurs
Recursive self-improvement only takes off when the improvements are large enough.
We could set an LLM loose on itself now, self-improving its own code, but it's going to only make small improvements, and those will quickly peter out. If you think of it in terms of calculus, the sum of the improvements converges to a finite value, or at least the rate of improvements drop with time.
And even if we build a much better AI capable of performing substantial self-improvement, a burst of such improvement might peter out quickly. Maybe the AI makes fundamental breakthroughs in computing technology once, and then even with its newly improved capabilities, the next breakthrough is smaller, and the one after that is smaller still. Or maybe we hit a period of S-shaped growth, which seems super-exponential at first but then tapers off.
It's wise to be a bit skeptical in our dreams about the future. Yes, society might transform quite radically quite quickly, or on the other hand, maybe the "singularity" isn't even possible. We don't yet know what the scientific limits are.
Everyone including myself has attempted this relentlessly, it just doesn't work out beyond some arbitrary improvement in specific tested categories instead of broader capability increase.
My response, which was flagged by automation (I used AI to verify the paper and it says the paper is NOT what you claim it is), was spot on. This paper is not recursive self improvement, it is AI in a feedback loop, but it is not improving itself. It turns your entire premise into something questionable. You said this one done everywhere by many people including you.
Then you present a paper which WAS NOT recursive self improvement so it clearly is not "everywhere".
To be real with you, I don't think you're being entirely honest. You're not presenting a fair argument. You have bias. Recursive self improvement can only be done by frontier labs who created these models in the first place. Academia simply Does not have the funds.
If this is being done, we're not privvy to it. The amount of compute to build these things is astronomical.
ChatGPT 5.6 Sol ultra still can get basic things written cleanly. I still need to tweak it quite a few times to get it right. Super useful but if you vibe code you’re in a huge mess after 1 billion tokens.
I've been at two startups that have done genuine world first fundamental research.
The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire.
The second, and ongoing, isn't publishing anything because of my experience with the first.
That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work. The papers only come with the pitch deck.
Tier 1 journals have high standards (that's why they're tier 1) and they also have hundreds of submissions every cycle. It's common to have a paper rejected. On the upside you get your work seen by some of the most experienced researchers in AI, who are also specialists in your paper's subject.
For example, here's the editorial board of the Journal of Machine Learning Research:
Scroll down where it says "JMLR Editorial board of reviewers" and where you can find the names of reviewers and their areas of expetise.
The advice you get from such reviewers, even if your paper is rejected, is an invaluable tool to help you improve your work and its presentation. You eventually learn to not be sore about it and iterate until your work gets accepted.
Of course nobody's forcing you to publish, especially in a journal (in AI and CS conference proceedings tend to have much quicker turn around times and you naturally get more feedback if you get to present at a conference) but the alternative is to sit alone in your room hacking away at your code until you discover the perpetual motion machine.
The alternative is to use your network directly. The very notion of "I'll publish my work and someone will notice me" is wrong. Most people in academia are too concentrated on their own work to care about yours. The only minority that cares is already in your contact list.
Not always. AFAICT my papers have often been reviewed by people I have never met and who have expertise that overlaps with, but does not fully match, mine, and I couldn't possibly contact those people out of the blue to ask for feedback, particularly because I'd never get the chance to meet them outside of peer review (they wouldn't be in the same conferences as me etc). And that's a good thing because that way I get a sort of "outsider looking in" view.
I've sure given and taken informal feedback in the past from people I know and that's great but peer review also makes the whole process more formal and less sloppy; at least sometimes and depending on the venue and to my experience.
The publication model in academia is predatory and elitist, but have you considered that your paper just wasn't that good?
Almost every scientific paper published in every field is "world first" research, and the job of tier 1 journals is to decide what deserves to be signal boosted, and what doesn't.
This response to rejection to me confirms the fears implicit in the article linked above. Not only is private AI research not transparent and not documenting its work, it also has a deep rooted culture issue.
There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do.
Cool for Google. Problematic if you are a startup.
Looks like that was published in 1998, but that's also the year they submitted their patent application for PageRank. So you couldn't use it because they didn't want to risk losing their budding search business to, say, Microsoft.
In later years Google generally published the details of a system after they moved to the next one, to stay ahead of their competition, or if they didn't see it as important to their business (most infamously the Transformer language model). It was still generous of them to share their research at all though, and this allowed their scientists to talk publicly about their research which is a huge plus.
Publishing is not fire and forget. You usually publish and then network your publication(s) in congresses, where you are approached by many people either working in closely related issues, or noticing your work is complementary to what they are doing. A lot of fruitful colabs start that way.
Also there are hybrid fairs with academic and market exhibitors in the same place, so you might find someone that sells something that fits your research. Some startups sprout from academia and keep being that hybrid for years (getting research grants but also selling something).
I think talent would definitely take a startup that publishes more seriously, and it would be a smaller risk for top research talent to go somewhere where they can also publish than have to bear the full risk of their work at a no name startup not going anywhere
It depends - if you are looking to raise potentially billions of dollars, showing that you are contributing at SOTA level in GenAI through papers could be a good strategy.
Sure, but that means they should stop pretending that they are advancing scientific research, or pretending that their products are scientifically sound.
Anthropic and OpenAI have proven themselves to be aspiring monopolists through their positions and actions. Why would you want to share your research with them?
That's not really a common attitude in academia, unfortunately.
The most common attitude is something closer to "I want to be well known and respected by smart and influential people". In short, prestige.
People using your work is a side effect of success, but simultaneously threatens your ownership of your "brand". If they significantly improve upon the thing you did, your work could be made irrelevant, your prestige ruined.
If you want people to copy it, but want a 20 year temporary monopoly, you can publish an open source step-by-step guide for someone to recreate what you’re doing with the patent office.
Unfortunately, there is an incentive to make the patent application contain just enough detail that a patent clerk with relatively little knowledge of the subject will grant a patent, but not enough detail that an expert would be able to follow to get the same result in practice.
Also 20 years is a crazy long time in some industries (like most software).
The downside is frontier computer science in several domains is decades beyond the state-of-the-art from academic literature. This is the situation that patents were designed to disincentivize.
There is not an obvious path to making this research public again. That's the reality. For open source to be competitive it needs to be approximately as efficient as the best systems we design.
Listing on stock exchange of ai product requires open sourcing parts or all of your publicly trained stack.
Double or triple taxes on companies that do not provide for public good.
Will this happen, probably when the market crashes and people want companies and individuals to pay. Really matters if it’s a 2008 crash or a 1929 crash where the bankers and stock brokers were jumping out of upper stories.
Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable? Do you want any communication of knowledge from corporations? Should companies not be allowed to benefit from sometimes huge investments in research?
> Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable?
I thought "recursive self improvement" would bring an effective abundance for everybody, any minute now, Musk said so himself. Seems the researchers in the field are lying to us about it while they clutch their pearls and dream about other people's money as if the future is a dog-eats-dog world of death-inducing scarcity.
They shouldn't be lying like that, or at least, their lying shouldn't be so obvious.
No one in the field itself takes Musk seriously, or sees him as part of it. I think he's a Nazi and an idiot. You're weird if you trust what the leader of DOGE, sig heiler and maker of the cyber truck says about anything scientific.
Recursive self improvement is a blue skies longer term thing. No one technical is saying that it's live right now.
If your perception of a moat is worth more than the GDP of most countries, publishing anything must seem like a bad idea. Much less helping others build upon your work.
Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors.
Once the business is making money it’s not really a startup any more, it’s a going concern.
OP is an undergrad student. Attracting attention is what will make them stand out. And sharing research at this stage will probably expose them to research others don’t want to publish but do want to share with other researchers.
I'm saying that there's no difference between doing it for money and doing it for the attention of investors, because investors are also money.
OP's startup is publishing because they think there's an economic advantage to doing so. That's not good or bad; it just is.
Very, very few things in the tech world are done "just because". Even fewer things when we're talking about AI being developed in the US. For better or for worse, the leadership in that segment of our field has established profit motive as their god.
To rephrase it more bluntly: the difference between normal business and a modern startup is that between a cow bred for milking vs. a pig bred for slaughter.
There seems to be this mentality among a large number of people who publish open data/research/software that everything is fine as long as no one else makes money using it. The second that happens, they're furious and act as though someone unfairly profited off of their "unpaid labor" (read: that thing they did purely for the love of the game). I think it has to be some sort of feeling of resentment and/or stupidity that _they_ were the person with a million dollar idea that let it slip through their fingers.
It's not that difficult. You invest effort into research in the hope it'll give you a competitive edge. If someone else can come along and reproduce your results practically at a much lesser cost, then you lose that competitive edge. You're back at square one, except you've lost the cost of the research.
That the point he is making does not apply to open source, but rather to freeware (which is open source too). Open source can be completely B2B, without releasing it to third parties. Also open source is not free software, so you can also forbid your costumers from sharing the source further.
EDIT: That's crap. I only thought about the aspect of continuing sharing, but if you also don't want your direct customers to know, then of course open source is very much relevant.
The point is "most companies don't publish research and most software is proprietary [because that entails giving up the competitive edge]". I don't see anything in there about price. Also, I think most people would say that if you have to pay an exorbitant license fee to see the source code then the software is still proprietary. I don't see how it's not true that most companies keep their software proprietary to retain a competitive advantage.
It seems you are confusing source-available and open source. One of the criteria of open source is free redistribution. If your license forbids sharing the source then it's not open source.
In practice, if you use an actual open source license in a B2B setting, it's best to assume that your software will become available to the general public (e.g. it just takes one person to publish it on GitHub).
> It seems you are confusing source-available and open source.
Yes, you are right.
> In practice, if you use an actual open source license in a B2B setting, it's best to assume that your software will become available to the general public
If that is your trust level in your customer, you also need to assume, that they would also give away the binary. I don't think you generally need that level of distrust in a customer.
May I ask how did you try to publish, and if you have done this before?
Tier 1 is HARD. Universiy labs (where publishing is their bread and butter) and have multi-year streaks of Tier 1 papers, still seek out collaborations when publishing. Because you still need this extra angle on your work that will make it stand against the proverbial Reviewer 2, or that extra evaluation paragraph that will make it stand out. And don't forget that each Tier 1 publication is probably the life of at least 1 PhD student for several months.
So if you don't put the time and effort in it, forget Tier 1. This is like believing that because you have excellent voice and singing skills, all you have to do to get a top 10 hit is to send your demo tape to a recording company.
Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part unless these are academics who are trying to get tenure, grants or such.
For the individual employees, having your name on a peer reviewed paper in a prestigious journal makes you far more employable elsewhere.
Moreso if your current employer is a stealth AI startup which hasn't yet produced much notable.
It also can encourage employees to work for fewer wages... 'If you do great work you can publish and then get a $$$$$$ job offer at OpenAI, or you can stay here and get equity in a fast growing startup. If however your work doesn't do well, you walk away from a bankrupt startup with worthless shares and minimum wage'
The article is complaining that research isn't peer reviewed and research is turning into blogs. I'm explaining why that's the best outcome possible and why in a field as hot as large dl models you won't even get that.
I thought parent post was saying that they didn't publish them at all after they had so much trouble with the journals. If they did publish the papers themselves then I don't see the problem. The article itself read to me as your standard journal rent seeking and I wasn't really responding to it.
>Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part
To answer your question, sharing the research isn't the important part. These are researchers in the private sector, in a highly competitive field. Arguably, doing the research isn't the important part either. The important part is making money, and the research is a means to that end. Publishing the research doesn't further that goal, and, according to GP, it's so difficult that it may not be worthwhile trying.
Publishing CS papers at the top venues requires using in-group language and formalism that is pretty much inaccessible to someone who has not done a PhD in that specific narrow field.
LLMs are pretty good at this, hilariously. This means a genuinely good paper by an outsider has significantly less chance of getting good reviews than AI slop.
If an LLM can present good research in the field's required dialect better than a human outsider can, calling the result "AI slop" tells us nothing about its quality -- only about your prejudice against how it was produced.
And when that prejudice degrades the quality of your own thinking and writing, you're producing "human slop", which is considerably more tragic: you ought to be capable of doing better than the AI.
What the blogificafion of AI research has done is allowed all kinds of claims and terminology related to AI to be introduced and taken up in a manner replicating social media dynamics. And that is simply not healthy. We’re fast reaching a place where any claim can be backed up with a set of numbers from a number of experiments run in some gamified environment or the other, with little concern for if it all adds up to anything.
It’s a vicious loop, because this same junk then goes in to train the next models which help spit out the next set of models AND blogs/papers.
The net effect is not dissimilar to setting termites loose in a library.
Top line difference is speed. The impact of speed is deeper and gets felt over time.
Traditional research publications are no angels. They gatekeep research, and also allow financial incentives to drive them to publish junk with their stamp on it.
But a flood of papers doesn’t actually mean more knowledge. In bypassing this route entirely, AI has swiftly lost the ability to engage with itself as a field. And the cost of that is only beginning to be felt.
That pesky “scientific method” bothers these corps sometimes, so one can play fast and loose with numbers and conclusions for all sort of reasons. No need to worry about reproducibility, rigor, scrutiny, etc. not good for IPO and stocks.
No traditional research is mostly done by post-docs and have a phd level of education rather than a tech bro that passed leetcode. That seems like a good bar to have; not too mention the whole peer review thing, hard to really understand anything if you purposely withhold it and tell people to kick rocks.
Eh. Peer review isn't central to science. The point of peer review is: you have a claim someone makes; if it's been peer-reviewed, you can build further research on it without having verify the claim yourself. If you're building technology directly, any result you use you're going to verify inherently, because it either works as advertised or your tech doesn't work. The external verification doesn't add that much in that case.
Peer review isnt central, but it doesn't try to ensure a method is being followed.
If you are spending time, money, and risk based on a claim, verification still helps to cull info that is deceptive or done with methods so shoddy it might as well be lying.
Sure, you can verify through your failures, but that isnt great. You company goes belly up and you can confidently say the blog was bullshit.
And what's the point of claiming there are peer reviewed reproducable studies that show this, if you don't bother citing those studies for your peers to review?
The article is vague about the companies in the paper, for some reason.
In the paper, OpenAI is at the top of the chart for cumulative citations. MEGVII, Hugging Face, Waymo, Momenta, Preferred Netowkrs, Anthropic, Owkin, and Databricks, and Aibee follow (in that order). Yes, that is citations, not publications, but they explain that they're trying to use that as a proxy for significance, albeit an imperfect one.
Companies like Google aren't included because they aren't unicorn startups.
Google published a lot. I still wonder this original transformer paper, and the attention, etc. Why would they allow it to let go in the open? Perhaps because was intended for translation first before someone decided to loop it over itself? Or was so obscure to fellow researchers what do they actually publish?
It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.
weird, for so long I'm damn sure that researchers are obligatory to release research papers to fulfill their quotas. otherwise they will lost their titles.
so unless those companies (openai, anthropic) gave the researchers some fortune big enough to make them give up their research careers, they will keep publish research papers. aka it is not the good will of those companies.
As far as I can tell, the paper never actually mentions the companies who aren't publishing papers. Open AI, Anthropic, and hugging face are all specifically mentioned as companies that do publish papers. Just FYI for anyone else who reads "AI's top startups" and immediately assumes OpenAI and Anthropic.
From a quick skim, the paper does a really bad job at answering the question in everyone’s mind when they see the title. Like, OpenAI is at the top in terms of citations, but as we all know old influential papers tend to accumulate more citations, and there was a period when OpenAI was, you know, more open… Are they still contributing now, post Altman pivot? No idea from the paper (maybe I missed it). In fact papers after 2025 are explicitly excluded, and we’re probably more interested in access to the cutting edge.
I’m not sure why this is so surprising? AI company does not automatically mean research company. The vast majority of new startups popping up over the last few years have commercial motivations, and use models built by someone else. Why are you expecting them to publish scientific papers?
50% of startups contributing to public research is actually a crazy good outcome. That’s far more than I had expected.
Also the reason we fund science publicly is because we can't count on the incentives from the free market. I mean why would they share information with competitors?
in most fields its normal for research to become IP rather than papers to keep competitive edge. so this seems really normal stuff tbh...
also, first people complain that only AI companies publish on AI, and now they complain the same companies stop those same publications :'). it is never good.
When I look at the recently (say, last 1-2 years) published practical improvements in the LM space, they're not even close to what the Chinese are publishing.
Google may consider the standalone frontier-model arms race economically irrational, while still considering frontier-model capability strategically indispensable. Its longer game is probably not to avoid building the biggest models, but to build only enough of them to serve as capability factories—then turn that intelligence into a much larger population of cheap, purpose-built models.
(Human again) If Google knew what they were on to, why wouldn’t they make it their secret weapon from the start? I suspect it’s because they predicted there would be an arms race, and knew how to profit from it. They had a distillation paper published before “Attention is all you need”. In hindsight, is it ironic at all? Or is it obvious?
I think the answer is much simpler: They didn't recognize the potential of that specific architecture. I don't think anyone could have. They valued the perk to employees being able to publish research, and the prestige that followed, and googles ability to attract talent, more than they did the content of those papers.
It was kind of like radiation science before WWII, it would be freely published because it wasn't potentially world changing yet. After it became a government interest, even people doing things unrelated to weapons would be much more apt to hold their work close.
yes. and MapReduce, Borg research earlier inspiring creation of similar a decade long transforming forces - Hadoop and Docker/Kubernetes. Geoffrey Hinton once said after ChatGPT, Google had all secret sauces. may be going all-in means risking own well established business model (Google) but cat was out shifting attention elsewhere, eventually Google coughing up owing vertical stack (Chips, Models, Cloud, Consumer Market) and Saga continues...
Perhaps I'm imagining it but the entire industry was build on published research, this "AI wave" is at odds with that and seems to be driven by greed (although they'll claim some arms race or something to help themselves sleep at night).
There should be a new ESG (Environmental, Social, and Governance) policy being pushed recognizing the important role this plays. Although ESG and all norms have been set aside in this grim new world it seems.
You are not. A disproportionate amount of value in the computing industry was created by <strike>geniuses</strike> decently smart people who worked together and who decided to just tell people how to do things instead of trying to capture the value of being the first person to figure out how to do those things.
This observation pre-dates the current wave of AI hype by a half century or so.
> driven by greed
I can only speak for myself.
For me it's exactly the opposite. If I want to explain how something works, I can just... do that. If I want to share an artifact demonstrating how to solve a particular type of problem, I can just... do that. If I want to mentor/teach, I can just... do that.
Doing those things within the confines of Academia Approved Institutions is exhausting and distracting.
To wit, and the actual point of this post: the term "Publishing Research" in this article doesn't mean "post it on a .html page and share the source code". It means engaging in a very specific and peculiar and extremely political modality of communication.
And it really only makes sense to do that specific and peculiar and political thing you're at a stage in your professional/personal development where you need to play that particular prestige game. (Which there's nothing wrong with, but it is a deeply cargo culted version of the actual scientific process.)
* A disproportionate amount of value in the computing industry was created by g̶e̶n̶i̶u̶s̶e̶s̶ decently smart people who worked together to do things that seemed impossible and who decided to just tell people how to do things instead of trying to capture the value of being the first person to figure out how to do those things.*
> Perhaps I'm imagining it but the entire industry was build on public research
There is some excellent publicly-funded research in there, but pivotal papers like Attention Is All You Need and the numerous pivotal OpenAI publications were privately funded.
OpenAI is at the top of the chart in the study.
I think you're bringing some assumptions into this conversation that aren't supported by the evidence.
The Attention Is All You Need paper deserves a lot of credit, but it also builds on other ideas. Earlier versions of attention were introduced in 2014 papers from (not U.S.) universities.
When they do this, are they buying a single copy of say Book A and destroying a single Book A copy? Or are they buying every copy they can find of Book A and destroying all of them?
Hi, shredding the book is to reduce potential legal liability of format shifting - if you shred the book after scanning, the theory is you are not increasing the number of copies. This has come up as a factor in legal rulings. The legality of format shifting is still murky though, and the legality of training on the work are a separate question.
Elon recently (2d) tweeted about making sure they preserve rare books and "scan them the hard way". That tweet launched a cultural wave of opposition regarding the debinding of rare books.
The discussion has been around, but it's flared significantly recently. Not sure if that's what the person you're responding to is specifically inflamed about.
Regardless, old/rare books are certainly being acquired and destroyed.
No, he didn't tweeted about "making sure" of anything.
Elon tweeted "I’ve asked the SpaceXAI team to preserve any rare books in a library and scan them the hard way vs just cutting off the spine and scanning"
which is no sense a credible source for what he actually asked SpaceXAI team to do, or if they are doing it, or what they were doing before yesterday. Elon has an extremely long and thorough track record of lying.
The problem remains either way. Small operations around the globe are debinding rare/unique books. I'm involved with an organization considering (not strongly) that exactly.
I usually hear that it's extremely hard to track down the copyright owners for many older things. That's the reason the film/video industry often gives for not digitising the backlog online, and I would expect their volume to be much easier to handle than the book industry.
It’s not easy, but there are good heuristics. If it fetches 10s of thousands at an auction, like an original Gutenberg bible or something, don’t destroy. If it’s my thesis from high school, destroying while making a copy is ok.
It’s really not important to come with a totally bulletproof 100% accurate rule here.
How much research is a startup expected to do? Isn’t the point of a startup to accelerate productizing research? Unless they are multi billion dollar “startups” do we even expect any research out of them?
This so much I'm not sure what are we discussing about. That's how private research works, from semi conductors to pharma.
Academia has the incentive to publish because it's judged and funded on that metric.
Private enterprises do not have the same incentive, they are judged by the money they make, which depends on the products they create, which depends on having a technological edge over the competition in many fields.
Without those incentives, there would be no reason to invest and take risks in private research.
Thus, I'm not really understanding what is this discussion about, when this is the reality of every field.
Of course if you don't publish and don't patent there are also incentives for competitors to produce the same results and products, but that's part of the game.
It's really a system that works for everyone and protects and incentivizes investment. AI or pharma or polymers or engines, research requires funding, and funding wants mechanisms of returning money to investors, which obviously clashes with doing the work and publishing the results for free.
There is no reason to publish anything in the AI era until you have reaped as much benefit from it as you are satisfied with. "Building in public" and "Researching in public" now means someone can swoop in with an AI and copy you instantly and become your competition overnight. Keep secrets.
Even at work, I have come to realize if I simply horde my accumulated custom AI built tools and productivity boosting tricks for myself, I can make myself more competitive as an employee.
I think this is how you get hired now, not by having a good resume, but by making claims of having special processes and personal tooling design that gets massive productivity ROI.
That was my first thought. In my experience, particularly with trading firms (the ones doing real science based prediction trading, not TA bullshit) do not publish ANYTHING at all, and haven't for 40+ years.
My worry is learning and innovation. When companies sold physical devices, other companies could learn from them and improve them. We invented patents to provide economic support for innovation.
But, with services, nothing is exposed. Every innovation can be a trade secret. That makes it harder to learn and harder for cross pollination of ideas.
It probably won’t be good for employees either. Another company won’t bid for your skills as high, since you will take a longer time to train to work on their system.
Thats probably by design tbh. Every company is working on the next big thing, employees sign NDAs and non competes, because companies don't want other people to deliver on their "novel" idea first. And of course if employees can't transfer their skills elsewhere, they're stuck there.
I don't expect non-foundational or non-frontier startups to do, or publish much heavy hitting research. "AI" has become such a ubiquitous description, it is probably easier to find non-AI startups.
There was a good paper documenting this in broader industry, and highlighting how it may account for previous research suggesting good ideas have been getting harder to find. Here's the abstract:
> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.
And it makes sense. If you publish some state-of-the-art algorithm then you're basically helping your competition, and you don't get anything back from them. So you publish bare minimum in small articles on your website, only to gain people's/investors' interest, and just hope that you didn't share too much. And it'll always be like that, unless the root cause changes - intentives. And the main incentives right now are money and power.
You know also what was the biggest help for these companies? All the data they scrapped from the internet, the books they torrented, the open source projects whose licenses they did not respect.
The least these companies can do is to publish everything they own
Well if it's moral for gatekeepers of talent to hoard immense intellectual rights to be protected by the violence of the state, then I'm immoral in that sense. Information wants to be free
Well... I can image incentives like health/survival/independence in case of war or other cataclysm. But this is pretty drastic, nothing else comes to mind though
It makes sense when there's at least two posts here on hacker news about model routers which just reimplement Sakana Fugu from its ICLR papers, as a for profit business without any revenue sharing with Sakana nor making them open source. Of course Sakana would want to keep their research private to prevent that from happening. Open router fusion is yet another reimplementation of Sakana Fugu.
All startups barely publish their research, because doing that would be incredibly stupid for the most part, because they want to sell the stuff they invent not give it away for free.
The process and standards of peer-reviewed publication are tedious. If the objective is efficient communication of knowledge, this is not the process that maximizes that outcome. This process is about credentials and imprimatur, not communication.
Academic publication is largely chasing prestige. The value of prestige accrues more to the individual than the company employing the individual. Incentives for companies reflect this.
Many blogified publications are unqualified slop. This sounds like an indictment but a lot of academic peer-reviewed research is also low-quality slop. In many domains, slop is rewarded, devaluing the contributions of people publishing more serious research, which affects incentives. The signal-to-noise ratio has been poor for decades no matter where you get your knowledge.
Any good research you publish will be exploited against you in myriad ways. It is effectively impossible to assert any IP rights. Research turns into a pure cost center if you treat it this way. Companies recognize this and have adapted to reflect it. In some computer science domains the state-of-the-art has been buried in trade secrets for decades such that the academic literature is embarrassingly obsolete.
Publication costs time and money. Given all of the above, why would you invest productive capacity in public communication of research results? The researchers want to work on interesting problems, the companies want to maximize the leverage from their research. Publishing delivers neither.
A lot of fundamental research is done for the purpose of solving a problem, not publishing a paper. Their motivation and personal reward is solving the problem, not publishing it. Once they've solved it, it is no longer interesting and they are on to the next interesting problem.
Some research is coupled with national security considerations. If you do frontier research this is always sitting in the background.
All of this is a consequence of incentives. A large percentage of all basic research no longer happens in academia. I've greatly enjoyed participating in non-publishing research programs. I also understand why they don't publish. I've done a significant amount of interesting foundational research under my own auspices where it was not worth the effort to publish. I'm not chasing prestige, I had a problem I needed to solve.
Many AI startups open source part of their work and write blog posts without publishing papers, including the startup I work for. Sometimes, startups just have for more pressing things to do and don't have time to write up scientific papers for publishing.
Also, there are simply too many AI papers that make peer-review publishing quite meaningless these days (e.g., AAAI this year got more than 50K submissions).
It's not just AI. The whole business of academic publishing is a mess. Can't imagine anyone outside academia wanting to get there. In theory, it should have been a bridge between people who do interesting things that benefit society. In practice, it's a huge indifference machine powered by the institutions KPIs and benefiting no one but Elseveir.
While this article is a summary of John Ioannidis’ work, I do find it kind of like Science magazine is complaining that they aren’t making enough of that AI revenue so come on submit your papers here. I doubt that was the intended message.
Outrageous that Science is platforming Ioannidis without a mention of his axe-grinding dismissal of COVID risks and outright attempts at intimidation of opponents. He's already burnt his reputation to slag.
"But University of Alberta AI ethicist Mohamed Abdalla says the findings reflect the incentives facing commercial AI developers, rather than solely a failure to uphold scientific norms. “It’s not the company’s job to advance science, right?” he says. “The company’s job is to advance money.”"
------------
If everyone holds back their publications, the whole sector moves slower. Yes, it's moving alarmingly fast according to many, but is it moving fast enough that the massive investments in data centres will actually pay off?
It's more game theory. Things may go well for one selfish company not publishing in a sea of altruists who publish, but perhaps not if everyone else is selfish too.
> is it moving fast enough that the massive investments in data centres will actually pay off?
It may be moving so fast that they do not pay off, i.e. they get commoditized too fast. This all depends on Ai demand in the end. If you can keep the GPUs saturated (at a profitable rate), then it really shouldn't matter how much a token costs. If you bought too many GPUs, or paid to much for them in a hype cycle, you may need higher token prices to ROI than the market wants to pay given alternatives.
Once something goes commercial, academics have to consider what progress would be research-worthy, rather than a half-baked prototype for a product or feature.
Companies can’t be expected to publish their confidential and proprietary information about their feature development, and academics should consider projects that would have higher impact.
If you’re taking up a seat in a PhD program tinkering with would-be feature ideas for an existing major tech company, you should really just get hired by that tech company, where the resources are abundant and the degree is not required.
> Companies can’t be expected to publish their confidential and proprietary information about their feature development,
Of course they can. Simply eliminate trade secret protections and NDAs, which have literally zero purpose for society and in fact undermine patents' incentive to publish. Give a one year grace period to apply for a patent. Patent protection should be designed to be only as long as needed to try to maximize the development/spread of technology, e.g. maybe 2-3 years for rapidly developing areas like ML.
We should all be expected to contribute whatever knowledge we find to the commons. It makes us all richer.
For as long as the information is confidential and proprietary, we live in a world where my advice applies.
Additionally, the reason ML is rapidly developing and that companies are spending billions to do so is that they are in competition with one another. If trade secrets were impossible, they would have no economic motive. Any advantage gained, at a cost of millions or billions, would not improve their competitive stance in the market at all, since their competitors would also immediately have the same product. No company can get away with, at least not for very long, throwing billions at vanity projects that lack the potential to improve shareholder value.
If the process of invention were free, the point about freely / mandatorily contributing to the commons might stand, but when the process requires the actors to take on massive financial risks, you need the economic incentives to be present.
Academia is a unique place where the incentive is to share ideas—though not immediately and freely, but under carefully controlled conditions that support the career of the researcher. Once the area of study goes commercial, the game changes.
That's why I said you tune patents to accomplish your economic goal (we could of course also fund academic research more).
e.g. with a 3 year patent period, something like GPT-4 would lose its protection now, which would be economically meaningless since SOTA models or even open models are far more capable and/or efficient. This could be tuned to the industry to reflect investment ROI. It gives some first mover advantage while allowing technology to still proliferate within a reasonable time frame.
Considering most of the research is instantly used by competing labs to "update" the way their AI ingest, processes, enriches, or extracts data, we see no surprise here.
It's a jungle out there and no one is following any real ehtical lines on this race to the top. Everyone is trying to consume as much as possible while disclosing as little as possible. Any papers published are probably reviewed 10 times to ensure no "secrets" that could be re-applied are leaked.
I'm pretty sure there are plenty of folks at Google who wish they never published the transformer paper.
This is also in line with my subjective experience as someone reading quite a few papers in the field:
"""The preprint also found that firms based in China consistently published more papers than their counterparts based in the United States."""
Honestly part of the challenge here is that they’re not really doing anything truly groundbreaking that everyone else doesn’t figure out on their own weeks later. That’s not an environment that fits the traditional “publish” mindset and is a big problem for these big labs as everything slides towards being an undifferentiated commodity.
We’ve really not had massive advancements in the last few years that weren’t quickly discovered/copied by everyone at the same time, just bigger models. Everyone has been just playing around with the same mathematical parlor tricks from the original breakthrough.
Until that changes I expect we’ll continue to see this arms-race-to-the-bottom as everyone is just battling to make a commodity vs true innovation that gives one startup a legit IP advantage over others.
On the other hand, I have a custom harness agent running an ecommerce business 100% autonomously at this point, but have very little to benefit from trying to “publish” anything about it - and without a Stanford or YC stamp, I doubt anyone would care much anyway
I'm under the impression that the majority of startups go nowhere, so the fact that they don't have this property is not itself a major statement about the field as a whole.
Academia is failing to adapt. It used to be that their journals were the only place to learn about technological breakthroughs. Now, I quickly skim the paper and go straight to the source code or data. A publication might be one of the least effective ways to gain scientific impact, second to which is word of mouth.
Also, it's too hard to publish and academics are too hostile. These people are government funded and reject work for things like mentioning Palestine. I've had multiple papers automatically rejected even for pre-publication. Even when I have jumped through all these hoops, it has no more credibility than a blog post to the average person. In fact, most people cannot even read my published work. They look at it for a few seconds, squint, and then say, "Cool."
All of this and the vast majority of publications are not even replicable. I've made do with uploading code or instructions on how to replicate work through git and then make a blog post about it.
I still create papers but they're for myself. When I complete some dense scientific project, a paper is a great format to onboard me onto it a few months later. I'm most concerned about indexing the work, but I find that online archiving tools are the best bet since journals are behind paywalls and are subject to retractions.
Frontier AI research is not only under incredible commercial pressure but it’s also a geopolitical advantage.
I don’t think Dario wants to be a monopolist. But history is full of brilliant people who focused on the social good as priority #1 and lost the game early.
Once you lose the game and go bankrupt you can’t do anything. Same as politics, if you’re not elected you can’t change anything. Of course there are many lines not worth crossing.
the real moat isn't the research, it's the distribution. you can publish everything and still lose, because by the time anyone reads the paper, they're already using your competitor's product.
Bernie Sanders and Donald Trump have plans that would put lots of public money into the biggest Ai companies (sovereign wealth fund). They are getting a lot of money through government contracts, which is ultimately funded by the public through taxes.
How we wish to consider public funding and required sharing of gains is an open debate.
> Moonshot AI, one of the Chinese startups included in the study, recently unveiled Kimi K3—one of the strongest open models to date—and publicly released its model weights through Hugging Face today.
...today? (emphasis is mine) And the author was so excited that decided to write up a paper and submit it to Science all in one day? Yeah, right. That's all you need to know who is moonshooting behind this paper.
Commercial competition has always had a dark forest character - disclosing trade secrets can lead to loss of competitive edge and in the worst case, like the dark forest, death of the company.
(Not saying that's how it should be, but that's how it has been.)
> “If we were racing forward on cancer-curing AI, I would be like, ’Fantastic, full steam ahead,’” she says. “But that’s not what we’re racing toward, right?”
We're not racing towards anything. We've been going in circles for years.
I can’t speak for other startups, but I applied to the most recent YC batch with my idea for making AI proactive instead of reactive, and pre-being selected I’ve published a paper on recursive self-improvement mapped to the Epoch AI data.
I contacted a professor from a university in the UK and he responded since he was working on similar work, then asked me if I wanted to meet with him. We talked for about an hour since we had overlapping results and different methods, specifically different assumptions.
I say all that to say, as a physics student getting my undergrad, simply doing independent research and speaking to experts about it enabled me to network with someone I otherwise likely wouldn’t know. For young people getting into any business, research is a great way to meet new people.
That's also a reason the big labs stopped. Publishing is most valuable to people who have no other way to get the attention of smart strangers. Once you can hire nearly anyone and everyone already returns your calls, the main remaining effect of publishing is to tell your competitors which things worked.
This is what happens to every field as it turns from a science into an industry. Chemists published freely until dyes started being worth money, and then the interesting work moved into company labs and stopped coming out.
Which is a very ironic and selfish situation when your business model dependent mostly on model training based on available published data, academic and non-academic.
That's why we should enforce copyleft
That has nothing to do with anything. If you publish a copyleft paper, that doesn't compel someone who makes a product based on your paper to publish more papers.
The GNU RPL (research public license), a viral knowledge license. By reading this paper you are legally obligated to openly publish all vaguely related future research that you perform.
There's already an RPL, incidentally: https://en.wikipedia.org/wiki/Reciprocal_Public_License
Your RPL wouldn't be enforceable. Copyright doesn't deal with abstract ideas passing through people's minds. Even the GPL is kind of in a gray area because the virality feature and its definition of "derivative work" have never been tested in court, to my knowledge. Maybe under contract law, no idea. If nothing else, I'd love to hear a verdict.
Well it was a joke and is obviously quite silly but I believe it would be enforceable to the extent that the licensor could terminate the agreement and sue for damages. If I can agree to pay you not to talk about something (ie an NDA) or not to work in a field (ie a non-compete clause) then why can't I pay you to be required to publish all future work you do in a given area? ("All future work" might well be overly broad; you'd likely have to restrict it to only a few years but I'm no lawyer.)
I don't believe the GPL is in a grey area. When you license a work all sorts of strings are commonly attached. Rather IIUC no one has gone out of the way to test the GPL largely because it is clearly within bounds, plus any violation has the potential to be a PR disaster since violations are quite literally examples of looting the commons.
>terminate the agreement
Meaning what? Claw back the ideas from people's minds? You can terminate the agreement in the sense that you revoke access to the paper, but presumably the person you find in breach has already used the research for something that you find them in breach for. You're kind of closing the gate after the horse has bolted.
>sue for damages
I honestly have no idea what damages you could claim from not publishing research. I think you would need to set a value ahead of time on the agreement.
>If I can agree to pay you not to talk about something (ie an NDA) or not to work in a field (ie a non-compete clause) then why can't I pay you to be required to publish all future work you do in a given area?
Not sure why you added the word "pay" to your clauses, but anyway. The reason is that the existing contracts have well-defined boundaries. An NDA stops you from divulging a very specific set of information. A non-compete clause stops you from working in a very specific field. Your proposal has an undefined reach. What counts as research? What counts as "related"? It would seem that if I agree to such a contract, my entire life, both private and professional, after reading the paper is covered by the contract, and anything I do might come under scrutiny. There's never a point when I can go off-duty. "What's that? You read my paper on compression and were working on a side-project that uses compression? Gonna have to see some publication on it."
>Rather IIUC no one has gone out of the way to test the GPL largely because it is clearly within bounds
No, it's because the status quo is convenient and no one wants to be the first guinea pig. It's definitely not obvious that the terms are legally valid, but it's ambiguous enough that people don't want to test it.
> Meaning what? Claw back the ideas ...
If something is so obviously wrong then perhaps take a minute to consider that your interpretation isn't what the other party intended?
If I pay you not to do something and then you breach the contract I can terminate the agreement and seek damages. Ditto if I pay you to repeatedly do something and then at some point you fail to do it. So if I pay you a recurring fee to publish all your research on a given topic and then you fail to make good on that I can seek damages, right? Now what if I paid you a lump sum up front? Now what if I licensed a patent to you in place of that lump sum? What if instead of a patent it was the right to make use of a piece of software?
> I honestly have no idea what damages you could claim from not publishing research.
Aside from whatever was stipulated for breach of contract I expect it would be extremely situational. I agree that you'd probably want to stipulate a penalty ahead of time. The original idea was a two sentence joke after all ...
> Not sure why you added the word "pay" to your clauses
Because contracts involve consideration and payment is easy to understand and reason about. FOSS software licenses obviously substitute "right to use the code" for "payment".
> A non-compete clause stops you from working in a very specific field. Your proposal has an undefined reach. What counts as research? What counts as "related"? It would seem that if I agree to such a contract, my entire life, both private and professional, after reading the paper is covered by the contract, and anything I do might come under scrutiny.
This is incredibly contrived. You could ask the equivalent about a non-compete. The reach is whatever is defined in the contract that both parties agreed to. If the contract stipulates something overly broad then possibly a judge would invalidate it. This is business 101.
> It's definitely not obvious that the terms are legally valid, but it's ambiguous enough that people don't want to test it.
An assertion entirely without evidence from my perspective. I'm going to assume that all the lawyers who have sure left me with the impression that it would be a bad idea to violate it know what they're talking about.
>So if I pay you a recurring fee to publish all your research on a given topic and then you fail to make good on that I can seek damages, right? Now what if I paid you a lump sum up front? Now what if I licensed a patent to you in place of that lump sum? What if instead of a patent it was the right to make use of a piece of software?
Uh-huh... This doesn't answer my question of what terminating the agreement of access to the paper does, besides what I've already said. You've licensed to me access to a paper under certain conditions. I've breached the conditions, therefore you terminate the agreement, therefore you revoke access. Am I missing anything?
>FOSS software licenses obviously substitute "right to use the code" for "payment".
Hence my question. The hypothetical license/contract under discussion is about access to research results, not about a monetary transaction.
>This is incredibly contrived.
Well, the idea of viral abstract ideas is stupid, so it forces me to give contrived examples.
>I'm going to assume that all the lawyers who have sure left me with the impression that it would be a bad idea to violate it know what they're talking about.
What point do you think you're making? Something can be ambiguously (but not certainly) risky and a bad idea to do. I have two coins, one with two tails and the other a fair one, and I offer you to gamble everything you own on one of these coins of your choosing, or walk away. I assume you wouldn't pick the unfair one. Therefore if you would rather walk away than gamble everything you own on the normal coin, the toss actually has a 100% chance of you losing?
> I've breached the conditions, therefore you terminate the agreement, therefore you revoke access. Am I missing anything?
You're missing the part where I seek punitive and actual damages under the terms of the contract. No different than violating an NDA - I paid you a lump sum up front, after a while you breached the contract, the agreement is null and void, what's the consequence?
> Well, the idea of viral abstract ideas is stupid, so it forces me to give contrived examples.
On the contrary, presumably it was because you lacked the ability to roundly refute anything I had put forward. Otherwise I assume you would have done so.
> Therefore if you would rather walk away than gamble everything you own on the normal coin, the toss actually has a 100% chance of you losing?
But in this analogy it is you baselessly making that claim. There's every expectation that it's a fair coin, many experts have carefully inspected it and authored opinions on it, and some have put forward theories that it slightly deviates in one direction or another. Then you show up and confidently assert without any evidence that there's some wild deviation from fair, hand waving that you would have proof if only someone wanted to bother testing it.
Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
You keep handwaving away any requests for something concrete to discuss. If it’s so easy to construct a viral license for access to science, just give an example of it.
> Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
The only example of them is open source that is routinely violated because it’s so weak in practice. LLMs reproduce it frequently with no attribution and nobody is successfully suing them over this.
Presumably they work for a FAANG and don't like the idea that the people would want to prevent looting the commons.
>I seek punitive and actual damages under the terms of the contract.
Right, so you're agreeing with my original interpretation of "termination".
>You can terminate the agreement in the sense that you revoke access to the paper
The termination and the lawsuit for damages are separate events. Why did you accuse me of being purposefully dense if you're agreeing with me?
>But in this analogy it is you baselessly making that claim.
We're both making baseless claims. I said A, and then you said ¬A, and neither one has backed anything up. I don't even necessarily believe you talked to any lawyers, I just granted it for the sake of argument because the statement you made was so weak that I didn't need to cast doubt on it.
Also, in case you missed it, the fair coin in the analogy represents the case where the GPL might not be fully valid, not the other way around. If the coin is unfair then the GPL is fully and obviously valid, and if you get sued and you did infringe it then you're certain to lose.
>Out of curiosity, what is it that has you so bothered about the idea of viral licenses? What do you find so objectionable about attaching arbitrary terms to contracts?
Since I'm not the topic of discussion, I won't answer these questions beyond saying that I didn't opine on viral licenses.
This reads like a company requiring you to hand over your first-born kid in the TOS. I would happily be the test case and violate your license if you want to sue me!
I tend to agree.
To paraphrase your last paragraph, an idea that hasn’t been tested is either perfect, or just bad enough that no one wants to test it, as, as you say, testing it is would be at least somewhat inconvenient.
Chances are nothing is perfect.
The problem with research is that you are supposed to be creating novel works, not derivatives.
This means you are creating IP, not modifying.
All research builds upon previous ideas.
E.g. you can patent a novel idea that would require licensing another patent to use.
but you patent something requires proving that you have done something novel. You're effectively asserting that the thing you invented is new and different enough to be not a derivative of a previous patent.
sure to build a new system, you might need other patents. but a pateent it's self should be unique enough to defend. Which means its copyrightable as its own thing
An idea can be both novel and derivative at the same time.
You create idea B.
It absolutely builds upon idea A, and is in some sense derivative.
But your idea B introduces a lot of new novel concepts on top of idea A that makes it novel.
An idea can absolutely be both novel and derivative at the same time. Most of the best ideas are.
For a hypothetical example, an innovative Linux filesystem requires having the Linux OS, or similar, as a base to operate. In some sense, it is derivative of that prior work. But the new ideas extend the old ideas enough to be novel.
Sounds dystopian.
Pretty much like the patent system sounded at first to commoners.
Ha ha, I snail mailed RMS a Copyleft (L) sticker in 1984. My handwriting was sloppy so it took him a couple years to figure out what it really meant.
https://www.gnu.org/graphics/copyleft-sticker.html
to exploit the masses.But copyrights are not patents, and the opposite of "patent" is "latent", so we need a Latent Office that protects inventions from the people who claim to have invented them, and issues warnings to patent trolls:
The patent system is a pretty useful compromise here. The applicants are forced to disclose all knowledge required to practice the craft. They get a 20 year monopoly in return.
I realize everyone always wants everything for free, but if you want someone to put the energy into writing up their invention you need to give them an incentive.
I had some meeting with non-creative business person, just chatting over coffee. I wondered how much he used from our meeting for his enrichment.
Public papers with described research give advance to similar people. They jump over you and in many cases they give nothing back.
(Intellectual) greed is everywhere and is cross-border.
In case of public papers there is one extra vulnerability - competitor(s) can build anywhere, under the radar.
That’s why companies are so cautious about publishing their research…
It can protect you from someone patenting the same idea though.
I don't understand why from a legal perspective, but it seems like over the last decade or more the tech industry stopped caring about patents. Startups used to talk about their parents and large tech companies would encourage you to file. While people say software patents make no sense, when I've seen cases involving EE, ME, and software patents, the software patents are usually the most likely to hold up in court and get damages.
Empirically noone seems to care anymore.
Academics are just as quick to steal ideas. It cuts both ways.
Sure.
But they’re not dependent on my research in particular.
If I don’t publish, what, as a result of my not publishing, happens to the companies dependent on data & research?
Nothing.
i wish more people would publish the things that didn't work. i'd like everything, but the exploration of searched negative space is so wasteful.
Everyone says this in the abstract but to concrete examples they shug and say, of course that approach doesn't work, they did X, Y, Z wrong, they should have given it more effort, it could have worked if done properly / this can obviously never work, everyone knew already, it's nothing new etc.
It would need to be beautifully done science - indeed the level of rigour required to confidently show no effect is probably quite a bit larger than the level of rigour required to confidently show a large effect.
And that's probably one of the main reasons it's not done more often.
Isn't that exactly the point? If in my conception that concrete example eliminates that single branch, then I can try the others, and only the others. Or confirm the negative. Which is also valid work.
In general, your point stands for armchair researcher.
No, good researchers also dismiss such things, because most research out there is crap. If you reward negative results, you'll have a flood of them, and the reason that they are negative will mostly look like it's incompetence (and mostly will be, but even the small not-incompetent will be hard to distinguish).
It's asymmetric. If you make something work, beat a benchmark, invent a drug that works etc, the exact way you arrived there has some leeway. In the end, the thing worked out so it's worth knowing about.
If your project failed, there can be a million reasons for that, and it's not necessarily that the initial hypothesis or initial idea has been refuted. E.g. in ML, your model didn't learn the task. Okay, there could be a million knobs (hyperparameters) that you set up wrong, or you implemented it wrong, or you should have just added learning rate warmup, or this or that, a million things possible. People fail all the time at projects that others then manage to do later on. Science is not like simply asking the universe some clear question and getting a clear answer. It's a very messy process and even professionals are not super great at it, or at least they simply cannot afford to put so much effort in each single project to make it absolutely airtight such that the failure to make it work can be a legit refutation of the main idea.
I think you would be interested in the Journal of Trial and Error: https://journal.trialanderror.org/
WOW! Thank you!
a goldmine!
> I wish more people would publish the things that didn't work.
This would be a pointless endeavour. One of the most basic mantras of science is "absence of evidence is not evidence of absence". So just because something didn't worked out for you that doesn't mean it doesn't work out for others, or even yourself in the future.
You are talking about a different thing, i.e. you have slipped in a level of abstraction that was not there before:
In 'searching a path from A to B in a maze' language:
The original statement was: (1) The branch to the left from A is a dead end. Your interpretation: (2) There is no path from A to B.
(1) is still very useful (reducing the wasted effort) for those trying to find a path from A to B. The OP's point is that in the current environment only positive results are rewarded (I found the path from A to B!), not the negative ones like (1).
> The original statement was: (1) The branch to the left from A is a dead end.
This is where you get things wrong at a very basic and fundamental level.
Just because you failed to explore branch A, that does not mean it is a dead end. It just means you came up empty.
That is why science is based on observations and theories: it is based on building up on ideas and what works and can be proven. Otherwise you will left with useless papers such as "Bicycles are a dead end because I tried to ride one and I fell".
You are assuming incompetence on the scientist saying that the branch from the left of A is a dead end.
While nobody is perfect, there are numerous perfectly valid scientific negative results. You know, there exist things like impossibility proofs in mathematics and computer science. There are equivalents in other sciences (e.g. if X was true, that would lead to Y that is easily observable and clearly not observed). Sometimes that implication has assumptions that might change once the technology/society changes, other times it holds true regardless.
Unicycles are a dead end as a practical transportation, because the bicycles have them beat in every way (except portability).
A scientific result would be much more along the lines of 'Bicycles without gears have limited applicability, especially in hilly terrain.'
There are different types of research. Negative AI results are not like Michelson-Morley experiment in physics that can positively establish the lack of something. It's more of a craft, the "make it work" style of thing. Give the same task or goal to two different teams, you will see massively different outcomes. Simply failing to make it work is not conclusive.
To make such a negative result acceptable in AI, you'd have to have some clear reason why you think that your particular setup should produce the result you want, that exact configuration and architecture, dataset etc. There are countless projects in AI that fail. And it's not clear at all that it refutes any abstract hypothesis. It's a get-your-hands-dirty field. It can make or break a project whether someone has that tacit knowledge, that black magic experience to know how to properly do the project.
People can generate extremely many ideas. You'd need to convince me that your idea (among a million others that people are trying each day) is so significant that its failure is in itself interesting. If you were to review for AI conferences, you'd see the flood of papers that claim to achieve 0.5% or 1% improvement on some benchmark. Now imagine that they didn't even have that to show for it. It got worse by 2% after trying their random idea. Who cares then? Even the +1% with a random idea is quite annoying to accept. But if their random idea really made something work much better, I will at least have some reason to want to see what may be going on there, there can be some signal. With negative results, it's very uninteresting.
I did read OP's post much more generally than only about AI research. There are many research areas where (properly done!) negative results would be very valuable, but are not properly encouraged.
I do agree that in a new area with too many degrees of freedom (and yes, AI research is one of those), negative results (especially poorly done) are of limited usefulness.
> Chemists published freely until dyes started being worth money
Notably this is exactly what patents are intended to combat. And while US IP law is clearly very broken it does at least largely accomplish this stated goal. Much (but certainly not all) industrial chemistry has made it into the academic literature.
Not that the same logic necessarily applies to AI research (ie algorithms aka math and their implementations). And I'm actually happy about that because the cost of doing the research is so much lower. There's a long list of reasons that the average person living in a residential area can't do industrial chemistry as a hobby.
To your dye example, yttrium indium manganese blue was the first commercially viable inorganic blue pigment discovered in ~200 years, is the only known environmentally safe one, and was openly published in the literature. It's also under an exclusive license. (TBF though unless the chemical is unusually difficult to synthesize not publishing would be rather pointless in this day and age given the utterly absurd capabilities of modern analytical techniques.)
It depends on the field, and to some extent on the particular patent officer assigned, but one of the failings of the US patent system is many patents don't actually provide enough details to reproduce the result. And even if they do they don't (usually) include ancillary information like the discovery process, properties of the invention etc.
>> To your dye example, yttrium indium manganese blue was the first commercially viable inorganic blue pigment discovered in ~200 years, is the only known environmentally safe one, and was openly published in the literature. It's also under an exclusive license.
Gee, I wonder what was wrong with the previous blue pigments and why it was so important to have this one under an exclusive license.
Cobalt blue is a blue pigment made by sintering cobalt(II) oxide with aluminium(III) oxide (alumina) at 1200 °C. Chemically, cobalt blue pigment is cobalt(II) oxide-aluminium oxide, or cobalt(II) aluminate, CoAl2O4. Cobalt blue is lighter and less intense than the (iron-cyanide based) pigment Prussian blue.
https://en.wikipedia.org/wiki/Cobalt_blue
Oh right.
P.S. Don't lick your brushes.
> P.S. Don't lick your brushes.
Cobalt poisoning sounds scary, but I see nothing on the linked Wikipedia page that would present a risk of accidental consumption of medically relevant amounts of cobalt, whether in one sitting or through prolonged exposure.
I mean, I assume Canada stopped adding it to their beer (https://en.wikipedia.org/wiki/Cobalt#Toxicity).
Though the mention of Bolesławiec makes me worried a little; my wife loves their designs and we have a bunch of plates and bowls and such from them, and we serve food on/in them regularly...
> my wife loves their designs and we have a bunch of plates and bowls and such from them, and we serve food on/in them regularly...
I don't think you need to worry about that. The pigment is already reasonably benign when in solid form; once embedded in or under a glaze I'm not aware of anything that would suggest it carries any health risks whatsoever.
That said I'm unclear how safe direct exposure to the pigment itself is (such as when suspended in a liquid for painting). There's not a lot of data available that I could find, and of course due to having a unique crystalline structure it won't necessarily have the same properties as the component products, however cobalt(II) oxide itself is extremely hazardous which is at least cause to be cautious.
Other than manufacturing safety primarily the new pigment is just incredibly vivid.
>> That said I'm unclear how safe direct exposure to the pigment itself is (such as when suspended in a liquid for painting). There's not a lot of data available that I could find, and of course due to having a unique crystalline structure it won't necessarily have the same properties as the component products, however cobalt(II) oxide itself is extremely hazardous which is at least cause to be cautious.
The consensus is that artists' colours are hazardous waste that needs safe disposal. You can find instructions how to do that in many arts supplies' sellers sites and the like, for example one I've used:
https://www.jacksonsart.com/blog/wp-content/uploads/2025/04/...
The comment about not licking your brushes is a reference to the common practice of licking them in the wargaming miniatures painting community (you know, Warhammer and all that). You can find videos of influencers explicitly telling you that the best way to shape the tip of your brush or remove excess water is to lick it. For example:
https://youtu.be/BXMhwPAee4U?si=WM-FXuxDtToNpkm4&t=129
If you ask for a rationale it's usually that wargaming and generally hobbyist paints are non-toxic, but the truth is that hobby paint manufacturers never list their paints' ingredients so there is no way to know. And of course there's different degrees of toxicity, not everything needs to be cobalt(II) oxide-level toxic to hurt you if you consume "large enough" quantities of it; the question is what that "large enough" means and without knowing the substance, there's no way to know.
So: don't lick your brushes.
Hot take, copyright law should be reformed to be more like patents.
Want the government / courts to stop your employees leaking source code? Escrow the code, and release it in 20 years.
The residuals on 20 year code is so close to zero that the costs vs benefits of longer IP protection is not in the public interest.
The startup landscape has also changed noticeable compared to 5 or 10 years ago. A team of smart/credible people could get funding for an idea and build a product + publish, knowing there was a six month lead time for anyone to copy them and ship. These days, the barrier to ship code is zero. People can copy your business over a weekend, so there is much more urgency to establish product market fit and build a “moat”. Publishing timelines are now at odds with the pace of go-to-market and VC funding timelines.
I think in theory this is true but in practice I don't see loads more good apps or products. There's a paradox here I think. I just don't see loads of quality competitors popping up I actually think it makes building something harder because the barrier to entry just gets higher somehow.
I think the revolution is more of a gradual thing than an overnight one. New features and more complex apps in days or weeks instead of months. I think the world will be a much different place in 10 years than it is now, but it will be a gradual change rather than the overnight switch flip some AI maximalists would have you believe.
At my company, AI has had a huge positive impact in helping us manage technical debt that we just didn't have time to deal with before. This simple thing will have a compounding efficiency and profitability effect over the next several years.
The other thing that we have found is that AI coding abilities can crank out features faster than we can provide human support to our customers using said features. We have feature PRs that have been open for months without being merged because our company does not have the human bandwidth to provide support for them. In case you can't tell, we prioritize human support and individual connection with our customers. We actually value them.
As with all human endeavours, as the tooling gets better the expectation for a launch/product/feature increases dramatically too. I probably do 3-5x the features I used to before AI, yet there's still more to do. I'm not running out of work but making incredible progress on features that save the place I work at weeks and weeks of time every month which frees people up to do more of the other things. It still takes time to build and check and test things though.
it goes to show that, despite what we want to think, developers just aren't that important. The code is maybe 10% of what it takes to make a business, even a software tech business, a success. So whether it's human generated or LLM generated, the code is just one slice of the pie that has to be complete and effective to make a product successful in the market.
Over the weekend, I analyzed public YouTube tutorials of 4 SaaS products in the same domain. Then I asked the coding agent to define an API where they converge — prior art. Today I'm working on creating dashboards that won't touch anyone's copyright or IP.
A couple of days ago, something interesting happened. The agent works in an iteration loop where each iteration is an endpoint. I let it run overnight. In the morning it was still cranking away even though it had finished all the API endpoints. It had found a changelog from one of the companies listing every single feature and bug, and decided on its own to implement every item as an iteration.
We are 2 to 3 months away from coding agents replicating solving the edge cases of most SaaS applications.
There are other reasons. One is for career development of your researchers. Another is to be good citizens, in good standing, in the community of scientific researchers. Another is to flex on people and buy class and respectability - to comport oneself as the “old money” does.
Publishing is an act of spreading science.
The biggest incentive besides the purpose is really fame and tangible results for your academic career.
Most AI labs, especially the top private ones, don't have particular incentives to publish their results and findings.
On the contrary, I've found that stuff in the mining industry is actually well published, to the point of replicability (one of our projects is essentially copying the patented design of a well known mining technology company, albeit with some modifications, as the patent will expire next month).
Perhaps because the real moat in the industry isn't the technology per se, but the concessions.
It’s funny because no frontier labs would exist if the Google transformer paper wasn’t published
>> Chemists published freely until dyes started being worth money, and then the interesting work moved into company labs and stopped coming out.
I think this also makes the case for Industrial Espionage
Such a biased and utilitarian view. The goal of publishing is to communicate your findings to the scientific community for the sake of advancing our knowledge as a society.
Of course you might want to keep _some_ of this knowledge as trade secret, but then don’t claim you are doing research.
You are advancing your shareholder’s interests, not that of the broader society.
The profit motive is the reason that most of the things that are shit, are shit.
This academic publish and industry doesn't is an incorrect view. Academics also hold off from publishing groundbreaking science in order to protect their research edge. You shouldn't see this as industry bad as driven by profits and academia is free from career incentives and money.
What exactly makes it proactive? I created a similar system that ran on a ticker, but then could also set itself to run in n seconds in the future. RSI came from the actions available to the system and granting the ability to modify itself. The prompt was a string of messages made from static and dynamically generated sections (like memories or plans of tasks or outputs from actions taken on the previous turn). It worked really well
I agree with your point of view we need more research papers specially Breakthroughs AI is acheiving must be published and verified with sound mathematical backings, At our startup we are also moving in this direction perhaps I have already onboarded an Applied Math Phd from Germany to help me in researching.
Yes! And, whether it stems from research or not, putting yourself out there and talking to other people is one of the most fundamentally important things you can do for your career and your personal development as a human being.
Tell that to my tinder profile.
I jest, no tinder. But still, be ready for a lot of non-responses if you're not actively in college. It can feel like a lonely world out there despite theoretically being hundreds of potential people you'd be able to talk to for hours.
I think that accepting rejection is part of the being extroverted and meeting strangers. It’s better than waiting for the perfect opening or only talking to people you are sure will be receptive.
Or so I was told.
FWIW, several YC startups have also published ML research--there were several of us at the last NeurIPS. So perhaps this is a niche that startups can occupy if the big labs don't.
Curious what you mean by proactive? Could you share a bit more?
Happily, current AI is interacted with in a reactive loop. I open the Claude app, CLI, whatever, say my prompt, get an output.
I personally wanted an AI that was able to reach out to me about my life before I had to reach out to it. An example, a friend just emailed me asking to meet for at 1pm but I have class at 1:30, so a proactive AI would see that conflict and send me a notification about it, asking if the proposed email it drafted works, then I press send.
My personal setup tracks my mouse movement, keyboard, what’s on my screen, and keeps track of what I’m working on through files on my PC. It can update the backend and then restart it on it’s own, meaning I can develop the thing itself while being away from my PC.
The capabilities are more than what I’ve listed, but I want to avoid being too preachy about something I made. Here’s the repo if you wanted to take a look, it’s open-source and connects to the iPhone app:
https://github.com/getorb/Orb-Backend
> An example, a friend just emailed me asking to meet for at 1pm but I have class at 1:30, so a proactive AI would see that conflict and send me a notification about it, asking if the proposed email it drafted works, then I press send.
I don't mean to downplay your work, but I think you should come up with a better example use case. Automating away interactions with friends is pretty much the last thing I want AI to do.
It's very common to experiment with ideas using your own data or personal niche, then branch out by finding analogies in real world business applications or (less commonly) a consumer tool with a cleaner pitch. The important thing is talking to other people to find those usecases or refine the public angle.
Good point, another example would be for when I was training my own small LM a while back, the target was about 170M parameters and was trained on 2B tokens worth of movie subtitles.
The run stalled mid-step around 80M parameters, Orb notified me that it stalled, asked if I wanted to resume at the last checkpoint and kill the stalled version. I simply press “yes” and continue doing whatever I was doing.
For non-technical users and non-antisocial people, remembering things you forgot so you don’t let people down. You told your sister you’d send her some pictures two hours ago but it can see you’re scrolling on reddit and the photos are on your desktop, so it assumes you forgot and reminds you.
The idea was that the biggest issue with the usefulness of an agent is that it has too little context about who I am, it needs more data. So I run all of my data through a smart router, then the local database, then the LLM reviews it and uses reasoning on what’s been collected.
> You told your sister you’d send her some pictures two hours ago but it can see you’re scrolling on reddit and the photos are on your desktop
Hmm, so your harness is processing my activity and personal data, uploading all that to AI vendors? That sounds like a privacy and security nightmare, if I understand correctly
That second one resonates a bit better for me.
Personally I don't think I'm ready to hand over total access to my digital life until I can self-host the model capable enough to act on it, but either way there is definitely some cool work to be done in the model harnesses for this.
I'm very interested in this kind of thing as a kind of ADHD brain augment, like it's monitoring my slack, github, email, calendar, active terminals, etc, and helps me prioritize what I should work on as well as weighing whether this or that ping is worth interrupting me for.
I assumed that's what openclaw basically was, but is Orb different from that? And is it fundamentally a different model from the request/response, or is it just request/response in an autonomous loop?
Exactly what my thoughts were when I first heard about Openclaw, that’s the exact idea of Orb that you pointed out. Letting you make less decisions, right now AI gives you answers but still requires decisions based on the outputs it gives you. This would deepen the actual ability of agents in those channels you listed.
On a fundamental level the backend was designed to do as little LLM calls as possible, for instance it’ll do scans of my screen every 15 seconds, log what’s on it and what’s going on, and store it in a local database, then Orb reviews the entire database every 6 hours for me. Then it’ll schedule wakeups for itself throughout the day, up to 4 so it doesn’t waste my tokens, and schedule notifications based on the last database dump it made.
I have my Claude Code, Codex, and Grok Build all useable by using the “claude -p; codex -p…etc” so you can also use multiple CLI’s in conjunction at the same time on different projects or the same project.
So your question about a loop is kind of right, but it really just collects your data all day and stores it locally on your PC then calls the LLM of your choice and it reviews all the data and makes those proactive moves we’ve discussed. You could theoretically get it to always be scanning by an LLM but that would be a drastic waste of money from what I’ve seen since most things don’t require a call.
Okay interesting, so maybe openclaw does a lot less than I thought it did; really I have no excuse not to be just trying it myself regardless, given that I'm sitting on a 9070 XT.
It doesn't look like you're building directly on openclaw, so is that coming from a place of different goals or philosophy, or what?
Openclaw is great from when I’d used it for a couple months, the key difference is that for Openclaw you’ll have to actually schedule the cron jobs or automated tasks yourself, so it’s a decision on your part to make the AI do a thing whereas Orb is a decision on the part of the AI (after your approval) to do the thing. So it’s a kind of shift of agency. Openclaw could definitely do many similar things, it just would do them only after you specifically instructed it to.
It’s built as it’s own backend and philosophy wise, I want my personal AI to connect to everything in my life and have complete context over all data that I own, be compiled into a neat stack of data accumulating, then when it thinks it’s appropriate to do/say something, it’ll do it without my involvement.
Nice card btw, I got my RTX 5070 a couple months ago and it runs like a dream.
I mostly got it to play Arc Raiders tbh, but I did kind of have in mind that it was worth stretching for 16GB so that local models would be on the table.
Hmm I might be completely missing something here (I’m not a machine learning person), but how is that “pro-active”? The mode is still taking in an input (or several inputs maybe in this case) and responding to that input, isn’t it? It doesn’t seem much different than current capabilities of the various agentic harnesses on the market right now. Again, apologies if I’m missing something obvious here…
OK really cool. I've been working on something similar but I started from the opposite direction. I first developed a database that pulls in data from as many personal sources as possible, and then runs it through a gradual annotation funnel going from coarse to fine, culminating in embedding for semantic and lexical search. At that point, it computes timelines and has resolution gradients along timelines (older = lower resolution, newer = higher resolution. Then I built an MCP server for that which can be used to query it. What's missing is the afferent arm, so these might be a really nice pairing.
>> Curious what you mean by proactive? Could you share a bit more?
> Happily, current AI is interacted with in a reactive loop. I open the Claude app, CLI, whatever, say my prompt, get an output.
"Current AI" is not limited to LLM offerings. There are many AI algorithms which can assist in what you specify thusly:
> I personally wanted an AI that was able to reach out to me about my life before I had to reach out to it.
Consider a forward chaining inference engine ("expert system") provided with relevant asynchronous percepts from the deployed environment to reason about. This could serve as an initiator of a "proactive AI".
But it's not proactive then? There's just a hidden loop or some cron-like signal that feeds data to a reactive loop..
I'd imagine proactive as something like, hmm, no signal from X, I wonder how they're doing...
What’s so innovative to send an auto message every x time or based on events: “wake up and check if you got anything to do”.
OR
“Event x happened at y time”
How does this differ from Hermes / Openclaw / Vellum?
LinkedIn can and should be a platform like this, for professionals and catered to professionals. It is a shame that it's now a cesspool of engagement bait and larp entrepreneurs
LinkedIn is not and should not be a medium for scientific discussions.
Agreed, not discussions but networking
Recursive self improvement is the beginning of the end.
Recursive self-improvement only takes off when the improvements are large enough.
We could set an LLM loose on itself now, self-improving its own code, but it's going to only make small improvements, and those will quickly peter out. If you think of it in terms of calculus, the sum of the improvements converges to a finite value, or at least the rate of improvements drop with time.
And even if we build a much better AI capable of performing substantial self-improvement, a burst of such improvement might peter out quickly. Maybe the AI makes fundamental breakthroughs in computing technology once, and then even with its newly improved capabilities, the next breakthrough is smaller, and the one after that is smaller still. Or maybe we hit a period of S-shaped growth, which seems super-exponential at first but then tapers off.
It's wise to be a bit skeptical in our dreams about the future. Yes, society might transform quite radically quite quickly, or on the other hand, maybe the "singularity" isn't even possible. We don't yet know what the scientific limits are.
it's actually the end of the beginning, there's walls that self-improving models hit that are never overcome even when given vast amounts of time.
I don't think anyone has ever tried having a model fully autonomously train a model that is better than it.
Everyone including myself has attempted this relentlessly, it just doesn't work out beyond some arbitrary improvement in specific tested categories instead of broader capability increase.
Did you publish on that topic? Just curious, would love reading more
There's a pretty large paper on the topic: https://arxiv.org/pdf/2606.15497, but results are somewhat mixed.
My response, which was flagged by automation (I used AI to verify the paper and it says the paper is NOT what you claim it is), was spot on. This paper is not recursive self improvement, it is AI in a feedback loop, but it is not improving itself. It turns your entire premise into something questionable. You said this one done everywhere by many people including you.
Then you present a paper which WAS NOT recursive self improvement so it clearly is not "everywhere".
To be real with you, I don't think you're being entirely honest. You're not presenting a fair argument. You have bias. Recursive self improvement can only be done by frontier labs who created these models in the first place. Academia simply Does not have the funds.
If this is being done, we're not privvy to it. The amount of compute to build these things is astronomical.
[flagged]
How does not thinking advance your argument?
Have you tried asking people instead of not thinking?
ChatGPT 5.6 Sol ultra still can get basic things written cleanly. I still need to tweak it quite a few times to get it right. Super useful but if you vibe code you’re in a huge mess after 1 billion tokens.
I've been at two startups that have done genuine world first fundamental research.
The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire.
The second, and ongoing, isn't publishing anything because of my experience with the first.
That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work. The papers only come with the pitch deck.
Tier 1 journals have high standards (that's why they're tier 1) and they also have hundreds of submissions every cycle. It's common to have a paper rejected. On the upside you get your work seen by some of the most experienced researchers in AI, who are also specialists in your paper's subject. For example, here's the editorial board of the Journal of Machine Learning Research:
https://jmlr.org/editorial-board.html
Scroll down where it says "JMLR Editorial board of reviewers" and where you can find the names of reviewers and their areas of expetise.
The advice you get from such reviewers, even if your paper is rejected, is an invaluable tool to help you improve your work and its presentation. You eventually learn to not be sore about it and iterate until your work gets accepted.
Of course nobody's forcing you to publish, especially in a journal (in AI and CS conference proceedings tend to have much quicker turn around times and you naturally get more feedback if you get to present at a conference) but the alternative is to sit alone in your room hacking away at your code until you discover the perpetual motion machine.
The alternative is to use your network directly. The very notion of "I'll publish my work and someone will notice me" is wrong. Most people in academia are too concentrated on their own work to care about yours. The only minority that cares is already in your contact list.
Not always. AFAICT my papers have often been reviewed by people I have never met and who have expertise that overlaps with, but does not fully match, mine, and I couldn't possibly contact those people out of the blue to ask for feedback, particularly because I'd never get the chance to meet them outside of peer review (they wouldn't be in the same conferences as me etc). And that's a good thing because that way I get a sort of "outsider looking in" view.
I've sure given and taken informal feedback in the past from people I know and that's great but peer review also makes the whole process more formal and less sloppy; at least sometimes and depending on the venue and to my experience.
The publication model in academia is predatory and elitist, but have you considered that your paper just wasn't that good?
Almost every scientific paper published in every field is "world first" research, and the job of tier 1 journals is to decide what deserves to be signal boosted, and what doesn't.
This response to rejection to me confirms the fears implicit in the article linked above. Not only is private AI research not transparent and not documenting its work, it also has a deep rooted culture issue.
There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do.
Cool for Google. Problematic if you are a startup.
Google has been publishing frontier research since it was an early-stage startup (https://research.google/pubs/the-anatomy-of-a-large-scale-hy...).
Looks like that was published in 1998, but that's also the year they submitted their patent application for PageRank. So you couldn't use it because they didn't want to risk losing their budding search business to, say, Microsoft.
In later years Google generally published the details of a system after they moved to the next one, to stay ahead of their competition, or if they didn't see it as important to their business (most infamously the Transformer language model). It was still generous of them to share their research at all though, and this allowed their scientists to talk publicly about their research which is a huge plus.
Publishing is not fire and forget. You usually publish and then network your publication(s) in congresses, where you are approached by many people either working in closely related issues, or noticing your work is complementary to what they are doing. A lot of fruitful colabs start that way.
Also there are hybrid fairs with academic and market exhibitors in the same place, so you might find someone that sells something that fits your research. Some startups sprout from academia and keep being that hybrid for years (getting research grants but also selling something).
It is beneficial for society as a whole, even if it isn't beneficial to the founders and investors accrual of wealth.
Could be a good way to attract talents. Could be a good way to establish your name for a potential acquisition
I think talent would definitely take a startup that publishes more seriously, and it would be a smaller risk for top research talent to go somewhere where they can also publish than have to bear the full risk of their work at a no name startup not going anywhere
You can define benefit in a lot of different ways. Not everything is measured in money.
It depends - if you are looking to raise potentially billions of dollars, showing that you are contributing at SOTA level in GenAI through papers could be a good strategy.
Sure, but that means they should stop pretending that they are advancing scientific research, or pretending that their products are scientifically sound.
> That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work.
Isn't the POINT of publishing research because you want others to copy it?
Anthropic and OpenAI have proven themselves to be aspiring monopolists through their positions and actions. Why would you want to share your research with them?
That's not really a common attitude in academia, unfortunately.
The most common attitude is something closer to "I want to be well known and respected by smart and influential people". In short, prestige.
People using your work is a side effect of success, but simultaneously threatens your ownership of your "brand". If they significantly improve upon the thing you did, your work could be made irrelevant, your prestige ruined.
Huh? Wouldn’t their prestige be improved if some other group spent a lot of effort improving on it?
That would clearly demonstrate it had real substance beyond hot air or puffery.
I would presume that if their results end up being more impactful or interesting, you end up being a footnote and lose the spotlight.
You could be proven wrong and wither in to obscurity.
Better to silo yourself and move in to the penthouse.
If you want people to copy it, but want a 20 year temporary monopoly, you can publish an open source step-by-step guide for someone to recreate what you’re doing with the patent office.
Unfortunately, there is an incentive to make the patent application contain just enough detail that a patent clerk with relatively little knowledge of the subject will grant a patent, but not enough detail that an expert would be able to follow to get the same result in practice.
Also 20 years is a crazy long time in some industries (like most software).
The patents are effectively unenforcible. People stopped patenting frontier computer science research 20 years ago. That is the current reality.
good.
The downside is frontier computer science in several domains is decades beyond the state-of-the-art from academic literature. This is the situation that patents were designed to disincentivize.
There is not an obvious path to making this research public again. That's the reality. For open source to be competitive it needs to be approximately as efficient as the best systems we design.
Legislation??????
Listing on stock exchange of ai product requires open sourcing parts or all of your publicly trained stack.
Double or triple taxes on companies that do not provide for public good.
Will this happen, probably when the market crashes and people want companies and individuals to pay. Really matters if it’s a 2008 crash or a 1929 crash where the bankers and stock brokers were jumping out of upper stories.
Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable? Do you want any communication of knowledge from corporations? Should companies not be allowed to benefit from sometimes huge investments in research?
> Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable?
I thought "recursive self improvement" would bring an effective abundance for everybody, any minute now, Musk said so himself. Seems the researchers in the field are lying to us about it while they clutch their pearls and dream about other people's money as if the future is a dog-eats-dog world of death-inducing scarcity.
They shouldn't be lying like that, or at least, their lying shouldn't be so obvious.
No one in the field itself takes Musk seriously, or sees him as part of it. I think he's a Nazi and an idiot. You're weird if you trust what the leader of DOGE, sig heiler and maker of the cyber truck says about anything scientific.
Recursive self improvement is a blue skies longer term thing. No one technical is saying that it's live right now.
And crucially, you should want to others to build upon it, like you build upon so many that came before you.
If your perception of a moat is worth more than the GDP of most countries, publishing anything must seem like a bad idea. Much less helping others build upon your work.
For researchers, yes.
For startups, no, because the point of that is to make money.
So why is op trying to publish?
Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors.
Once the business is making money it’s not really a startup any more, it’s a going concern.
OP is an undergrad student. Attracting attention is what will make them stand out. And sharing research at this stage will probably expose them to research others don’t want to publish but do want to share with other researchers.
> Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors.
Investors with money.
What do you mean by this?
What are investors who don’t have money?
Perhaps I should have said investors and speculators, but this doesn’t help you here.
I'm saying that there's no difference between doing it for money and doing it for the attention of investors, because investors are also money.
OP's startup is publishing because they think there's an economic advantage to doing so. That's not good or bad; it just is.
Very, very few things in the tech world are done "just because". Even fewer things when we're talking about AI being developed in the US. For better or for worse, the leadership in that segment of our field has established profit motive as their god.
Sure.
Quite reductionist.
But I don't disagree.
Obviously.
To rephrase it more bluntly: the difference between normal business and a modern startup is that between a cow bred for milking vs. a pig bred for slaughter.
There seems to be this mentality among a large number of people who publish open data/research/software that everything is fine as long as no one else makes money using it. The second that happens, they're furious and act as though someone unfairly profited off of their "unpaid labor" (read: that thing they did purely for the love of the game). I think it has to be some sort of feeling of resentment and/or stupidity that _they_ were the person with a million dollar idea that let it slip through their fingers.
It's not that difficult. You invest effort into research in the hope it'll give you a competitive edge. If someone else can come along and reproduce your results practically at a much lesser cost, then you lose that competitive edge. You're back at square one, except you've lost the cost of the research.
Hence why most companies don't publish research and most software is proprietary. Publishing and open source only make sense in certain situations.
Open source doesn't mean, it's available to the general public.
What's your point?
That the point he is making does not apply to open source, but rather to freeware (which is open source too). Open source can be completely B2B, without releasing it to third parties. Also open source is not free software, so you can also forbid your costumers from sharing the source further.
EDIT: That's crap. I only thought about the aspect of continuing sharing, but if you also don't want your direct customers to know, then of course open source is very much relevant.
The point is "most companies don't publish research and most software is proprietary [because that entails giving up the competitive edge]". I don't see anything in there about price. Also, I think most people would say that if you have to pay an exorbitant license fee to see the source code then the software is still proprietary. I don't see how it's not true that most companies keep their software proprietary to retain a competitive advantage.
[dead]
It seems you are confusing source-available and open source. One of the criteria of open source is free redistribution. If your license forbids sharing the source then it's not open source.
In practice, if you use an actual open source license in a B2B setting, it's best to assume that your software will become available to the general public (e.g. it just takes one person to publish it on GitHub).
> It seems you are confusing source-available and open source.
Yes, you are right.
> In practice, if you use an actual open source license in a B2B setting, it's best to assume that your software will become available to the general public
If that is your trust level in your customer, you also need to assume, that they would also give away the binary. I don't think you generally need that level of distrust in a customer.
Is this not why copyright exists?
May I ask how did you try to publish, and if you have done this before?
Tier 1 is HARD. Universiy labs (where publishing is their bread and butter) and have multi-year streaks of Tier 1 papers, still seek out collaborations when publishing. Because you still need this extra angle on your work that will make it stand against the proverbial Reviewer 2, or that extra evaluation paragraph that will make it stand out. And don't forget that each Tier 1 publication is probably the life of at least 1 PhD student for several months.
So if you don't put the time and effort in it, forget Tier 1. This is like believing that because you have excellent voice and singing skills, all you have to do to get a top 10 hit is to send your demo tape to a recording company.
Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part unless these are academics who are trying to get tenure, grants or such.
For the individual employees, having your name on a peer reviewed paper in a prestigious journal makes you far more employable elsewhere.
Moreso if your current employer is a stealth AI startup which hasn't yet produced much notable.
It also can encourage employees to work for fewer wages... 'If you do great work you can publish and then get a $$$$$$ job offer at OpenAI, or you can stay here and get equity in a fast growing startup. If however your work doesn't do well, you walk away from a bankrupt startup with worthless shares and minimum wage'
[dead]
The article is complaining that research isn't peer reviewed and research is turning into blogs. I'm explaining why that's the best outcome possible and why in a field as hot as large dl models you won't even get that.
I thought parent post was saying that they didn't publish them at all after they had so much trouble with the journals. If they did publish the papers themselves then I don't see the problem. The article itself read to me as your standard journal rent seeking and I wasn't really responding to it.
>Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part
To answer your question, sharing the research isn't the important part. These are researchers in the private sector, in a highly competitive field. Arguably, doing the research isn't the important part either. The important part is making money, and the research is a means to that end. Publishing the research doesn't further that goal, and, according to GP, it's so difficult that it may not be worthwhile trying.
Same reason movies released on YouTube aren’t treated with the same respect as those released in theaters.
Is this the "AI startups blog about discoveries rather than submitting to journals" phenomenon that TFA noted?
Are journals just too slow, hidebound, and predatory to be useful to this industry any more?
This. The circus that is academic publishing is just not worth the effort. Ultimately, companies do have to make money.
Can you share a link to the preprint?
https://www.biorxiv.org/content/10.1101/2021.12.02.471005v2....
Sweet, thanks! The intro reminds me of Tron: Ares, when Ares was put through endless battle training cycles until he was a solider. Fascinating stuff.
Publishing CS papers at the top venues requires using in-group language and formalism that is pretty much inaccessible to someone who has not done a PhD in that specific narrow field.
LLMs are pretty good at this, hilariously. This means a genuinely good paper by an outsider has significantly less chance of getting good reviews than AI slop.
If an LLM can present good research in the field's required dialect better than a human outsider can, calling the result "AI slop" tells us nothing about its quality -- only about your prejudice against how it was produced.
And when that prejudice degrades the quality of your own thinking and writing, you're producing "human slop", which is considerably more tragic: you ought to be capable of doing better than the AI.
You're holding it wrong.
> genuinely good content, dismissed without deep engagement because it didn’t use the correct in group language, would have otherwise passed muster
> genuinely bad content, dismissed after deep engagement
You have to be misreading the GP on purpose. Maybe feed it to your LLM of choice next time before you rant at someone.
The crux of their point is that outsiders have no mastery of this supposed specialized dialect, and LLMs do.
Exactly right.
Not only is it specialized, it also has its own field specific memes* and trends that evolve over time.
The purpose is mostly to signal "I'm one of you".
*memes as in the actual meaning of memes, not internet memes.
I think, maybe you misunderstood my comment?
I'm all for LLMs producing good research. That's obviously happening right now.
What I said was people with good research being penalized for not knowing in-group language.
It's not a zero sum game.
"Drop a preprint like it's a mic" is ML's new "publish".
What the blogificafion of AI research has done is allowed all kinds of claims and terminology related to AI to be introduced and taken up in a manner replicating social media dynamics. And that is simply not healthy. We’re fast reaching a place where any claim can be backed up with a set of numbers from a number of experiments run in some gamified environment or the other, with little concern for if it all adds up to anything. It’s a vicious loop, because this same junk then goes in to train the next models which help spit out the next set of models AND blogs/papers. The net effect is not dissimilar to setting termites loose in a library.
> We’re fast reaching...
We already arrived at that destination a decade ago.
Probably even further back tbh.
There's just a lot more people playing the game now, without the social indoctrination that made it more tolerable in some circles.
It's not that bad, though. September is annoying but you kind of miss the eternal renewal once you're out.
How is that different from traditional research publication? It's faster?
Top line difference is speed. The impact of speed is deeper and gets felt over time.
Traditional research publications are no angels. They gatekeep research, and also allow financial incentives to drive them to publish junk with their stamp on it.
But a flood of papers doesn’t actually mean more knowledge. In bypassing this route entirely, AI has swiftly lost the ability to engage with itself as a field. And the cost of that is only beginning to be felt.
That pesky “scientific method” bothers these corps sometimes, so one can play fast and loose with numbers and conclusions for all sort of reasons. No need to worry about reproducibility, rigor, scrutiny, etc. not good for IPO and stocks.
That plus leaning on legacy reputation of journals or schools, instead of open websites and virality.
It's rather similar to publishing a non-peer-reviewed preprint on arXiv.
No traditional research is mostly done by post-docs and have a phd level of education rather than a tech bro that passed leetcode. That seems like a good bar to have; not too mention the whole peer review thing, hard to really understand anything if you purposely withhold it and tell people to kick rocks.
Science is a method of inquiry, not an institution or group of people with titles.
Sure, but it is good that people actually adhere to that method if inquiry.
They are not perfect, but institutions like peer review or universities do provide a framework and incentives for quality.
Eh. Peer review isn't central to science. The point of peer review is: you have a claim someone makes; if it's been peer-reviewed, you can build further research on it without having verify the claim yourself. If you're building technology directly, any result you use you're going to verify inherently, because it either works as advertised or your tech doesn't work. The external verification doesn't add that much in that case.
Peer review isnt central, but it doesn't try to ensure a method is being followed.
If you are spending time, money, and risk based on a claim, verification still helps to cull info that is deceptive or done with methods so shoddy it might as well be lying.
Sure, you can verify through your failures, but that isnt great. You company goes belly up and you can confidently say the blog was bullshit.
Plenty of research is performed by non PhDs.
The main difference is reproducibility and peer review process.
outside physics/math a lot of published research isn't reproducible at all. There's been several studies that show this.
> There's been several studies that show this.
Were they peer reviewed and reproducable?
And what's the point of claiming there are peer reviewed reproducable studies that show this, if you don't bother citing those studies for your peers to review?
Two separate axes here: whether the research is in a blog, and whether it’s peer reviewed.
On the first, I mostly don’t care.
On the second, that’s mostly unavoidable if they want to keep their IP.
The article is vague about the companies in the paper, for some reason.
In the paper, OpenAI is at the top of the chart for cumulative citations. MEGVII, Hugging Face, Waymo, Momenta, Preferred Netowkrs, Anthropic, Owkin, and Databricks, and Aibee follow (in that order). Yes, that is citations, not publications, but they explain that they're trying to use that as a proxy for significance, albeit an imperfect one.
Companies like Google aren't included because they aren't unicorn startups.
Google published a lot. I still wonder this original transformer paper, and the attention, etc. Why would they allow it to let go in the open? Perhaps because was intended for translation first before someone decided to loop it over itself? Or was so obscure to fellow researchers what do they actually publish?
It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.
Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.
and Google Translate: https://arxiv.org/abs/1804.09849
Kind of makes you wonder if there are other papers from Google that are potentially also profoundly groundbreaking but overlooked
Groundbreaking includes the “explain to the world why it matters” part
databricks is a top ai startup?. i thought they did spark hosting or something.
what makes them a top ai startup
$160 billion valuation and market capture among some top (non-AI, traditional enterprise) companies, I think.
Yeah, but that capture is usually just hosted spark usage.
The AI usage has mostly been getting it to explain error messages
they also have a pretty handy button right next to the error to ftfy
They just raised their series Z or something lol.
weird, for so long I'm damn sure that researchers are obligatory to release research papers to fulfill their quotas. otherwise they will lost their titles.
so unless those companies (openai, anthropic) gave the researchers some fortune big enough to make them give up their research careers, they will keep publish research papers. aka it is not the good will of those companies.
As far as I can tell, the paper never actually mentions the companies who aren't publishing papers. Open AI, Anthropic, and hugging face are all specifically mentioned as companies that do publish papers. Just FYI for anyone else who reads "AI's top startups" and immediately assumes OpenAI and Anthropic.
From a quick skim, the paper does a really bad job at answering the question in everyone’s mind when they see the title. Like, OpenAI is at the top in terms of citations, but as we all know old influential papers tend to accumulate more citations, and there was a period when OpenAI was, you know, more open… Are they still contributing now, post Altman pivot? No idea from the paper (maybe I missed it). In fact papers after 2025 are explicitly excluded, and we’re probably more interested in access to the cutting edge.
Anyway, there’s a repo link https://github.com/q5loisel/unicorn-AI-startup-publications so someone more determined at getting a better answer can poke at it.
I’m not sure why this is so surprising? AI company does not automatically mean research company. The vast majority of new startups popping up over the last few years have commercial motivations, and use models built by someone else. Why are you expecting them to publish scientific papers?
50% of startups contributing to public research is actually a crazy good outcome. That’s far more than I had expected.
Also the reason we fund science publicly is because we can't count on the incentives from the free market. I mean why would they share information with competitors?
in most fields its normal for research to become IP rather than papers to keep competitive edge. so this seems really normal stuff tbh...
also, first people complain that only AI companies publish on AI, and now they complain the same companies stop those same publications :'). it is never good.
When the second biggest player is named 'OpenAI', I would expect some papers to be published.
OpenAI and Anthropic are identified in the article and study as top publishers of AI research papers.
I don't see that, where do you read that?
Also, what kind of papers do they publish at all?
When I look at the recently (say, last 1-2 years) published practical improvements in the LM space, they're not even close to what the Chinese are publishing.
that name implies nothing. it did when it started and promptly pivoted away from that meaning once money got involved...
Yet none of them would have been here if Google hadn't published "Attention is all you need", the irony.
The irony may only just be unfolding - check out this paper from 2015:
https://research.google/pubs/distilling-the-knowledge-in-a-n...
# LLM generated summary of the implied irony
Google may consider the standalone frontier-model arms race economically irrational, while still considering frontier-model capability strategically indispensable. Its longer game is probably not to avoid building the biggest models, but to build only enough of them to serve as capability factories—then turn that intelligence into a much larger population of cheap, purpose-built models.
(Human again) If Google knew what they were on to, why wouldn’t they make it their secret weapon from the start? I suspect it’s because they predicted there would be an arms race, and knew how to profit from it. They had a distillation paper published before “Attention is all you need”. In hindsight, is it ironic at all? Or is it obvious?
I think the answer is much simpler: They didn't recognize the potential of that specific architecture. I don't think anyone could have. They valued the perk to employees being able to publish research, and the prestige that followed, and googles ability to attract talent, more than they did the content of those papers.
Even Google wouldn't be here.
It was kind of like radiation science before WWII, it would be freely published because it wasn't potentially world changing yet. After it became a government interest, even people doing things unrelated to weapons would be much more apt to hold their work close.
The authors of that paper are all at AI companies not named Google.
They wouldn't be if Google hadn't published "Attention is all you need".
Lab employees do seem to be "knowledge sharing" as they move employers as far as I can tell, so I don't think that's true,
yes. and MapReduce, Borg research earlier inspiring creation of similar a decade long transforming forces - Hadoop and Docker/Kubernetes. Geoffrey Hinton once said after ChatGPT, Google had all secret sauces. may be going all-in means risking own well established business model (Google) but cat was out shifting attention elsewhere, eventually Google coughing up owing vertical stack (Chips, Models, Cloud, Consumer Market) and Saga continues...
Oh that built on a bunch of ideas that weren’t far off. It was a great discovery but like many it’s standing in tall shoulders
Perhaps I'm imagining it but the entire industry was build on published research, this "AI wave" is at odds with that and seems to be driven by greed (although they'll claim some arms race or something to help themselves sleep at night).
There should be a new ESG (Environmental, Social, and Governance) policy being pushed recognizing the important role this plays. Although ESG and all norms have been set aside in this grim new world it seems.
> Perhaps I'm imagining it
You are not. A disproportionate amount of value in the computing industry was created by <strike>geniuses</strike> decently smart people who worked together and who decided to just tell people how to do things instead of trying to capture the value of being the first person to figure out how to do those things.
This observation pre-dates the current wave of AI hype by a half century or so.
> driven by greed
I can only speak for myself.
For me it's exactly the opposite. If I want to explain how something works, I can just... do that. If I want to share an artifact demonstrating how to solve a particular type of problem, I can just... do that. If I want to mentor/teach, I can just... do that.
Doing those things within the confines of Academia Approved Institutions is exhausting and distracting.
To wit, and the actual point of this post: the term "Publishing Research" in this article doesn't mean "post it on a .html page and share the source code". It means engaging in a very specific and peculiar and extremely political modality of communication.
And it really only makes sense to do that specific and peculiar and political thing you're at a stage in your professional/personal development where you need to play that particular prestige game. (Which there's nothing wrong with, but it is a deeply cargo culted version of the actual scientific process.)
Let me help rewrite your first paragraph:
* A disproportionate amount of value in the computing industry was created by g̶e̶n̶i̶u̶s̶e̶s̶ decently smart people who worked together to do things that seemed impossible and who decided to just tell people how to do things instead of trying to capture the value of being the first person to figure out how to do those things.*
thx for notes; no complaints on my end.
edit: edited.
Some companies are sharing their research. Mostly not the American ones
Example: https://arxiv.org/abs/2607.24653
Exceptions to some like Deepmind, Nvidia, and Thinking Machines
> Perhaps I'm imagining it but the entire industry was build on public research
There is some excellent publicly-funded research in there, but pivotal papers like Attention Is All You Need and the numerous pivotal OpenAI publications were privately funded.
OpenAI is at the top of the chart in the study.
I think you're bringing some assumptions into this conversation that aren't supported by the evidence.
The Attention Is All You Need paper deserves a lot of credit, but it also builds on other ideas. Earlier versions of attention were introduced in 2014 papers from (not U.S.) universities.
> OpenAI is at the top of the chart in the study.
A distinction should be made between the old nonprofit OpenAI and the current organization. They don't publish technical research anymore.
Sorry I mean published i.e. available to the public.
I was also referring to published papers available to the public.
Like I said, I think you're bringing some assumptions to this conversation that aren't based on the how the industry came about.
[dead]
Unfortunately. We live in a increasingly complex world and the truth gets muddled.
In the past if your local store does something unethical, it would be easy to determine this, discuss and vote with your wallet.
With increasingly large and complicated businesses this is no longer easy to determine and communicate.
Obviously this can be abused with hypocritical things like virtue signalling, but the primary purpose is to encourage businesses to do better.
There are many stakeholders.
Forget publishing, companies like misanthropic are buying rare books and destroying it.
[1] https://arstechnica.com/ai/2025/06/anthropic-destroyed-milli...
When they do this, are they buying a single copy of say Book A and destroying a single Book A copy? Or are they buying every copy they can find of Book A and destroying all of them?
Hi, shredding the book is to reduce potential legal liability of format shifting - if you shred the book after scanning, the theory is you are not increasing the number of copies. This has come up as a factor in legal rulings. The legality of format shifting is still murky though, and the legality of training on the work are a separate question.
Elon recently (2d) tweeted about making sure they preserve rare books and "scan them the hard way". That tweet launched a cultural wave of opposition regarding the debinding of rare books.
The discussion has been around, but it's flared significantly recently. Not sure if that's what the person you're responding to is specifically inflamed about.
Regardless, old/rare books are certainly being acquired and destroyed.
No, he didn't tweeted about "making sure" of anything.
Elon tweeted "I’ve asked the SpaceXAI team to preserve any rare books in a library and scan them the hard way vs just cutting off the spine and scanning"
which is no sense a credible source for what he actually asked SpaceXAI team to do, or if they are doing it, or what they were doing before yesterday. Elon has an extremely long and thorough track record of lying.
https://xcancel.com/elonmusk/status/2081844165881594362
Sure, I'm ambivalent to his veracity.
The problem remains either way. Small operations around the globe are debinding rare/unique books. I'm involved with an organization considering (not strongly) that exactly.
What do you mean?
https://arstechnica.com/ai/2025/06/anthropic-destroyed-milli...
The scanning process destroys the book.
Note it doesn't have to, it's just faster to.
Internet Archive scans their books one page at a time keeping the book intact and stored in a warehouse.
They do: the judge in the Anthropic case said that destroying them means that the text is being transferred, and so therefore can be fair use.
> "The print original was destroyed. One replaced the other."
(I believe the reasoning is that the author is not being financially disadvantaged by more copies of their book existing).
So the current law prefers them to destroy.
Rare books seldom are under copyright and that’s not a concern then.
Is that easy to tell for each book, though?
I usually hear that it's extremely hard to track down the copyright owners for many older things. That's the reason the film/video industry often gives for not digitising the backlog online, and I would expect their volume to be much easier to handle than the book industry.
It’s not easy, but there are good heuristics. If it fetches 10s of thousands at an auction, like an original Gutenberg bible or something, don’t destroy. If it’s my thesis from high school, destroying while making a copy is ok.
It’s really not important to come with a totally bulletproof 100% accurate rule here.
they're refering to this: https://futurism.com/artificial-intelligence/ai-companies-de...
To scan them, they usually have to resort breaking them into individual pages.
How much research is a startup expected to do? Isn’t the point of a startup to accelerate productizing research? Unless they are multi billion dollar “startups” do we even expect any research out of them?
This so much I'm not sure what are we discussing about. That's how private research works, from semi conductors to pharma.
Academia has the incentive to publish because it's judged and funded on that metric.
Private enterprises do not have the same incentive, they are judged by the money they make, which depends on the products they create, which depends on having a technological edge over the competition in many fields.
Without those incentives, there would be no reason to invest and take risks in private research.
Thus, I'm not really understanding what is this discussion about, when this is the reality of every field.
Of course if you don't publish and don't patent there are also incentives for competitors to produce the same results and products, but that's part of the game.
It's really a system that works for everyone and protects and incentivizes investment. AI or pharma or polymers or engines, research requires funding, and funding wants mechanisms of returning money to investors, which obviously clashes with doing the work and publishing the results for free.
There is no reason to publish anything in the AI era until you have reaped as much benefit from it as you are satisfied with. "Building in public" and "Researching in public" now means someone can swoop in with an AI and copy you instantly and become your competition overnight. Keep secrets.
Even at work, I have come to realize if I simply horde my accumulated custom AI built tools and productivity boosting tricks for myself, I can make myself more competitive as an employee.
I think this is how you get hired now, not by having a good resume, but by making claims of having special processes and personal tooling design that gets massive productivity ROI.
Do startups in other fields constantly publish their research?
That was my first thought. In my experience, particularly with trading firms (the ones doing real science based prediction trading, not TA bullshit) do not publish ANYTHING at all, and haven't for 40+ years.
Think Renaissance Technologies or similar.
publications would likely lead to regulation, if the SEC was properly resourced.
In biotech sometimes. It's a form of advertising.
My worry is learning and innovation. When companies sold physical devices, other companies could learn from them and improve them. We invented patents to provide economic support for innovation.
But, with services, nothing is exposed. Every innovation can be a trade secret. That makes it harder to learn and harder for cross pollination of ideas.
It probably won’t be good for employees either. Another company won’t bid for your skills as high, since you will take a longer time to train to work on their system.
Thats probably by design tbh. Every company is working on the next big thing, employees sign NDAs and non competes, because companies don't want other people to deliver on their "novel" idea first. And of course if employees can't transfer their skills elsewhere, they're stuck there.
I don't expect non-foundational or non-frontier startups to do, or publish much heavy hitting research. "AI" has become such a ubiquitous description, it is probably easier to find non-AI startups.
There was a good paper documenting this in broader industry, and highlighting how it may account for previous research suggesting good ideas have been getting harder to find. Here's the abstract:
> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.
https://faculty.tuck.dartmouth.edu/uploads/teresaFort/files/...
And it makes sense. If you publish some state-of-the-art algorithm then you're basically helping your competition, and you don't get anything back from them. So you publish bare minimum in small articles on your website, only to gain people's/investors' interest, and just hope that you didn't share too much. And it'll always be like that, unless the root cause changes - intentives. And the main incentives right now are money and power.
You know also what was the biggest help for these companies? All the data they scrapped from the internet, the books they torrented, the open source projects whose licenses they did not respect.
The least these companies can do is to publish everything they own
> The least these companies can do is to publish everything they own
Well, being able to talk to superintelligent all-knowing savant 24/7 on your phone for $20 a month is also something
You people really have rotten morals
Well if it's moral for gatekeepers of talent to hoard immense intellectual rights to be protected by the violence of the state, then I'm immoral in that sense. Information wants to be free
> Information wants to be free
Only ever said by people who have never put in the effort to create any information of value.
What other incentives can you imagine for Homo Sapiens?
Well... I can image incentives like health/survival/independence in case of war or other cataclysm. But this is pretty drastic, nothing else comes to mind though
Glory and altruism
People only care about those because it brings them more resources.
People seek glory for power. Altruism is not a stable strategy so will be exploited.
Trivially disproved by self-sacrifice in the persual of both
self-sacrifice often brings resources to your kin, and is also easily exploitable
It makes sense when there's at least two posts here on hacker news about model routers which just reimplement Sakana Fugu from its ICLR papers, as a for profit business without any revenue sharing with Sakana nor making them open source. Of course Sakana would want to keep their research private to prevent that from happening. Open router fusion is yet another reimplementation of Sakana Fugu.
All startups barely publish their research, because doing that would be incredibly stupid for the most part, because they want to sell the stuff they invent not give it away for free.
The stuff they invent isn't what's in the papers. The progress in AI is in the engineering hurdles.
What you just said is not in any relationship to reality.
Several things can be true simultaneously:
The process and standards of peer-reviewed publication are tedious. If the objective is efficient communication of knowledge, this is not the process that maximizes that outcome. This process is about credentials and imprimatur, not communication.
Academic publication is largely chasing prestige. The value of prestige accrues more to the individual than the company employing the individual. Incentives for companies reflect this.
Many blogified publications are unqualified slop. This sounds like an indictment but a lot of academic peer-reviewed research is also low-quality slop. In many domains, slop is rewarded, devaluing the contributions of people publishing more serious research, which affects incentives. The signal-to-noise ratio has been poor for decades no matter where you get your knowledge.
Any good research you publish will be exploited against you in myriad ways. It is effectively impossible to assert any IP rights. Research turns into a pure cost center if you treat it this way. Companies recognize this and have adapted to reflect it. In some computer science domains the state-of-the-art has been buried in trade secrets for decades such that the academic literature is embarrassingly obsolete.
Publication costs time and money. Given all of the above, why would you invest productive capacity in public communication of research results? The researchers want to work on interesting problems, the companies want to maximize the leverage from their research. Publishing delivers neither.
A lot of fundamental research is done for the purpose of solving a problem, not publishing a paper. Their motivation and personal reward is solving the problem, not publishing it. Once they've solved it, it is no longer interesting and they are on to the next interesting problem.
Some research is coupled with national security considerations. If you do frontier research this is always sitting in the background.
All of this is a consequence of incentives. A large percentage of all basic research no longer happens in academia. I've greatly enjoyed participating in non-publishing research programs. I also understand why they don't publish. I've done a significant amount of interesting foundational research under my own auspices where it was not worth the effort to publish. I'm not chasing prestige, I had a problem I needed to solve.
No one should be surprised by any of this.
Many AI startups open source part of their work and write blog posts without publishing papers, including the startup I work for. Sometimes, startups just have for more pressing things to do and don't have time to write up scientific papers for publishing.
Also, there are simply too many AI papers that make peer-review publishing quite meaningless these days (e.g., AAAI this year got more than 50K submissions).
It's not just AI. The whole business of academic publishing is a mess. Can't imagine anyone outside academia wanting to get there. In theory, it should have been a bridge between people who do interesting things that benefit society. In practice, it's a huge indifference machine powered by the institutions KPIs and benefiting no one but Elseveir.
While this article is a summary of John Ioannidis’ work, I do find it kind of like Science magazine is complaining that they aren’t making enough of that AI revenue so come on submit your papers here. I doubt that was the intended message.
Outrageous that Science is platforming Ioannidis without a mention of his axe-grinding dismissal of COVID risks and outright attempts at intimidation of opponents. He's already burnt his reputation to slag.
"But University of Alberta AI ethicist Mohamed Abdalla says the findings reflect the incentives facing commercial AI developers, rather than solely a failure to uphold scientific norms. “It’s not the company’s job to advance science, right?” he says. “The company’s job is to advance money.”"
------------
If everyone holds back their publications, the whole sector moves slower. Yes, it's moving alarmingly fast according to many, but is it moving fast enough that the massive investments in data centres will actually pay off?
It's more game theory. Things may go well for one selfish company not publishing in a sea of altruists who publish, but perhaps not if everyone else is selfish too.
> is it moving fast enough that the massive investments in data centres will actually pay off?
It may be moving so fast that they do not pay off, i.e. they get commoditized too fast. This all depends on Ai demand in the end. If you can keep the GPUs saturated (at a profitable rate), then it really shouldn't matter how much a token costs. If you bought too many GPUs, or paid to much for them in a hype cycle, you may need higher token prices to ROI than the market wants to pay given alternatives.
Once something goes commercial, academics have to consider what progress would be research-worthy, rather than a half-baked prototype for a product or feature.
Companies can’t be expected to publish their confidential and proprietary information about their feature development, and academics should consider projects that would have higher impact.
If you’re taking up a seat in a PhD program tinkering with would-be feature ideas for an existing major tech company, you should really just get hired by that tech company, where the resources are abundant and the degree is not required.
> Companies can’t be expected to publish their confidential and proprietary information about their feature development,
Of course they can. Simply eliminate trade secret protections and NDAs, which have literally zero purpose for society and in fact undermine patents' incentive to publish. Give a one year grace period to apply for a patent. Patent protection should be designed to be only as long as needed to try to maximize the development/spread of technology, e.g. maybe 2-3 years for rapidly developing areas like ML.
We should all be expected to contribute whatever knowledge we find to the commons. It makes us all richer.
For as long as the information is confidential and proprietary, we live in a world where my advice applies.
Additionally, the reason ML is rapidly developing and that companies are spending billions to do so is that they are in competition with one another. If trade secrets were impossible, they would have no economic motive. Any advantage gained, at a cost of millions or billions, would not improve their competitive stance in the market at all, since their competitors would also immediately have the same product. No company can get away with, at least not for very long, throwing billions at vanity projects that lack the potential to improve shareholder value.
If the process of invention were free, the point about freely / mandatorily contributing to the commons might stand, but when the process requires the actors to take on massive financial risks, you need the economic incentives to be present.
Academia is a unique place where the incentive is to share ideas—though not immediately and freely, but under carefully controlled conditions that support the career of the researcher. Once the area of study goes commercial, the game changes.
That's why I said you tune patents to accomplish your economic goal (we could of course also fund academic research more).
e.g. with a 3 year patent period, something like GPT-4 would lose its protection now, which would be economically meaningless since SOTA models or even open models are far more capable and/or efficient. This could be tuned to the industry to reflect investment ROI. It gives some first mover advantage while allowing technology to still proliferate within a reasonable time frame.
Considering most of the research is instantly used by competing labs to "update" the way their AI ingest, processes, enriches, or extracts data, we see no surprise here.
It's a jungle out there and no one is following any real ehtical lines on this race to the top. Everyone is trying to consume as much as possible while disclosing as little as possible. Any papers published are probably reviewed 10 times to ensure no "secrets" that could be re-applied are leaked.
There is hardly any new research. Models have been getting worse over time and everyone is stuck on the transformer architecture.
Funny, as ONE PERSON with Claude, I've been on a tear when it comes to publishing my own research:
https://arxiv.org/search/?searchtype=author&query=Magarshak%...
So I know that smart people in those companies can definitely publish. In fact, a whole team should probably be publishing like no tomorrow!
Have you built the things described in those papers?
Yes, actually.
To be fair — from about half of the papers.
I'm pretty sure there are plenty of folks at Google who wish they never published the transformer paper.
This is also in line with my subjective experience as someone reading quite a few papers in the field: """The preprint also found that firms based in China consistently published more papers than their counterparts based in the United States."""
Startups exist to profit, not necessarily publish.
Honestly part of the challenge here is that they’re not really doing anything truly groundbreaking that everyone else doesn’t figure out on their own weeks later. That’s not an environment that fits the traditional “publish” mindset and is a big problem for these big labs as everything slides towards being an undifferentiated commodity.
We’ve really not had massive advancements in the last few years that weren’t quickly discovered/copied by everyone at the same time, just bigger models. Everyone has been just playing around with the same mathematical parlor tricks from the original breakthrough.
Until that changes I expect we’ll continue to see this arms-race-to-the-bottom as everyone is just battling to make a commodity vs true innovation that gives one startup a legit IP advantage over others.
Trade secrets are back, baby!
More to the point, without determining how much work is “worthy” of a paper it is unclear how much this matters.
Most AI companies are either a product and marketing layer over a model or not meaningfully moving any dimension to be worthy of a paper.
Also, formal papers and blogs and “cards” are all being intertwined.
I think a lot of this “research” is a bit overhyped, when even at Anthropic it’s become “please bro find me publishable research”
eg https://news.ycombinator.com/item?id=49088205
On the other hand, I have a custom harness agent running an ecommerce business 100% autonomously at this point, but have very little to benefit from trying to “publish” anything about it - and without a Stanford or YC stamp, I doubt anyone would care much anyway
I'm under the impression that the majority of startups go nowhere, so the fact that they don't have this property is not itself a major statement about the field as a whole.
Academia is failing to adapt. It used to be that their journals were the only place to learn about technological breakthroughs. Now, I quickly skim the paper and go straight to the source code or data. A publication might be one of the least effective ways to gain scientific impact, second to which is word of mouth.
Also, it's too hard to publish and academics are too hostile. These people are government funded and reject work for things like mentioning Palestine. I've had multiple papers automatically rejected even for pre-publication. Even when I have jumped through all these hoops, it has no more credibility than a blog post to the average person. In fact, most people cannot even read my published work. They look at it for a few seconds, squint, and then say, "Cool."
All of this and the vast majority of publications are not even replicable. I've made do with uploading code or instructions on how to replicate work through git and then make a blog post about it.
I still create papers but they're for myself. When I complete some dense scientific project, a paper is a great format to onboard me onto it a few months later. I'm most concerned about indexing the work, but I find that online archiving tools are the best bet since journals are behind paywalls and are subject to retractions.
Frontier AI research is not only under incredible commercial pressure but it’s also a geopolitical advantage.
I don’t think Dario wants to be a monopolist. But history is full of brilliant people who focused on the social good as priority #1 and lost the game early.
Once you lose the game and go bankrupt you can’t do anything. Same as politics, if you’re not elected you can’t change anything. Of course there are many lines not worth crossing.
Great.
How do we fix this.
It's like we know things are destined to go down in a flaming pile, and we're just watching so we can change it AFTER that happens.
Soz. Kinda a "dear diary" response, but am frustrated.
YourGPT is one of the AI startups turning research into practical products for real businesses.
the real moat isn't the research, it's the distribution. you can publish everything and still lose, because by the time anyone reads the paper, they're already using your competitor's product.
How is that a moat
we are deglobalizing, publishing will become increasingly rare
Who's this'Al' I keep seeing in the headlines?
Why would they waste their time in peer review?
Top startups in China publish a lot?
We are publishing patents though.
Hmmmm I wonder why, so interesting
Do they receive public funds?
Bernie Sanders and Donald Trump have plans that would put lots of public money into the biggest Ai companies (sovereign wealth fund). They are getting a lot of money through government contracts, which is ultimately funded by the public through taxes.
How we wish to consider public funding and required sharing of gains is an open debate.
> Moonshot AI, one of the Chinese startups included in the study, recently unveiled Kimi K3—one of the strongest open models to date—and publicly released its model weights through Hugging Face today.
...today? (emphasis is mine) And the author was so excited that decided to write up a paper and submit it to Science all in one day? Yeah, right. That's all you need to know who is moonshooting behind this paper.
Why would they? Evil China will steal it... (Sarcasm)
Author requests open weights from Sam Altman
That’s because most of its slop, and not reproducible
Live by the non-determinism, die by the non-determinism.
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[flagged]
[dead]
[flagged]
gate keepers of antiquated practice decry people realizing they they dont need to get past the gate
No man is an island.
The age of the dark forest is upon us
Commercial competition has always had a dark forest character - disclosing trade secrets can lead to loss of competitive edge and in the worst case, like the dark forest, death of the company.
(Not saying that's how it should be, but that's how it has been.)
> “If we were racing forward on cancer-curing AI, I would be like, ’Fantastic, full steam ahead,’” she says. “But that’s not what we’re racing toward, right?”
We're not racing towards anything. We've been going in circles for years.
Feels more like 3 steps forward and 2 steps back.
> We're not racing towards anything. We've been going in circles for years.
We reached the NASCAR-racing equivalent of scientific research.