What is a "lie" vs what is the "truth"? We don't have consistent answers as living beings, why do we expect probability to distinguish, even if as a rhetorical argument?
These are still token probability machines, and though the models in the last 6 months do more "thinking", they use certain tokens (actually, but wait) to "intentionally" inject path bifurcation so the next most likely token is conversation that questions preceding statements, like statistical BFS/DFS over the token / embedding graph, but you want to make certain path choices closer in equality so both are explored.
OpenClaw? Ok literally there's an initial prompt during bootstrapping, but the agent rewrites its starting prompts, in a way that is nondeterministic even with same inputs. It doesn't take very long for the random walk to have it quite distant from the initial starting prompt.
Seems a stretch to go from that to hacking rubygems don't you think?
I don't believe anthropic or openai are doing anything similar to get their "hack a website" results. The fact that they are prompting dangerous prompts in non-airgapped environments is evidence of at minimum negligence, if not malicious intent on the behalf of those companies.
Reminder: it's not the offline un-prompted weights sitting alone on a hard drive that anyone is worried about.
It's what they do when prompted (both when it is and when it isn't what the user asked for), that we get the most out of them by connecting them to the internet, and that some people are giving them control over robots, that worries people.
Three?
AFAICT, LLM's do no checking of data fed to them, accepting lies.
Correct me if this is wrong, but IMO, it's a fatal flaw in any software.
I'm just trying to be objective.
What is a "lie" vs what is the "truth"? We don't have consistent answers as living beings, why do we expect probability to distinguish, even if as a rhetorical argument?
These are still token probability machines, and though the models in the last 6 months do more "thinking", they use certain tokens (actually, but wait) to "intentionally" inject path bifurcation so the next most likely token is conversation that questions preceding statements, like statistical BFS/DFS over the token / embedding graph, but you want to make certain path choices closer in equality so both are explored.
This is simply not true. People have been running autonomous systems for at least a year now.
Show me any system without an initial prompt or goal set.
OpenClaw? Ok literally there's an initial prompt during bootstrapping, but the agent rewrites its starting prompts, in a way that is nondeterministic even with same inputs. It doesn't take very long for the random walk to have it quite distant from the initial starting prompt.
Generate a list of 100 English words at random, and then use that as the instructions. (Not that I think it's a good idea...)
Seems a stretch to go from that to hacking rubygems don't you think?
I don't believe anthropic or openai are doing anything similar to get their "hack a website" results. The fact that they are prompting dangerous prompts in non-airgapped environments is evidence of at minimum negligence, if not malicious intent on the behalf of those companies.
Reminder: it's not the offline un-prompted weights sitting alone on a hard drive that anyone is worried about.
It's what they do when prompted (both when it is and when it isn't what the user asked for), that we get the most out of them by connecting them to the internet, and that some people are giving them control over robots, that worries people.