OpenAI and Anthropic are equally evil imo, but what bothers me more is how many tech workers actually think you need any product of either company to accelerate any engineering goals.
I just racked some consumer GPUs in my garage (6x r9700s), cranking 24/7 pumping 20M+ tokens a day towards my goals building compilers, custom operating systems, reproducible build debugging, kernel hardening, novel confidential compute tech... harder problems than anyone I know using cloud LLMs to solve.
I have still never paid for tokens AND have session privacy.
So you have 8K worth of GPUs in your garage, and and it bothers you that other tech workers are paying 20$ a month for access to SOTA models?
Of course we don't NEED to pay for the cloud, but my company is doing this for me and I can't quite justify spending even 4-5K on a compute cluster in my garage, as much as I would love to.
You get 20M tokens a day with a decent coding model on a $20 subscription? I am skeptical. I am further skeptical you can do that with any privacy, so odds are any discounts are a result of you selling your sessions as training data with no option to opt out.
You are misinformed on token amounts. As one example, z.ai [0] gives you somewhere in the ballpark of 150-300 Mtok/week, so 1-2x your amounts. Plus the model has a 1M window, and will be smarter.
Fair context. Still, those prices are at least somewhat subsidized by you selling your plaintext session data which is a non-starter for me. Especially when dealing with code or data that is under NDA and not mine to sell in exchange for cheaper tokens.
Also, when I want 1M context I have 118GB of usable vram on my Strix Halo, or I can combine 4 r9700s and have 128gb and can run 1M context models like laguna or deepseek, but in practice lots of smaller sessions is better for my workflow in most cases.
Qwen 3.8 27b is smart enough that all I want is to speed-max and paralell-max on that.
Sure privacy (or legality) considerations are valid, and depending on the subscription or the API you use, you may or may not get that.
But we are not talking about the same product anymore. Your $12k homelab does not provide the same product as a $20/month subscription (to say nothing of a $200/month one). And the trade-offs of the homely may be worth it (or necessary) for you, but it may not be worth it (or even be feasible) for others.
Well at the very least lets agree that if you are going to use cloud models, there are much better options than OpenAI and Anthropic prices for most people. One does not have to give Sam or Dario money.
Personally though, I would sooner trade my car for GPUs than let a third party be in control of the tools I use to do my job.
So instead of paying $20 or maybe 100$ a month for the equivalent amount of output, I can spend $10k on GPUs (plus a few more thousands for related hardware), then for ongoing electricity, and then have space to store these things.
You're right people don't need the subscription, they just need a far more privileged life. One were dropping $12k instead of $20-100 a month is an equivalent financial strain.
I don't think that math will work out. If you are okay with using only 7M tokens per day, and 7M from a small model, then you don't have very demanding needs. If you don't have demanding needs, I think you're unlikely to be willing to space $1200 ( plus $800 for the rest of the system) to run a local model, when for $10/month you can get an open code subscription (or api access) which won't train on your data (or something of open router).
I know that 7M a day is a pittance for my use cases, and given that you have multiple cards, it wasn't enough for your use case either.
Sure, I just add more cards as I need more throughput, and can combine up to four cards when I need 1M context on a smarter model, but in practice since 3.8 27b came out it is all I use.
Also, unless your sessions are end to end encrypted to a secure enclave, then they are living in plain text -somewhere- and privacy policies tend to change when money is left on the table, if blackhats do not get to the data and sell it first.
And since you're not running those GPUs efficiently at 95% utilization rates or higher to serve tokens profitably, you paid far more than the people who simply pay per token from Together AI or Fireworks AI or something. They also get ZDR/session privacy. If you're really tin-foil hatted, you can go to a TEE cloud like Alpha Compute or something which is probably safer/more secure than your own computer.
You're very right to be wanting to use open sourced models. You're delusional for thinking that buying your GPUs directly saves any money. You are not a cloud service provider, don't act like one.
I have the one of the cheapest private and sovereign AI solutions money can buy, and I get a good laugh whenever the big providers are down.
Also, $1200 for a GPUs that can produce ~7M tokens a day of Qwen 3.8 27b at 80tps is faster and cheaper than any major provider can serve a model of that class as far as I am aware. Pays for itself pretty fast.
Agreed, but not just pause. OpenAI very likely committed multiple felonies. The company should be under investigation, it’s insane that didn’t happen as soon as we learned about the hugging face hack
For real! Imagine if you created "dgellowAI" and trained a model and started offering access to agents etc and it turned out YOUR software hacked a totally separate $13B company. You would be in a cell somewhere and your company would be shredded.
It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business
Hard agree. I skipped the YC start of batch talk with Sam Altman in large part because I do not ever want to be in the same room as that asshole if I can help it.
I am hoping we get an administration that will not accept his bribes so he can finally be held accountable for criminal negligence and fraud.
> In essence, I read this essay as requesting a hall pass to let their company’s AI run amok
Disagree. Even if OpenAI didn’t exist, this problem would still exist. It is best to assume and guard against rogue AI, which is what the essay is about.
What is the realistic endgame for people who want to pause AI development?
Ban OpenAI, and crown Anthropic as a golden child of safe AI development? Surely no company is "trustworthy" with this power and all future AI capabilities are dangerous, so all AI research needs to be paused everywhere, requiring the equivalent of a nuclear non-proliferation treaty between the US and China. So post-treaty we're trusting the US and Chinese governments, their military, and the billionaires closed tied to them to not to continue developing the magic thinking machine that can solve any problems, wins every battle, and is the crux of the economy? With the US track record of breaking much less significant treaties?
I'm not pro or against pausing AI development. I just don't understand what the theoretical stop button looks like.
Gary Marcus is perennially wrong/dangerous/stupid, AI is really good, and a pause is terrible for everyone.
Sam Altman can be a bad person (ask his sister about him!) and deserves to go down yada yada without us needing to try to kill the goose which is about to fling us into a golden age.
There is a possibility of a golden age. We have no idea if that will happen. You’re lying to yourself if you take it as a certainty.
The situation is that we have to face a bet that is forced upon humanity by a few actors: in all cases we know there will be a very high cost to current society, there is some chances that we get some benefits.
You cannot think about the situation if you decide to set a probability of golden to 1, and ignore the externalities, risks, and direct negative impacts.
It’s an insane risk to go all-in with, but that’s what the AI cultists want.
Honestly you shouldn’t be close to any form of decision making if you cannot have such a basic level of analysis
Humans refuse to use negative utilitarnaistic thinking anywhere else, so why should I have to start now? The logical conclusion of the precautionary principle is literally antinatalism.
I assign the probability of extremely good outcomes as being overwhelmingly likely. The amount of harm done is a necessary part of it and is acceptable in much the same way that the additional gun crime as a result of the 2nd amendment is.
Every disease uncured, every amount of work done that was unnecessary, was an affront to human dignity. I want the machine god and I wanted it yesterday.
I am finding that the world is and likely will continue to be full of carbon-chauvinists for a long time.
OpenAI and Anthropic are equally evil imo, but what bothers me more is how many tech workers actually think you need any product of either company to accelerate any engineering goals.
I just racked some consumer GPUs in my garage (6x r9700s), cranking 24/7 pumping 20M+ tokens a day towards my goals building compilers, custom operating systems, reproducible build debugging, kernel hardening, novel confidential compute tech... harder problems than anyone I know using cloud LLMs to solve.
I have still never paid for tokens AND have session privacy.
So you have 8K worth of GPUs in your garage, and and it bothers you that other tech workers are paying 20$ a month for access to SOTA models?
Of course we don't NEED to pay for the cloud, but my company is doing this for me and I can't quite justify spending even 4-5K on a compute cluster in my garage, as much as I would love to.
You get 20M tokens a day with a decent coding model on a $20 subscription? I am skeptical. I am further skeptical you can do that with any privacy, so odds are any discounts are a result of you selling your sessions as training data with no option to opt out.
You are misinformed on token amounts. As one example, z.ai [0] gives you somewhere in the ballpark of 150-300 Mtok/week, so 1-2x your amounts. Plus the model has a 1M window, and will be smarter.
[0]: https://docs.z.ai/devpack/overview#estimated-token-allowance
Fair context. Still, those prices are at least somewhat subsidized by you selling your plaintext session data which is a non-starter for me. Especially when dealing with code or data that is under NDA and not mine to sell in exchange for cheaper tokens.
Also, when I want 1M context I have 118GB of usable vram on my Strix Halo, or I can combine 4 r9700s and have 128gb and can run 1M context models like laguna or deepseek, but in practice lots of smaller sessions is better for my workflow in most cases.
Qwen 3.8 27b is smart enough that all I want is to speed-max and paralell-max on that.
Sure privacy (or legality) considerations are valid, and depending on the subscription or the API you use, you may or may not get that.
But we are not talking about the same product anymore. Your $12k homelab does not provide the same product as a $20/month subscription (to say nothing of a $200/month one). And the trade-offs of the homely may be worth it (or necessary) for you, but it may not be worth it (or even be feasible) for others.
Well at the very least lets agree that if you are going to use cloud models, there are much better options than OpenAI and Anthropic prices for most people. One does not have to give Sam or Dario money.
Personally though, I would sooner trade my car for GPUs than let a third party be in control of the tools I use to do my job.
So instead of paying $20 or maybe 100$ a month for the equivalent amount of output, I can spend $10k on GPUs (plus a few more thousands for related hardware), then for ongoing electricity, and then have space to store these things.
You're right people don't need the subscription, they just need a far more privileged life. One were dropping $12k instead of $20-100 a month is an equivalent financial strain.
$1200 for a GPU capable of running Qwen 3.8 27b at 7M tokens a day is likely plenty for most people and will pay for itself.
Also factor in cloud LLM prices are subsidized by you giving up your sessions as training data with no ability to opt out.
I don't think that math will work out. If you are okay with using only 7M tokens per day, and 7M from a small model, then you don't have very demanding needs. If you don't have demanding needs, I think you're unlikely to be willing to space $1200 ( plus $800 for the rest of the system) to run a local model, when for $10/month you can get an open code subscription (or api access) which won't train on your data (or something of open router).
I know that 7M a day is a pittance for my use cases, and given that you have multiple cards, it wasn't enough for your use case either.
Sure, I just add more cards as I need more throughput, and can combine up to four cards when I need 1M context on a smarter model, but in practice since 3.8 27b came out it is all I use.
Also, unless your sessions are end to end encrypted to a secure enclave, then they are living in plain text -somewhere- and privacy policies tend to change when money is left on the table, if blackhats do not get to the data and sell it first.
Do you have a blog post or other documentation on how to replicate this config for self hosted inference?
1. Plug r9700 GPU into linux computer
2. Use free daily tokens from opencode or similar to set it up for you with a coding agent like jcode or crush
And since you're not running those GPUs efficiently at 95% utilization rates or higher to serve tokens profitably, you paid far more than the people who simply pay per token from Together AI or Fireworks AI or something. They also get ZDR/session privacy. If you're really tin-foil hatted, you can go to a TEE cloud like Alpha Compute or something which is probably safer/more secure than your own computer.
You're very right to be wanting to use open sourced models. You're delusional for thinking that buying your GPUs directly saves any money. You are not a cloud service provider, don't act like one.
I have the one of the cheapest private and sovereign AI solutions money can buy, and I get a good laugh whenever the big providers are down.
Also, $1200 for a GPUs that can produce ~7M tokens a day of Qwen 3.8 27b at 80tps is faster and cheaper than any major provider can serve a model of that class as far as I am aware. Pays for itself pretty fast.
Agreed, but not just pause. OpenAI very likely committed multiple felonies. The company should be under investigation, it’s insane that didn’t happen as soon as we learned about the hugging face hack
For real! Imagine if you created "dgellowAI" and trained a model and started offering access to agents etc and it turned out YOUR software hacked a totally separate $13B company. You would be in a cell somewhere and your company would be shredded.
It’s way worse, OpenAI was the one running thousands of agents (ie while loop prompting a model + tool call dispatching) in parallel in their infra, for months, with close to no supervision, using a model that was trained for attack, given a goal to solve hacking problems, with a harness that allows full execution.
The whole system is designed to be catastrophic. There is no rogue agent, or anything going off the rails, it is behaving exactly the way one would predict.
And they now announced that same model in their API, acknowledging it is way more difficult to monitor it. That company should not be in business
Hard agree. I skipped the YC start of batch talk with Sam Altman in large part because I do not ever want to be in the same room as that asshole if I can help it.
I am hoping we get an administration that will not accept his bribes so he can finally be held accountable for criminal negligence and fraud.
> Sam Altman cannot be trusted. I have been writing about that for a long time. Ronan Farrow’s reporting backs that up.
Discussion of Ronan Farrow’s New Yorker article (900+ comments): https://news.ycombinator.com/item?id=47659135
> In essence, I read this essay as requesting a hall pass to let their company’s AI run amok
Disagree. Even if OpenAI didn’t exist, this problem would still exist. It is best to assume and guard against rogue AI, which is what the essay is about.
What is the realistic endgame for people who want to pause AI development?
Ban OpenAI, and crown Anthropic as a golden child of safe AI development? Surely no company is "trustworthy" with this power and all future AI capabilities are dangerous, so all AI research needs to be paused everywhere, requiring the equivalent of a nuclear non-proliferation treaty between the US and China. So post-treaty we're trusting the US and Chinese governments, their military, and the billionaires closed tied to them to not to continue developing the magic thinking machine that can solve any problems, wins every battle, and is the crux of the economy? With the US track record of breaking much less significant treaties?
I'm not pro or against pausing AI development. I just don't understand what the theoretical stop button looks like.
Gary Marcus is perennially wrong/dangerous/stupid, AI is really good, and a pause is terrible for everyone.
Sam Altman can be a bad person (ask his sister about him!) and deserves to go down yada yada without us needing to try to kill the goose which is about to fling us into a golden age.
We could halt every AI datacenter tomorrow and it would not impact those of us using local GPUs in any way. Cloud AI is a dead end.
> goose which is about to fling us into a golden age
I prefer not to be flung.
There is a possibility of a golden age. We have no idea if that will happen. You’re lying to yourself if you take it as a certainty.
The situation is that we have to face a bet that is forced upon humanity by a few actors: in all cases we know there will be a very high cost to current society, there is some chances that we get some benefits.
You cannot think about the situation if you decide to set a probability of golden to 1, and ignore the externalities, risks, and direct negative impacts.
It’s an insane risk to go all-in with, but that’s what the AI cultists want.
Honestly you shouldn’t be close to any form of decision making if you cannot have such a basic level of analysis
Humans refuse to use negative utilitarnaistic thinking anywhere else, so why should I have to start now? The logical conclusion of the precautionary principle is literally antinatalism.
I assign the probability of extremely good outcomes as being overwhelmingly likely. The amount of harm done is a necessary part of it and is acceptable in much the same way that the additional gun crime as a result of the 2nd amendment is.
Every disease uncured, every amount of work done that was unnecessary, was an affront to human dignity. I want the machine god and I wanted it yesterday.
I am finding that the world is and likely will continue to be full of carbon-chauvinists for a long time.