Whatever model you decide on, look for an uncensored version. They are more likely to do "unsafe" things, like tell you what happened in Tiananmen Square in 1989, but they'll also accept what you ask or say at face value, without being argumentative or deferring to professionals. They generally run better and don't get stuck in loops as easily, too.
thank you! I'm researching some information to understand the best setup (cost vs. performance and reasoning) to build a desktop computer for running an LLM, specifically for general tasks and reasoning in coding
Saw it on here a few days ago, you put in your GPUs and it helps give you an idea of what you can run. Although, the specific model weight is dramatically altered by use case. Do you want code creation? Creative writing? Assistant workloads? I would suggest doing some Googles/AI searches on “[Model returned by RunItLocal] for [use case]”.
I don’t run my own model, unfortunately I didn’t jump into getting a better GPU when I had the chance so I’m waiting out the market. In my specific use-case I use DeepSeek for programming since it’s the best bang for buck.
I use local models mostly for coding, system admin tasks (linux), and occasionally for things like image/video/music gen. I run a dual-node cluster of Dell Pro Max GB10s although 90% of the time I run 1 model per node. My favorite primary models at the moment are Qwen3.8:27b-Flash-Next FP8 and Qwen3.8:27b-gsq-iq3s. Flash next is the best model overall but it is very (overly) zealous and can get itself off on tiresome tangential tasks mid-turn. It's just too eager to let it run alone on longer-horizon coding tasks, but if you have the time and patience to monitor it and keep it on task it's output is excellent and it's also very fast for the model size. The iq3s variant is not Flash-Next (not using the Qwen4 architecture) but is still just about as good, especially for the 3bit quant, much more calm and manageable than Flash-Next. I tend to use iq3s most of the time and Flash-Next for tougher tasks when I can afford to be involved in the process and keep it on track.
Qwen4 should be dropping soon, hopefully another 27b that is less aggressively eager than 3.8-flash-next
sounds interesting, but are you saying that Qwen3.8:27b-Flash tends to skip instructions to do more than what the task requires? What have you tried to control it and make it more autonomous? I’m asking because I don’t want to constantly focus on the task execution, it’s exhausting to keep reading the reasoning of what a model is doing. but I think I'll try with Qwen3.8:27b-Flash
No, not at all, pretty much the opposite. It wants to do everything you ask for and will usually get there on its own but it will spend large chunks of time diagnosing things that it finds along the way that may or may not be what you want it focusing on. Compare it to a frontier model like Claude in Claude Code that will mention things it finds but continue on the main task at hand and always stays on track, only diverting to fix something that might be impeding its ability to finish the task, Flash-Next is more like an ADHD model that sees shiny things and can't help but chase after them Lol. It's really not as bad as Im probably making it sound, after all it is my #1 choice for many tasks, I just have to be in the mood to keep an eye on it and interrupt now and then. 3.8-iq3s is perfectly good for most things though and requires less oversight. I know many people have complained about the same thing related to Flash-Next so Im really hoping Qwen4 drops with a much less eager 'personality'.
ahh, I understand now, I hope Qwen4 launches in a couple of months, Until then, I might try with 3.8-Flash, Thank you for your insights, any suggestions for the installation?
Whatever model you decide on, look for an uncensored version. They are more likely to do "unsafe" things, like tell you what happened in Tiananmen Square in 1989, but they'll also accept what you ask or say at face value, without being argumentative or deferring to professionals. They generally run better and don't get stuck in loops as easily, too.
What do you recommend?
if you have a GTX 1060 6GB this might help you: "https://github.com/jpsanders/pascal-llm-stack"
thank you! I'm researching some information to understand the best setup (cost vs. performance and reasoning) to build a desktop computer for running an LLM, specifically for general tasks and reasoning in coding
You can try: https://runitlocal.io/
Saw it on here a few days ago, you put in your GPUs and it helps give you an idea of what you can run. Although, the specific model weight is dramatically altered by use case. Do you want code creation? Creative writing? Assistant workloads? I would suggest doing some Googles/AI searches on “[Model returned by RunItLocal] for [use case]”.
thanks, that site is useful, do you run a model on your own machine? if yes, what hardware and model are you using, and is it good enough for coding?
I don’t run my own model, unfortunately I didn’t jump into getting a better GPU when I had the chance so I’m waiting out the market. In my specific use-case I use DeepSeek for programming since it’s the best bang for buck.
oh thank you, I'll review it, good resource thank you again
best for what?
fair enough, I mostly meant for reasoning and coding, but feel free to share whatever you use and its strengths
I use local models mostly for coding, system admin tasks (linux), and occasionally for things like image/video/music gen. I run a dual-node cluster of Dell Pro Max GB10s although 90% of the time I run 1 model per node. My favorite primary models at the moment are Qwen3.8:27b-Flash-Next FP8 and Qwen3.8:27b-gsq-iq3s. Flash next is the best model overall but it is very (overly) zealous and can get itself off on tiresome tangential tasks mid-turn. It's just too eager to let it run alone on longer-horizon coding tasks, but if you have the time and patience to monitor it and keep it on task it's output is excellent and it's also very fast for the model size. The iq3s variant is not Flash-Next (not using the Qwen4 architecture) but is still just about as good, especially for the 3bit quant, much more calm and manageable than Flash-Next. I tend to use iq3s most of the time and Flash-Next for tougher tasks when I can afford to be involved in the process and keep it on track.
Qwen4 should be dropping soon, hopefully another 27b that is less aggressively eager than 3.8-flash-next
sounds interesting, but are you saying that Qwen3.8:27b-Flash tends to skip instructions to do more than what the task requires? What have you tried to control it and make it more autonomous? I’m asking because I don’t want to constantly focus on the task execution, it’s exhausting to keep reading the reasoning of what a model is doing. but I think I'll try with Qwen3.8:27b-Flash
No, not at all, pretty much the opposite. It wants to do everything you ask for and will usually get there on its own but it will spend large chunks of time diagnosing things that it finds along the way that may or may not be what you want it focusing on. Compare it to a frontier model like Claude in Claude Code that will mention things it finds but continue on the main task at hand and always stays on track, only diverting to fix something that might be impeding its ability to finish the task, Flash-Next is more like an ADHD model that sees shiny things and can't help but chase after them Lol. It's really not as bad as Im probably making it sound, after all it is my #1 choice for many tasks, I just have to be in the mood to keep an eye on it and interrupt now and then. 3.8-iq3s is perfectly good for most things though and requires less oversight. I know many people have complained about the same thing related to Flash-Next so Im really hoping Qwen4 drops with a much less eager 'personality'.
ahh, I understand now, I hope Qwen4 launches in a couple of months, Until then, I might try with 3.8-Flash, Thank you for your insights, any suggestions for the installation?
What hardware?