Thanks for bringing that up - it's exactly why I approached it this way from the start.
Technically you can use the original Codex CLI with a local LLM - if your inference provider implements the OpenAI Chat Completions API, with function calling, etc. included.
But based on what I had in mind - the idea that small models can be really useful if optimized for very specific use cases - I figured the current architecture of Codex CLI wasn't the best fit for that. So instead of forking it, I started from scratch.
Here's the rough thinking behind it:
1. You still have to manually set up and run your own inference server (e.g., with ollama, lmstudio, vllm, etc.).
2. You need to ensure that the model you choose works well with Codex's pre-defined prompt setup and configuration.
3. Prompting patterns for small open-source models (like phi-4-mini) often need to be very different - they don't generalize as well.
4. The function calling format (or structured output) might not even be supported by your local inference provider.
Codex CLI's implementation and prompts seem tailored for a specific class of hosted, large-scale models (e.g. GPT, Gemini, Grok). But if you want to get good results with small, local models, everything - prompting, reasoning chains, output structure - often needs to be different.
So I built this with a few assumptions in mind:
- Write the tool specifically to run _locally_ out of the box, no inference API server required.
- Use model directly (currently for phi-4-mini via llama-cpp-python).
- Optimize the prompt and execution logic _per model_ to get the best performance.
Instead of forcing small models into a system meant for large, general-purpose APIs, I wanted to explore a local-first, model-specific alternative that's easy to install and extend — and free to run.
Good correction - while the SDK used has supported changing the API through environment variables for a long time, Codex only recently added Chat Completions support recently.
That’s one of the reasons I went with phi-4-mini - surprisingly high quality for its size and speed. It handled multi-step reasoning, math, structured data extraction, and code pretty well, all on modest hardware. Phi-1.5 / Phi-2 (quantized versions) also run on raspberry pi as others have demonstrated.
OpenAI rejected the request. Error details: Status: 400, Code: unknown, Type: api_error, Message: 400
registry.ollama.ai/library/phi4:latest does not support tools. Please verify your settings and try again.
Hey, that is a very good question, I have answered that before. I hope you don't mind, if I simply copy paste my previous answer:
Technically you can use the original Codex CLI with a local LLM - if your inference provider implements the OpenAI Chat Completions API, with function calling, etc. included.
But based on what I had in mind - the idea that small models can be really useful if optimized for very specific use cases - I figured the current architecture of Codex CLI wasn't the best fit for that. So instead of forking it, I started from scratch.
Here's the rough thinking behind it:
1. You still have to manually set up and run your own inference server (e.g., with ollama, lmstudio, vllm, etc.).
2. You need to ensure that the model you choose works well with Codex's pre-defined prompt setup and configuration.
3. Prompting patterns for small open-source models (like phi-4-mini) often need to be very different - they don't generalize as well.
4. The function calling format (or structured output) might not even be supported by your local inference provider.
Codex CLI's implementation and prompts seem tailored for a specific class of hosted, large-scale models (e.g. GPT, Gemini, Grok). But if you want to get good results with small, local models, everything - prompting, reasoning chains, output structure - often needs to be different.
So I built this with a few assumptions in mind:
- Write the tool specifically to run _locally_ out of the box, no inference API server required.
- Use model directly (currently for phi-4-mini via llama-cpp-python).
- Optimize the prompt and execution logic _per model_ to get the best performance.
Instead of forcing small models into a system meant for large, general-purpose APIs, I wanted to explore a local-first, model-specific alternative that's easy to install and extend — and free to run.
Sounds great! Although I would prefer Claude Code to be open sourced as it’s a tool that works best for Vibe coding. Albeit expensive using Anthropic‘s models via API. There is an inofficial clone though („Anon Kode“), but it’s not legitimate.
Unfortunately it does not. Where I can feed Claude Code with a file larger than 256k, Anon Code (like Roo) will complain that the file is too large, using Gemini 2.5 Pro.
I went with Phi as the default model because, after some testing, I was honestly surprised by how high the quality was relative to its size and speed. The responses felt better in some reasoning tasks-but were running on way less hardware.
What really convinced me, though, was the focus on the kinds of tasks I actually care about: multi-step reasoning, math, structured data extraction, and code understanding.There’s a great Microsoft paper on this: "Textbooks Are All You Need" and solid follow-ups with Phi‑2 and Phi‑3.
I saw pretty good reasoning quality with phi-4-mini. But alright - I’ll still run some tests with qwen2.5-coder and plan to add support for it next. Would be great to compare them side by side in practical shell tasks. Thanks so much for the pointer!
fair jab - haha; if we’re gonna go small, might as well go fully local and open. At least with phi-4-mini you don’t need an API key, and you can tweak/replace the model easily
Without any changes, you can already use Codex with a remote or local API by setting base URL and key environment variables.
Does it work for local though? It's my understanding this is still missing.
If your favorite LLM inference program can run a Chat Completions API.
Thanks for bringing that up - it's exactly why I approached it this way from the start.
Technically you can use the original Codex CLI with a local LLM - if your inference provider implements the OpenAI Chat Completions API, with function calling, etc. included.
But based on what I had in mind - the idea that small models can be really useful if optimized for very specific use cases - I figured the current architecture of Codex CLI wasn't the best fit for that. So instead of forking it, I started from scratch.
Here's the rough thinking behind it:
Codex CLI's implementation and prompts seem tailored for a specific class of hosted, large-scale models (e.g. GPT, Gemini, Grok). But if you want to get good results with small, local models, everything - prompting, reasoning chains, output structure - often needs to be different.So I built this with a few assumptions in mind:
Instead of forcing small models into a system meant for large, general-purpose APIs, I wanted to explore a local-first, model-specific alternative that's easy to install and extend — and free to run.i think this was made before that PR was merged into codex.
Good correction - while the SDK used has supported changing the API through environment variables for a long time, Codex only recently added Chat Completions support recently.
Maybe it was part of the reason that they accepted the PR. The fork would happen anyways if they don't allow any LLM.
A bit like how Android came after iPhone with open source implementation.
This is very convenient and nice! But I could not get it to work with the best small models available for Ollama for programming, like https://ollama.com/MFDoom/deepseek-coder-v2-tool-calling for example.
Thanks so much!
Was the model too big to run locally?
That’s one of the reasons I went with phi-4-mini - surprisingly high quality for its size and speed. It handled multi-step reasoning, math, structured data extraction, and code pretty well, all on modest hardware. Phi-1.5 / Phi-2 (quantized versions) also run on raspberry pi as others have demonstrated.
The models work fine with "ollama run" locally.
When trying out "phi4" locally with:
open-codex --provider ollama --full-auto --project-doc README.md --model phi4:latest
I get this error:
That's a really old model now. Even the old Qwen 2.5 coder 32b model is better than DSv2
I want to add support for qwen 2.5 next
QwQ-32 might be worth looking into also, as a high level planning tool.
Thank you so much!
Hopefully Qwen 3 and maybe if we're lucky Qwen 3 Coder might be out this week too.
Also GLM 4 is pretty amazing - https://www.reddit.com/r/LocalLLaMA/comments/1k4w9p2/i_uploa...
Thanks, I'll have a look
Why forking and use open codex when the original OpenAI opened it for multiple models? Just trying to understand.
Hey, that is a very good question, I have answered that before. I hope you don't mind, if I simply copy paste my previous answer:
Technically you can use the original Codex CLI with a local LLM - if your inference provider implements the OpenAI Chat Completions API, with function calling, etc. included.
But based on what I had in mind - the idea that small models can be really useful if optimized for very specific use cases - I figured the current architecture of Codex CLI wasn't the best fit for that. So instead of forking it, I started from scratch.
Here's the rough thinking behind it:
Codex CLI's implementation and prompts seem tailored for a specific class of hosted, large-scale models (e.g. GPT, Gemini, Grok). But if you want to get good results with small, local models, everything - prompting, reasoning chains, output structure - often needs to be different. So I built this with a few assumptions in mind: Instead of forcing small models into a system meant for large, general-purpose APIs, I wanted to explore a local-first, model-specific alternative that's easy to install and extend — and free to run.Sounds great! Although I would prefer Claude Code to be open sourced as it’s a tool that works best for Vibe coding. Albeit expensive using Anthropic‘s models via API. There is an inofficial clone though („Anon Kode“), but it’s not legitimate.
I believe anon-kode is a decompiled Claude Code, so it should work identically when paired with Claude.
Unfortunately it does not. Where I can feed Claude Code with a file larger than 256k, Anon Code (like Roo) will complain that the file is too large, using Gemini 2.5 Pro.
curious why you went with Phi as the default models, that seems a bit unusual compared to current trends
I went with Phi as the default model because, after some testing, I was honestly surprised by how high the quality was relative to its size and speed. The responses felt better in some reasoning tasks-but were running on way less hardware.
What really convinced me, though, was the focus on the kinds of tasks I actually care about: multi-step reasoning, math, structured data extraction, and code understanding.There’s a great Microsoft paper on this: "Textbooks Are All You Need" and solid follow-ups with Phi‑2 and Phi‑3.
agreed - thought the qwen2.5-coder was kind of standard non-reasoning small line of coding models right now
I saw pretty good reasoning quality with phi-4-mini. But alright - I’ll still run some tests with qwen2.5-coder and plan to add support for it next. Would be great to compare them side by side in practical shell tasks. Thanks so much for the pointer!
Don't forget https://ollama.com/library/deepcoder which ranks really well for its size
Great work making Codex easily accessible with open-source LLMs – really excited to try it!
At least it can't be worse than the original codex using o4-mini.
fair jab - haha; if we’re gonna go small, might as well go fully local and open. At least with phi-4-mini you don’t need an API key, and you can tweak/replace the model easily
Cool project! It's awesome to see someone taking on the challenge of a fully local Codex alternative.
> So I rewrote the whole thing from scratch using Python
So this isn't really codex then?
but what about the Codex Giggas my niggas?
Codex merged in to allow multiple providers today - https://github.com/openai/codex/pull/247
Sorry, does that mean I can use anthropic and gemini with codex? And switch during the session?
yes