Spent the weekend turning small LLMs into decision models, with some good results and a lot of learnings along the way. Topping the decision index leaderbard for their categories https://huggingface.co/spaces/multimodalart/jev-decision-ind...
Sharing a 0.8B model, when evaluated on the decision Index it tops the sub-1B category, 45.7% above the best other Qwen3.5-0.8B fine-tune on the leaderboard.
Most of work was data calibration, finding external datasets and readapting them towards this scenario, so i expect a lot of improvements and work like this to come from community and push these numbers even higher. Even more if we get qwen 3.8 releases for these model categories.
Spent the weekend turning small LLMs into decision models, with some good results and a lot of learnings along the way. Topping the decision index leaderbard for their categories https://huggingface.co/spaces/multimodalart/jev-decision-ind... Sharing a 0.8B model, when evaluated on the decision Index it tops the sub-1B category, 45.7% above the best other Qwen3.5-0.8B fine-tune on the leaderboard.
Most of work was data calibration, finding external datasets and readapting them towards this scenario, so i expect a lot of improvements and work like this to come from community and push these numbers even higher. Even more if we get qwen 3.8 releases for these model categories.