Check Deterministic State Machine idea - basically, on prod regulated / high stakes env. you don't want LLM to run almost anything directly.
You rather need exact reproducebale, tracable path of decisions. Either a hard-coded graph where LLM is a point of contact for the customer/user and parsing, function call etc. is all done deterministic way.
OR you use smth like JEV to decide about the next path in the chain (under the condition that JEV is trained for that path only and its only job is to do the best routing possible with close to 0 costs).
Check Deterministic State Machine idea - basically, on prod regulated / high stakes env. you don't want LLM to run almost anything directly.
You rather need exact reproducebale, tracable path of decisions. Either a hard-coded graph where LLM is a point of contact for the customer/user and parsing, function call etc. is all done deterministic way.
OR you use smth like JEV to decide about the next path in the chain (under the condition that JEV is trained for that path only and its only job is to do the best routing possible with close to 0 costs).
It only makes sense if traditional LLMs are either too expensive or too slow.
That’s it, that’s the only use case.
However there are use cases that meet that requirement, for example, near instant labeling of dynamically generated content on the screen.
posttrained models for a specific purpose usually outperforms model that are not posttrained for that purpose