We've developed a model router and a token optimizer exposed through a simple gateway compatible with OpenAI SDK. The model router would automatically route to the best model for the prompt and the token optimizer would optimize the tokens used in a thread to only the ones relevant to the query; essentially expanding the context window limit, reducing cost and increasing both inference and reliability. We would love your feedback as we continue to develop it.
We've developed a model router and a token optimizer exposed through a simple gateway compatible with OpenAI SDK. The model router would automatically route to the best model for the prompt and the token optimizer would optimize the tokens used in a thread to only the ones relevant to the query; essentially expanding the context window limit, reducing cost and increasing both inference and reliability. We would love your feedback as we continue to develop it.