cool, you mention cuts pay off after X calls, when the agent needs something that got cut, does it just re-read it? does that ever cost more than keeping it? or have some measurements around that ?
Right now the cost estimator assumes payoff needs to be met by 20 calls but its very conservative so it typically beats that. There are cases where we cut context and then it needs to be re-read e.g. https://thinker.sh/see-the-data?run=click-1. In this example it stays cost-effective since the re-read is small, but we're continuing to experiment with how long we wait before we consider context to be safe to remove.
cool, you mention cuts pay off after X calls, when the agent needs something that got cut, does it just re-read it? does that ever cost more than keeping it? or have some measurements around that ?
Right now the cost estimator assumes payoff needs to be met by 20 calls but its very conservative so it typically beats that. There are cases where we cut context and then it needs to be re-read e.g. https://thinker.sh/see-the-data?run=click-1. In this example it stays cost-effective since the re-read is small, but we're continuing to experiment with how long we wait before we consider context to be safe to remove.