It may seem useful at first, but eventually you'll hit a wall where this doesn't work, and you have to add another memory technique. Eventually you end up with a complex multi-layered system, because what people want by "memory" is actually 10 different things which all need their own solution.
Is the date really useful? Usually I want the AI to remember rules, like "always do X before committing Y" and a timestamp doesn't change anything there.
If you can make an agent reliably follow instructions, you've solved alignment completely. So I'd be particularly surprised if you can square that circle as a hobby project!
I can't resist the opportunity to paste some of the text I wrote in the Anthropic AI interview last December on an idea I had about adding temporal awareness to Claude.
Anthropic: Let's focus on AI with capabilities that feel within reach—not necessarily limited to exactly what exists today, but the kind of AI tools you can reasonably imagine existing in the near future. What would you want AI to help you with?
Me: There is something fundamental missing in today's capabilities that is well within reach of the tech today. I thought I would make a Github repo with a prototype but too much in my life right now. If a human in Anthropic reads this, please let me know if you run with this idea: The missing thing is lack of the bot's initiative. It is always the human who takes initiative and writes something, then the bot makes a (perhaps) elaborate response and shuts up. So something seemingly very simple like telling the bot: "Write me a limerick about a swedish chef, wait 50 seconds, then write a recipe for a danish christmas dinner not involving duck". The bot will not be able to do this. It has no concept of time. No ability to take initiative. There are many ways to frame this but you get the point. I am confident there must be a way to implement this. I read some research on this but all articles I could find are (in my opinion) unnecessarily complicated. The current chat style is Query, Response. The bot waits forever for the next input. This is not natural. When you (A) chat with customer support (B) (a real human), the context is such that it is natural for both A and B to give many inputs in a row before the other party gives their input. And, crucially, if either A or B takes too long to give input, the other party will react to that. This applies to most conversations between humans. So a way to get this behavior from a bot is as follows: the moment either the human or the bot gives any input, the chat system tries to understand the current context, and immediately decides how long time it is natural to wait until the bot should say something by its own initiative, and what it is that it should say. Then the system has a simple timer in the background that waits this amount of time and then says the thing. Very simple. Would it work? I have no idea, but it is worth a shot. This should probably be a separate mode, Proactive Messaging. it could be linked to an agent dynamically. So for example, i have a conversation in Proactive Messaging asking claude to follow the lowest price of something and let me now if it goes below a certain value. Then a few weeks down the line, i get a popup from claude about this if it happens. There should also be a menu in this case with what tasks like this have been defined.
There is a KISS way to do this with our current models:
use a time string that updates:
every second (that it is not responding) erase a byte string at the end, then send a new byte string (that represents time from last response): bot either outputs a null terminating token, or if the number gets high enough, instruct/train it to take initiative then.
> So for example, i have a conversation in Proactive Messaging asking claude to follow the lowest price of something and let me now if it goes below a certain value. Then a few weeks down the line, i get a popup from claude about this if it happens. There should also be a menu in this case with what tasks like this have been defined.
I've been looking for a new cooker. I have a Hermes agent running on VM that does this. I told it to keep an eye on prices. It created a cron job that triggers the scrapes and messages me if anything interesting happened since the last time it checked.
Claude can definitely wait 50 seconds and then do a second thing. I see it all the time when running tools that never stop but iterate towards a goal. It can either run a bash `sleep 50` or use the ScheduleWakeup tool.
I’ve seen some good research where xx% of the time, all LLMs essentially just get lazy and won’t actually look things up.
Been working on an informational retrieval system similar, but endless Nerfs combined with non deterministic and finicky behaviors make it hard to reliably depend on.
It may seem useful at first, but eventually you'll hit a wall where this doesn't work, and you have to add another memory technique. Eventually you end up with a complex multi-layered system, because what people want by "memory" is actually 10 different things which all need their own solution.
Is the date really useful? Usually I want the AI to remember rules, like "always do X before committing Y" and a timestamp doesn't change anything there.
Oh cool. I'm working on exactly that here!
https://innerloop.works/breadcrumb
If you can make an agent reliably follow instructions, you've solved alignment completely. So I'd be particularly surprised if you can square that circle as a hobby project!
Its working pretty well to be honest! here’s some more background :)
https://innerloop.works/blog/introducing-breadcrumb
I can't resist the opportunity to paste some of the text I wrote in the Anthropic AI interview last December on an idea I had about adding temporal awareness to Claude.
Anthropic: Let's focus on AI with capabilities that feel within reach—not necessarily limited to exactly what exists today, but the kind of AI tools you can reasonably imagine existing in the near future. What would you want AI to help you with?
Me: There is something fundamental missing in today's capabilities that is well within reach of the tech today. I thought I would make a Github repo with a prototype but too much in my life right now. If a human in Anthropic reads this, please let me know if you run with this idea: The missing thing is lack of the bot's initiative. It is always the human who takes initiative and writes something, then the bot makes a (perhaps) elaborate response and shuts up. So something seemingly very simple like telling the bot: "Write me a limerick about a swedish chef, wait 50 seconds, then write a recipe for a danish christmas dinner not involving duck". The bot will not be able to do this. It has no concept of time. No ability to take initiative. There are many ways to frame this but you get the point. I am confident there must be a way to implement this. I read some research on this but all articles I could find are (in my opinion) unnecessarily complicated. The current chat style is Query, Response. The bot waits forever for the next input. This is not natural. When you (A) chat with customer support (B) (a real human), the context is such that it is natural for both A and B to give many inputs in a row before the other party gives their input. And, crucially, if either A or B takes too long to give input, the other party will react to that. This applies to most conversations between humans. So a way to get this behavior from a bot is as follows: the moment either the human or the bot gives any input, the chat system tries to understand the current context, and immediately decides how long time it is natural to wait until the bot should say something by its own initiative, and what it is that it should say. Then the system has a simple timer in the background that waits this amount of time and then says the thing. Very simple. Would it work? I have no idea, but it is worth a shot. This should probably be a separate mode, Proactive Messaging. it could be linked to an agent dynamically. So for example, i have a conversation in Proactive Messaging asking claude to follow the lowest price of something and let me now if it goes below a certain value. Then a few weeks down the line, i get a popup from claude about this if it happens. There should also be a menu in this case with what tasks like this have been defined.
There is a KISS way to do this with our current models:
use a time string that updates:
every second (that it is not responding) erase a byte string at the end, then send a new byte string (that represents time from last response): bot either outputs a null terminating token, or if the number gets high enough, instruct/train it to take initiative then.
> So for example, i have a conversation in Proactive Messaging asking claude to follow the lowest price of something and let me now if it goes below a certain value. Then a few weeks down the line, i get a popup from claude about this if it happens. There should also be a menu in this case with what tasks like this have been defined.
I've been looking for a new cooker. I have a Hermes agent running on VM that does this. I told it to keep an eye on prices. It created a cron job that triggers the scrapes and messages me if anything interesting happened since the last time it checked.
Claude can definitely wait 50 seconds and then do a second thing. I see it all the time when running tools that never stop but iterate towards a goal. It can either run a bash `sleep 50` or use the ScheduleWakeup tool.
Have you thought about using llm-wiki? It's pretty powerful and works well.
I’ve been thinking about going that route.
I’ve seen some good research where xx% of the time, all LLMs essentially just get lazy and won’t actually look things up.
Been working on an informational retrieval system similar, but endless Nerfs combined with non deterministic and finicky behaviors make it hard to reliably depend on.
How’s your luck been with LLm-wiki?
looks like a nifty information retrieval approach. cool. temporal/versioned chronology is absolutely useful.