I wonder how much of this is directly related to age (biological aging), and how much is just someone's brain becoming "full" due to more memories getting added every year?
It seems that memories must be stored as embeddings with single multi-neuron assemblies (cortical columns?) storing multiple embeddings as a kind of contents-addressable memory that is able to keep memories distinct due to the very high dimensional space (# neurons per assembly) being used. However, you'd expect that at some point if you store too many memories in a single assembly the recall accuracy is going to go down.
You'd expect that with big brains being so costly, evolution has only equipped us with brains big enough to store a lifetime of memories, so it would be odd if memory didn't suffer as we get old.
To make a computer analogy, it's a bit like a hash table getting too full. Say you had a hash table without any overflow mechanism... up to a point recall may still be pretty good, but as the table gets closer to full there will be more hash collisions and likelyhood of "false recall". Obviously the brain is not a computer, but the analogy may hold up reasonably well if you consider the hash table keys and values as embeddings and the store operation being an embedding merge rather than overwrite.
The brain does not have the Von Neumann bottleneck. Unlike most current digital systems, the brain doesn’t have a separate memory registry it needs to pull from.
Engrams, that is, the physical trace of a memory, are not stable through life. They start out in the hippocampus, but as the stimulus recedes in time without reinforcement, it moves away.
No evidence exists though that the memory is encoded in one set of cells. This spatial segregation of memory is the worst hangover from the “brain is a computer” analogy. Even if it is, why in the world would it be like our digital devices which specifically have the Von Neumann bottleneck? In biology, memory and processing are not segregated.
There’s growing evidence the memory is much more distributed over the network, and is recomposed based on salience overlap with a new stimulus.
Another factor to keep in mind is circadian rhythms. There’s growing evidence for how much the memory system and timekeeping system overlap, at a molecular level. Every neuron (and other cell) has an intrinsic clock that ticks at roughly 24 hours, and continues to do so even in total darkness.
When you encode the memory has a lot to say, based on your chronotype, on how and how well you will remember it. Same with learning: there’s a time of day based variation.
Sleep, and dreaming, is when these memories seem to get replayed and critical features and connections are incorporated into the system and its regime, awaiting the right triggers to access a state similar to when the memory formed.
I’m stitching across a lot of different research, and I want to be clear many aspects of this system are not yet fully worked out.
But what we do know points to a system that works with different physical and algorithmic priors, and the dynamics are sharply distinct from current digital computers.
This is really fascinating. The actual mechanisms of the human mind are distinct from computer systems, yet there are some parallels.
There are some hints that increased memory access times scale with the amount of information the brain has stored vs the more typical narrative that aging decreases the capabilities of the brain.
> Our results indicate that older adults'; performance on cognitive tests reflects the predictable consequences of learning on information-processing, and not cognitive decline. We consider the implications of this for our scientific and cultural understanding of aging.
The paper certainly has its limitations. Only 61 participants, with almost nobody between 30 and 50, so we shouldn’t read the age trend as a decline across lifespan. What’s more interesting than the title suggests (and I find the title forcing the conclusion a bit) is that the attention measures were not linked to age or the brain patterns at all.
Then again, i'm 45 and I've been losing my keys and my IDs since I was 20
The entire article is AI-generated.I wish such sources would include the prompting used as well as the model so a (human) reader can better evaluate the document.
> This process includes multiple checks using different LLM platforms, human editorial verification, and cross-referencing with original sources to ensure accuracy and eliminate false information, misleading content, fabricated content, bias, and unreasonable speculations or editorialization
How much of this, I wonder, is a function of the fact that our circadian rhythms get less robust with aging. The circadian clock hugely influences learning and memory processes, gating when you can learn and how much, and shaping the storage and recall also.
We know that with age the amplitude of these rhythms can decrease, as can the synchrony between cells.
This kind of mid-management seems to have at least some circadian component, and it would have been great if they looked to see if the effects were equally bad at all times of day, and knew the chronotype of the participants to use as a reference. I’d love to see if every test participant was tested at their cognitive peak, too.
My theory behind this is that when you encounter something new/novel it is distinct enough to remember -- where you were when you learned of 9/11; the days around the start of covid/lockdowns; being in a car accident; getting married; etc..
However, when you experience the same thing repetitively -- having breakfast; getting ready for school/work/etc. -- it doesn't make sense for your brain to remember each of these events as distinct events but to mush them together into a single or combined memory. This may be due to how the brain indexes/references memories and that memories with the same references get combined together.
This is also partly why time appears to go faster when you get older -- the same day-to-day events blend together but the key milestones (holidays, etc.) stick out.
Worth keeping in mind also is that memories are not a reliable facsimile of objective reality.
Instead, they are selective representations of parts of a subjective experience. It also means that memories aren’t static, but shift over time and can even be imagined.
This comes as absolutely no surprise to me -- my mother was recently transferred into memory care, and the changes to her memories are definitely showing some interesting changes. The foremost was that her new "apartment" is shared with one other person, gives off a bit of a dorm room vibe. First day, she was talking about how excited she was to be going back to school and seeing her boyfriends again(?!). I said, "Oh, like Doug? [my dad]". Her response? "Ew. No."
But, she's come around on that, and recently been upset about the affair he's been having (they divorced in 1980), and is irate over him having another kid with his new wife (1983), but isn't surprised that I haven't spoken to him in 12 years. She's never surprised at how I've aged, but doesn't remember her grandkids even exist and insists on meeting them soon (but remembers them when she sees them). Two weeks ago, she called to yell at me for things I did in high school (1990). At this point, I seem to be the anchor for everything recent, but when I'm not around physically, it's scattered everywhere else.
It's hard to write this, but before my grandmother died about 45 years ago, it felt like she'd lost almost all her memories, but was easy to take care of. She was in her late 80's.
My mom & aunt took yearly turns keeping her in their homes. If you kept her away from the stove, she still seemed quite happy helping fix her own breakfast, taking her meds, etc. Give her a warm washrag after, and she'd wipe down the table and then proceed to putter all around the kitchen, wiping and straightening. She preferred cartoons on TV (easier to see, I guess), and was always glad to fold clean towels or clothes, if she had the chance.
Once, she actually recognized me, and remembered my name. It was a very precious 5 minute chat, but then the memories were gone. Still, I'll always remember her that way, because it was just like how I remembered her, when I was a child.
Man, that makes me happy. I'm guessing your grandmother was a very upbeat, happy person who dealt with change well. My mother's the opposite, so now every little thing going wrong is the end of the world. It's rough, but I also get to see these moments where I can see how she was when I was little/before I was born, and those are moments that I do really enjoy.
Compaction is not even an intrinsic function of an LLM, but of it's harness or whatever code is calling it. When an LLM compacts something it's just creating a summary of the previous conversation using the same mechanism that it uses to generate any other text. It doesn't "forget" the text from before the summary, it just no longer receives it when it is invoked.
In contrast, when we forget something we are not summarizing anything, we are just losing the ability to recall something. We might remember what we have forgotten later on, showing that it is still somewhere in the mind, and that the reason we have forgotten it is not necessarily because it has been discarded.
Even then, what this article is talking about is not even normal forgetting as a function of the brain. It's talking about a degenerative and dysfunctional form of memory loss where different memories are confused with each other.
I wonder how much of this is directly related to age (biological aging), and how much is just someone's brain becoming "full" due to more memories getting added every year?
It seems that memories must be stored as embeddings with single multi-neuron assemblies (cortical columns?) storing multiple embeddings as a kind of contents-addressable memory that is able to keep memories distinct due to the very high dimensional space (# neurons per assembly) being used. However, you'd expect that at some point if you store too many memories in a single assembly the recall accuracy is going to go down.
You'd expect that with big brains being so costly, evolution has only equipped us with brains big enough to store a lifetime of memories, so it would be odd if memory didn't suffer as we get old.
To make a computer analogy, it's a bit like a hash table getting too full. Say you had a hash table without any overflow mechanism... up to a point recall may still be pretty good, but as the table gets closer to full there will be more hash collisions and likelyhood of "false recall". Obviously the brain is not a computer, but the analogy may hold up reasonably well if you consider the hash table keys and values as embeddings and the store operation being an embedding merge rather than overwrite.
The brain does not have the Von Neumann bottleneck. Unlike most current digital systems, the brain doesn’t have a separate memory registry it needs to pull from.
Engrams, that is, the physical trace of a memory, are not stable through life. They start out in the hippocampus, but as the stimulus recedes in time without reinforcement, it moves away.
No evidence exists though that the memory is encoded in one set of cells. This spatial segregation of memory is the worst hangover from the “brain is a computer” analogy. Even if it is, why in the world would it be like our digital devices which specifically have the Von Neumann bottleneck? In biology, memory and processing are not segregated.
There’s growing evidence the memory is much more distributed over the network, and is recomposed based on salience overlap with a new stimulus.
Another factor to keep in mind is circadian rhythms. There’s growing evidence for how much the memory system and timekeeping system overlap, at a molecular level. Every neuron (and other cell) has an intrinsic clock that ticks at roughly 24 hours, and continues to do so even in total darkness.
When you encode the memory has a lot to say, based on your chronotype, on how and how well you will remember it. Same with learning: there’s a time of day based variation.
Sleep, and dreaming, is when these memories seem to get replayed and critical features and connections are incorporated into the system and its regime, awaiting the right triggers to access a state similar to when the memory formed.
I’m stitching across a lot of different research, and I want to be clear many aspects of this system are not yet fully worked out.
But what we do know points to a system that works with different physical and algorithmic priors, and the dynamics are sharply distinct from current digital computers.
This is really fascinating. The actual mechanisms of the human mind are distinct from computer systems, yet there are some parallels.
There are some hints that increased memory access times scale with the amount of information the brain has stored vs the more typical narrative that aging decreases the capabilities of the brain.
> Our results indicate that older adults'; performance on cognitive tests reflects the predictable consequences of learning on information-processing, and not cognitive decline. We consider the implications of this for our scientific and cultural understanding of aging.
[0] https://pubmed.ncbi.nlm.nih.gov/24421073/
I am not a biologist so I cannot say this authoritatively, but I’m fairly certain that’s not how neuroplasticity works.
I only mentioned neural assemblies storing multiple embeddings - what about that contradicts your understanding of neuroplasticity?
How so?
The paper certainly has its limitations. Only 61 participants, with almost nobody between 30 and 50, so we shouldn’t read the age trend as a decline across lifespan. What’s more interesting than the title suggests (and I find the title forcing the conclusion a bit) is that the attention measures were not linked to age or the brain patterns at all.
Then again, i'm 45 and I've been losing my keys and my IDs since I was 20
>I find the title forcing the conclusion a bit
The entire article is AI-generated.I wish such sources would include the prompting used as well as the model so a (human) reader can better evaluate the document.
> Reviewed by Patrisha Antonaros
There is a responsible signer.
If you think their method can be improved, write at https://studyfinds.com/contact/
They disclose an "AI policy" - https://studyfinds.com/ai-policy/
> This process includes multiple checks using different LLM platforms, human editorial verification, and cross-referencing with original sources to ensure accuracy and eliminate false information, misleading content, fabricated content, bias, and unreasonable speculations or editorialization
My dreams imprint stronger than reality. This has the unfortunate effect of sometimes recalling memories as real, when in fact it was all a dream.
Unless the dreams are the reality and the reality is the fake!
The new version of this tale is that reality is a simulation. Reality being a dream is so 4th Century China
How much of this, I wonder, is a function of the fact that our circadian rhythms get less robust with aging. The circadian clock hugely influences learning and memory processes, gating when you can learn and how much, and shaping the storage and recall also.
We know that with age the amplitude of these rhythms can decrease, as can the synchrony between cells.
This kind of mid-management seems to have at least some circadian component, and it would have been great if they looked to see if the effects were equally bad at all times of day, and knew the chronotype of the participants to use as a reference. I’d love to see if every test participant was tested at their cognitive peak, too.
For those getting locked out due to using an ad blocker: https://archive.ph/rPdSW
And the actual study: https://academic.oup.com/cercor/article/36/7/bhag114/8758582
My theory behind this is that when you encounter something new/novel it is distinct enough to remember -- where you were when you learned of 9/11; the days around the start of covid/lockdowns; being in a car accident; getting married; etc..
However, when you experience the same thing repetitively -- having breakfast; getting ready for school/work/etc. -- it doesn't make sense for your brain to remember each of these events as distinct events but to mush them together into a single or combined memory. This may be due to how the brain indexes/references memories and that memories with the same references get combined together.
This is also partly why time appears to go faster when you get older -- the same day-to-day events blend together but the key milestones (holidays, etc.) stick out.
We do know that memories get rewritten every time they are remembered
Worth keeping in mind also is that memories are not a reliable facsimile of objective reality.
Instead, they are selective representations of parts of a subjective experience. It also means that memories aren’t static, but shift over time and can even be imagined.
Compacting ———-~~~~~~~ 40%
Retrocompomorphism
Deticulating splines...
Clearly the hash table is filling up so you get more hash collisions.
This comes as absolutely no surprise to me -- my mother was recently transferred into memory care, and the changes to her memories are definitely showing some interesting changes. The foremost was that her new "apartment" is shared with one other person, gives off a bit of a dorm room vibe. First day, she was talking about how excited she was to be going back to school and seeing her boyfriends again(?!). I said, "Oh, like Doug? [my dad]". Her response? "Ew. No."
But, she's come around on that, and recently been upset about the affair he's been having (they divorced in 1980), and is irate over him having another kid with his new wife (1983), but isn't surprised that I haven't spoken to him in 12 years. She's never surprised at how I've aged, but doesn't remember her grandkids even exist and insists on meeting them soon (but remembers them when she sees them). Two weeks ago, she called to yell at me for things I did in high school (1990). At this point, I seem to be the anchor for everything recent, but when I'm not around physically, it's scattered everywhere else.
It's hard to write this, but before my grandmother died about 45 years ago, it felt like she'd lost almost all her memories, but was easy to take care of. She was in her late 80's.
My mom & aunt took yearly turns keeping her in their homes. If you kept her away from the stove, she still seemed quite happy helping fix her own breakfast, taking her meds, etc. Give her a warm washrag after, and she'd wipe down the table and then proceed to putter all around the kitchen, wiping and straightening. She preferred cartoons on TV (easier to see, I guess), and was always glad to fold clean towels or clothes, if she had the chance.
Once, she actually recognized me, and remembered my name. It was a very precious 5 minute chat, but then the memories were gone. Still, I'll always remember her that way, because it was just like how I remembered her, when I was a child.
Man, that makes me happy. I'm guessing your grandmother was a very upbeat, happy person who dealt with change well. My mother's the opposite, so now every little thing going wrong is the end of the world. It's rough, but I also get to see these moments where I can see how she was when I was little/before I was born, and those are moments that I do really enjoy.
I feel for you. Even though it's entertaining to read, that must be really hard to go through for you.
Sorry to hear that - my dad spent the last few years of his life in a care home, and I can very much relate.
Is so fascinating and horrifying to read this. We still have to learn so much about how the brain works
It's interesting to compare and contrast LLMs and biological brains. Both use a form of "compaction".
Compaction is not even an intrinsic function of an LLM, but of it's harness or whatever code is calling it. When an LLM compacts something it's just creating a summary of the previous conversation using the same mechanism that it uses to generate any other text. It doesn't "forget" the text from before the summary, it just no longer receives it when it is invoked.
In contrast, when we forget something we are not summarizing anything, we are just losing the ability to recall something. We might remember what we have forgotten later on, showing that it is still somewhere in the mind, and that the reason we have forgotten it is not necessarily because it has been discarded.
Even then, what this article is talking about is not even normal forgetting as a function of the brain. It's talking about a degenerative and dysfunctional form of memory loss where different memories are confused with each other.
That's largely because we intentionally model llm "brains" after the human brain.
Huffman Coding! :-)
FYI this is LLM slop. Consider the information to be entirely unfounded.
What, the paper from Mekbib and McDonough, or the divulgation from Antonaros?
Supposing it is the submitted (under Antonaros), see https://news.ycombinator.com/item?id=49538906
As a person who used to be a super genius then turned 40 and is now only a regular genius aging has been very hard to get used to.
/compact
This a covert op to buttress LLM support by planting this article and having people say it's just like compacting. The hype is getting more elaborate.