So they strengthened the hypothesis that gradual learning is more efficient than bulk learning or unstructured learning: a properly built NN has to progress like a human (learn to count, then arithmetic, then geometry, then algebra, then trigonometry, then calculus...) for efficiency.
What about training language models? Has anybody tried to train LLMs with an elementary subset of the corpus, then increasingly with more complex subsets?
So they strengthened the hypothesis that gradual learning is more efficient than bulk learning or unstructured learning: a properly built NN has to progress like a human (learn to count, then arithmetic, then geometry, then algebra, then trigonometry, then calculus...) for efficiency.
What about training language models? Has anybody tried to train LLMs with an elementary subset of the corpus, then increasingly with more complex subsets?