Directory
Published directory entries matching your search.
Art of Memory is an educational platform dedicated to memory improvement. It offers free training resources for mnemonic techniques including memory palaces, peg lists, spaced repetition and advanced mnemonics. Users can train for memory competitions, access guides for academic and everyday memorization, and join a community forum for memory enthusiasts.
An Ahead of AI article by Sebastian Raschka sharing lessons from hundreds of experiments on finetuning LLMs with low-rank adaptation (LoRA), billed as an essential technique for efficiently training custom open-source LLMs. It covers LoRA consistency, QLoRA compute-memory trade-offs, learning rate schedulers, Adam versus SGD, multiple epochs, enabling LoRA for more layers, and balancing rank and alpha.
StreamingLLM is a technique from researchers at Meta AI, MIT and Carnegie Mellon University that extends a language model's context to millions of tokens without large compute or memory resources. The article describes it as addressing the fixed context length of models such as Llama-2, which has a context of approximately 4,000 tokens.
