An Ahead of AI article by Sebastian Raschka sharing lessons from hundreds of experiments on finetuning LLMs with low-rank adaptation (LoRA), billed as an essential technique for efficiently training custom open-source LLMs. It covers LoRA consistency, QLoRA compute-memory trade-offs, learning rate schedulers, Adam versus SGD, multiple epochs, enabling LoRA for more layers, and balancing rank and alpha.
Apache MXNet
Apache MXNet is a deep learning framework mixing symbolic and imperative programming. It includes a dynamic dependency scheduler that parallelizes symbolic and imperative operations, with a graph optimization layer for symbolic execution. The repository was archived on Nov 17, 2023, and is read-only.
Features
- Lightweight, memory-efficient, and portable to smart devices Source: github.com
- This repository was archived by the owner on Nov 17, 2023 Source: github.com
Links
Tags
-
0An Ahead of AI article by Sebastian Raschka sharing lessons from hundreds of experiments on finetuning LLMs with low-rank adaptation (LoRA), billed as an essential technique for efficiently training custom open-source LLMs. It covers LoRA consistency, QLoRA compute-memory trade-offs, learning rate schedulers, Adam versus SGD, multiple epochs, enabling LoRA for more layers, and balancing rank and alpha.
-
0Flux is a machine learning library built as a pure-Julia stack, offering lightweight abstractions over Julia's native GPU and automatic differentiation support. It is compiled and eager, and existing Julia libraries can be incorporated into Flux models. Its ecosystem includes probabilistic programming, graph neural networks, computer vision and natural language processing packages, and it is maintained by a community team.
-
0Magenta is an open source research project exploring the role of machine learning as a tool in the creative process, focused on art and music. It includes Magenta Studio, a collection of music plugins for Ableton Live, and Magenta RealTime 2, an open model and real-time neural synthesis engine.
-
0gotoHuman is a platform for supervising AI agents, letting workflows request human review of critical steps. Reviewers approve and edit AI-generated content in an Agent Inbox, with email and Slack notifications, and completed reviews return to the workflow via webhook. Review types are configured in a no-code editor and can capture text, images, markdown and JSON.
-
0An index page from LabLab for discovering AI technologies and related topics. Each technology page is described as offering tutorials, upcoming hackathons, frameworks and boilerplates. Listed technologies include GPT-4, ChatGPT, BERT, AI21 Labs, Anthropic, AWS, Cohere, Hugging Face and DeepSeek.
-
0A visualisation of major large-language models ranked by performance using the MMLU benchmark, which consists of 16,000 multiple-choice questions across 57 academic subjects. It also presents selected lawsuits filed against AI companies and notes that MMLU has some critiques.
Community notes
Reviews and discussion
No reviews yet
Be the first to share how this project worked for you.
Join the discussion
Sign in with a verified account to leave a review or join the discussion.
Sign in to review