Directory
Published directory entries matching your search.
A research project on high-resolution video synthesis with latent diffusion models. It pre-trains an image latent diffusion model, then adds a temporal dimension and fine-tunes on encoded image sequences to produce video. Applications include simulation of in-the-wild driving data and text-to-video creative content creation. Updaters are temporally aligned into video super-resolution models.
AudioCraft is a code base for generative audio tasks including music, sound effects, and compression, trained on raw audio signals. MusicGen produces music samples from text inputs, and AudioGen produces audio from environmental sounds via text-to-sound generation. The models use the EnCodec neural audio codec to learn discrete audio tokens from waveforms.
A curated GitHub list of resources on audio-driven talking face generation. It organizes research papers and projects, including encoder-decoder and StyleGAN-based methods, pose-controllable and lip-sync work, and 3D model-based approaches.
LocalGPT is an on-premise document intelligence platform that lets users chat with their documents on their local device using GPT models. It supports PDF, DOCX, HTML, Markdown, and TXT files and combines dense vector search with LanceDB full-text search and a cross-encoder reranker. It runs with a plain Python RAG core and offers generation models via Ollama.
MusicGen is a single-stage auto-regressive Transformer model for music generation trained over a 32kHz EnCodec tokenizer with four codebooks, part of the AudioCraft library. It supports text-to-music and text-plus-melody-to-music models ranging from 300M to 3.3B parameters. Running it locally requires a GPU, with at least 16 GB of memory recommended for medium-sized models.
NALP is a Python library covering Natural Adversarial Language Processing. It combines natural language processing and adversarial learning, from embeddings to neural networks. The package offers corpus loading for raw text and audio, encoders, and pre-loaded neural network models. Version 3.0.2 requires Python 3.11+.
NASGym is a proof-of-concept OpenAI Gym environment for Neural Architecture Search. It implements reinforcement learning steps for NAS and encodes networks using the Neural Structure Code from BlockQNN. A network is modeled as a list of NSCs, an action adds a layer, and the reward is accuracy after early-stop training. CIFAR-10 is the default dataset. The code and documentation may contain minor bugs.
Phenaki is a Google Research model that synthesizes realistic videos from textual prompt sequences. It uses an encoder-decoder tokenizer for variable-length videos and a transformer that translates text embeddings to video tokens, trained jointly on image-text and video-text data.
A gentle illustrated introduction by Jay Alammar to how Stable Diffusion works, covering the components of Stable Diffusion, what diffusion is, painting images by removing noise, diffusion on compressed latent data, and the text encoder. It focuses on image generation from text only (text2img) and includes translations into Chinese and Vietnamese.
timm is a collection of PyTorch image encoders and backbones, including train, eval, inference and export scripts with pretrained weights. It covers architectures such as ResNet, EfficientNet, Vision Transformer, MobileNet and ConvNeXt.
