Models & Modalities

Foundation models, model toolkits, and multimodal capabilities.

35Projects5Subcategory45Tags
TrackedUnavailableArchivedInactive

Foundation model families and model releases.

Keras

Keras is a high-level deep learning API that enables fast experimentation with neural networks, running on top of TensorFlow and providing an intuitive interface for building and training models.

Stars --Last Commit --

PaddlePaddle

An open-source deep learning platform developed by Baidu, providing a comprehensive ecosystem for machine learning and deep learning research and production.

Stars --Last Commit --

utopia

An open-source enterprise world model from DeepLethe: a bitemporal knowledge graph that learns passively from your material, keeps ontology and time awareness in the base layer, and deploys offline as a single Rust binary plus Postgres.

Stars --Last Commit --

Tooling for model conversion, quantization, and packaging.

Detectron2

Facebook AI Research's next-generation object detection and segmentation library, offering state-of-the-art algorithms and a rich model zoo.

Stars --Last Commit --

Hugging Face Transformers

The model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal, for both inference and training.

Stars --Last Commit --

Cross-modal modeling for text, image, audio, and video.

Chandra

Chandra is a high‑accuracy OCR model that converts images and PDFs into structured outputs with layout information.

Stars --Last Commit --

ElevenLabs UI

ElevenLabs UI is a component library and custom registry built on top of shadcn/ui to help build multimodal agent interfaces faster.

Stars --Last Commit --

LeRobot

An open-source robotics library providing datasets, pretrained policies and simulation environments for reproducible robot learning and deployment.

Stars --Last Commit --

LightX2V

LightX2V provides lightweight image-to-vector models and tooling for efficient visual feature extraction and vector retrieval in resource-constrained environments.

Stars --Last Commit --

olmOCR

A toolkit for linearizing PDFs and image-based documents into readable plain text and Markdown, aimed at LLM dataset creation and large-scale document processing.

Stars --Last Commit --

PaddleOCR

PaddleOCR is a lightweight, high-performance open-source OCR toolkit that supports 100+ languages and converts images or PDFs into structured data.

Stars --Last Commit --

vLLM-Omni

A framework for high-performance, cost-efficient inference and serving of omni-modality models across text, image, video, and audio.

Stars --Last Commit --

Speech-to-text, text-to-speech, and audio understanding.

AutoSubs

Generate accurate, editable subtitles locally or integrated with DaVinci Resolve.

Stars --Last Commit --

CosyVoice

Multilingual, high-quality streaming TTS / speech generation toolkit supporting zero-shot cloning and low-latency generation.

Stars --Last Commit --

Fish Speech

State-of-the-art open source text-to-speech system with voice cloning capabilities, supporting multiple languages with natural-sounding output.

Stars --Last Commit --

GenMedia Creative Studio

GenMedia Creative Studio is a demo web application built on Vertex AI showcasing image, video, audio, and text-to-speech generation capabilities.

Stars --Last Commit --

GPT-SoVITS

GPT-SoVITS is an open-source few-shot voice conversion and TTS WebUI with cross-lingual inference and production-friendly tooling.

Stars --Last Commit --

Handy

A free, open-source, extensible offline speech-to-text desktop application that runs Whisper and Parakeet models locally.

Stars --Last Commit --

MockingBird

An open-source voice cloning and real-time speech generation toolkit that can clone a speaker from a short sample and synthesize arbitrary speech.

Stars --Last Commit --

noScribe

A local-first audio transcription and editing tool for qualitative research and journalism, built on Whisper and Pyannote.

Stars --Last Commit --

OpenAI Whisper

Robust speech recognition via large-scale weak supervision, supporting transcription and translation across 100+ languages with state-of-the-art accuracy.

Stars --Last Commit --

TEN Framework

An open-source framework and ecosystem for real-time, multimodal conversational voice and agent applications.

Stars --Last Commit --

Vibe

A privacy-first, cross-platform audio/video transcription tool that supports fully offline operation and batch processing.

Stars --Last Commit --

VibeVoice

Explore VibeVoice, a cutting-edge TTS framework for long, expressive audio synthesis. Ideal for research, media prototyping, and academic evaluation.

Stars --Last Commit --

Voicebox

Open-source AI voice studio for voice cloning, dictation, and audio creation with a modern web interface.

Stars --Last Commit --

Vosk API

Vosk API provides offline speech recognition for Android, iOS, Raspberry Pi and servers with bindings for Python, Java, C# and Node.

Stars --Last Commit --

whisper.cpp

whisper.cpp is a high-performance local Whisper implementation for speech recognition across edge devices and desktop platforms.

Stars --Last Commit --

Image/video generation models and creative pipelines.

Awesome Image & Video Prompts

An open collection of more than 2,200 copy-ready prompts for the leading AI image and video models (GPT Image, Nano Banana, Seedance, Kling, Veo, Wan, MiniMax), each quoted verbatim from its source and shown next to the real output it produced. It also includes agent recipes for editing video with ffmpeg, with the full set searchable in an online gallery.

Stars --Last Commit --

ComfyUI

A node-based visual workflow builder for Stable Diffusion, enabling graphical assembly and debugging of image-generation pipelines.

Stars --Last Commit --

Deep-Live-Cam

Deep-Live-Cam is an open-source real-time face swap and avatar tool that runs offline for creators and streamers.

Stars --Last Commit --

huggingface diffusers

Diffusers: a modular toolbox for state-of-the-art pretrained diffusion models for image, audio and 3D generation, suitable for inference and training.

Stars --Last Commit --

Hyperframes

Write HTML, render video. Built for AI agents to programmatically create videos from code.

Stars --Last Commit --

LTX-2

Official Python inference and LoRA trainer package for LTX-2, the first DiT-based audio-video foundation model that generates synchronized audio and video in one model, from Lightricks.

Stars --Last Commit --

Open Generative AI

Open-source alternative to AI video platforms with 200+ models including Flux, Midjourney-style generation, and video creation studio.

Stars --Last Commit --

OpenMontage

The first open-source, agentic video production system that turns an AI coding assistant into a full video studio, handling research, scripting, asset generation, editing, and final composition.

Stars --Last Commit --