https://github.com/Huanshere/VideoLingo

Netflix-level subtitle cutting, translation, alignment, and even dubbing - one-click fully automated AI video subtitle team | Netflix级字幕切割、翻译、对齐、甚至加上配音，一键全自动视频搬运AI字幕组

VideoLingo is an all-in-one video translation, localization, and dubbing tool aimed at generating Netflix-quality subtitles. It eliminates stiff machine translations and multi-line subtitles while adding high-quality dubbing, enabling global knowledge sharing across language barriers.

Key features:

• 🎥 YouTube video download via yt-dlp

• 🎙️ Word-level subtitle recognition with WhisperX

• 📝 NLP and GPT-based subtitle segmentation

• 📚 GPT-generated terminology for coherent translation

• 🔄 3-step direct translation, reflection, and adaptation for professional-level quality

• ✅ Netflix-standard single-line subtitles only

• 🗣️ Dubbing alignment with GPT-SoVITS and other methods

• 🚀 One-click startup and output in Streamlit

• 📝 Detailed logging with progress resumption

• 🌐 Comprehensive multi-language support

Difference from similar projects: Single-line subtitles only, superior translation quality

Russian Translation ru_demo.mp4

GPT-SoVITS sovits.mp4

OAITTS OAITTS.mp4

Current input language support and examples:

Input Language	Support Level	Translation Demo
English	🤩	English to Chinese
Russian	😊	Russian to Chinese
French	🤩	French to Japanese
German	🤩	German to Chinese
Italian	🤩	Italian to Chinese
Spanish	🤩	Spanish to Chinese
Japanese	😐	Japanese to Chinese
Chinese*	🤩	Chinese to English

*Chinese requires separate configuration of the whisperX model, only applicable for local source code installation. See the installation documentation for the configuration process, and be sure to specify the transcription language as zh in the webpage sidebar

Translation language support depends on the capabilities of the large language model used, while dubbing language depends on the chosen TTS method.

Commercial version provides free 20min credits, visit videolingo.io

Experience VideoLingo quickly in Colab in just 5 minutes:

VideoLingo supports all hardware platforms and operating systems, but performs best with GPU acceleration. For detailed installation instructions , refer to the documentation: English | 简体中文

VideoLingo provides a Dockerfile. Refer to the installation documentation: English | 简体中文

Usage instructions: English | 简体中文

1. WhisperX performance varies across different devices. Version 1.7 performs demucs voice separation first, but this may result in worse transcription after separation compared to before. This is because whisper itself was trained in environments with background music - before separation it won't transcribe BGM lyrics, but after separation it might transcribe them.

2. The dubbing feature quality may not be perfect as it's still in testing and development stage, with plans to integrate MascGCT. For best results currently, it's recommended to choose TTS with similar speech rates based on the original video's speed and content characteristics. See the demo for effects.

3. Multilingual video transcription recognition will only retain the main language. This is because whisperX uses a specialized model for a single language when forcibly aligning word-level subtitles, and will delete unrecognized languages.

4. Multi-character separate dubbing is under development. While whisperX has VAD potential, specific implementation work is needed, and this feature is not yet supported.

• SaaS service at videolingo.io
• VAD to distinguish speakers, multi-character dubbing
• Customizable translation styles
• Lip sync for dubbed videos

This project is licensed under the Apache 2.0 License.The following open source projects provide important support for the development of VideoLingo:

whisperX | yt-dlp | json_repair | GPT-SoVITS | BELLE

• Join our Discord: https://discord.gg/9F2G92CWPp
• Submit Issues or Pull Requests on GitHub
• Follow me on Twitter: @Huanshere
• Email me at: team@videolingo.io

---

If you find VideoLingo helpful, please give us a ⭐️!