LocalVocal: Local Live Captions & Translation On-the-Go v0.5.3

LocalVocal plugin allows you to transcribe & translate speech into text locally on your machine in real time. ✅ No GPU required*✅ no cloud costs✅ no network and ✅ minimal lag! Privacy first - all data stays on your machine. (* GPU acceleration via CUDA or AMD is supported!) Sponsor my work https://github.com/sponsors/royshil ^_^

If this plugin has been valuable to you consider adding a ⭐ to the GH repo or rating it here on OBS.


Do more with LocalVocal:

The plugin adds an Audio Filter - use it on a speech source (micvideo) to get a transcription. Send the captions to a Text Source to show on scene.

Current Features:

• Transcribe audio to text in real time in 100 languages
• Display captions on screen using text sources
• Send captions to a .txt or .srt file (to read by external sources or video playback) with and without aggregation option
• Sync'ed captions with OBS recording timestamps
• Send captions on a RTMP stream to e.g. YouTubeTwitch
• Bring your own Whisper model (any GGML)
• Translate captions in real time to major languages (both Whisper built-in translation as well as NMT models with CTranslate2)
• CUDAOpenCLApple Arm64AVX & SSE acceleration support

Roadmap:

• More robust built-in translation options
• Additional output options: .vtt.ssa.subetc.
• Speaker diarization (detecting speakers in a multi-person audio stream)

Internally the plugin is running a neural network (OpenAI Whisper) locally to predict in real time the speech and provide captions.

It's using the Whisper.cpp project from ggerganov to run the Whisper network in a very efficient way on CPUs and GPUs. For translation it's using CTranslate2 and the M2M100 model.

Check out my other plugins:

• Background Removal removes background from webcam without a green screen.
• Detect will detect and track >80 types of objects in real-time inside OBS
• URL/API Source that allows fetching live data from an API and displaying it in OBS.