Does this project currently have automatic timing of video's transcript?

[FurkanGozukara]

Hello. I have transcription of the speech video. However it doesn't have time.

So can this project process the audio and the transcription and generate automatically timed subtitles at the moment?

Or else how can I do that? Something like YouTube does.

Thank you.

[niksedk]

Hi @FurkanGozukara,

Sorry, SE does not currently have this feature.
I guess it could be possible via speech recognition (SE currently supports vosk/kaldi).

Saw you have tried Whisper - does that work well? And do you know if it can it be compiled to an executable file?

[FurkanGozukara]

It can't be compiled into an exe file at the moment. It works fantastic though if you ask me for transcribing. I am already using it.

I even prepared 2 videos regarding Whisper

How to do Free Speech-to-Text Transcription Better Than Google Premium API with OpenAI Whisper Model : https://www.youtube.com/watch?v=msj3wuYf3d8

This above video has raw transcriptions generated by Whisper

How Good is RTX 3060 for ML AI Deep Learning Tasks and Comparison With GTX 1050 Ti and i7 10700F CPU : https://www.youtube.com/watch?v=q8Q8CCDdSKo

Although sometimes its timing may not be perfect. Therefore I am kind of looking for perfect timing.

[niksedk]

Thx for the videos - good work :)

Whisper seems to work okay (I needed medium model to work with Danish which is rather slow - but it does work okay).
Impressive with all the languages!

I've tried to add support for Whisper in latest SE beta: https://github.com/SubtitleEdit/subtitleedit/releases/download/3.6.7/SubtitleEditBeta.zip
Should work on whole files + selected lines via context menu.

The timing seems to a bit off compared to Vosk/Kaldi. Do anyone know if it's possible to get one word at a time?

[ard-av]

[ard-av]

[ard-av]

openai/whisper#130
C/C++ implementation of the model #130

[ard-av]

Спасибо за видео - хорошая работа :)

Whisper, кажется, работает нормально (мне нужна была средняя модель для работы с датским, который довольно медленный, но работает нормально). Впечатляет со всеми языками!

Я пытался добавить поддержку Whisper в последнюю бета-версию SE: https://github.com/SubtitleEdit/subtitleedit/releases/download/3.6.7/SubtitleEditBeta.zip Должен работать со всеми файлами + выбранными строками через контекстное меню.

Время кажется немного неправильным по сравнению с Vosk/Kaldi. Кто-нибудь знает, можно ли получить по одному слову за раз?

openai/whisper#280
https://github.com/jianfch/stable-ts ???

[F-Emerick]

Thank you!
I did it on windows and it works!
It had transcribed almost perfectly in portuguese.
Important:
I had to download git (https://scoop.sh/) and use powershell.
I had to use Python 3.10.8

I guess it's not for everybody - but try this:

1. Download Python from: https://www.python.org/downloads/
2. Install pip: See https://phoenixnap.com/kb/install-pip-windows
3. Install whisper via cmd line, run: pip install git+https://github.com/openai/whisper.git

Models can be downloaded via the SE UI or will be auto downloaded when you run whisper.... I think.

[niksedk]

Try: whisper --model tiny.en --language English --fp16 False a.wav

[ard-av]

I'm doing this not in SE yet, but in the Python terminal, as in the video from the first post. In SE I had the same error as you

[niksedk]

@MbuguaDavid: You can press F2 in the SE Whisper window to toogle log-view... --fp16 False should already be present (from a hidden WhisperExtraSettings in Settings.xml).
Latest beta might work better: https://github.com/SubtitleEdit/subtitleedit/releases/download/3.6.7/SubtitleEditBeta.zip

[MbuguaDavid]

@MbuguaDavid: You can press F2 in the SE Whisper window to toogle log-view... --fp16 False should already be present (from a hidden WhisperExtraSettings in Settings.xml). Latest beta might work better: https://github.com/SubtitleEdit/subtitleedit/releases/download/3.6.7/SubtitleEditBeta.zip

It still doesn't work. I wish it did. The "No Text Found" Error still persists. :(

Whisper.-.No.Text.Found.Error.mp4

[shin3c]

@MbuguaDavid Try put ffmpeg.exe to "C:\Windows\system32" folder, or any folder that defined in "Environment Variables>PATH" It seems the problem was caused by Whisper couldn't find ffmpeg in your environment.

[coastal45]

That made no difference. But I added additional information on this problem in #6328 (comment)

[coastal45]

@niksedk Is it possible to find out the actual command that SE sends to whisper, and what result causes "No text found!" dialog?

[niksedk]

Pressing F2 might give the command line... or try to check the file "error_log.txt"
Also, latest beta of SE automatically uses ffmpeg if SE knows where it is (one less problem)

[coastal45]

@niksedk I did another test.
When running the sample file in SE (release 3.6.8), the F2 screen only shows the command sent to whisper, then nothing else until the window disappears and then "No text found!" dialog.

--EDIT--

Then I entered the same whisper command at Windows Command Prompt. The result is attached. They don't display correctly in Command Prompt being non-ASCII, but copying to a text file displays Japanese characters.
Also, vtt/srt files were created, so it seems the basic whisper operation is ok.

Related files are attached that you can check with. Again, this is only happening in Windows, Linux works correctly.
On Windows 10, Python 3.10.8.

files.zip

files_srt.zip