When should I specify the language by --language?

[KrahmalK]

I know this option is helpful to transcribe an audio file containing non-English speech, but I' m not sure about "contain". Does it mainly refers to English speech with some paragraphs of other language?

In these cases, will it be helpful if I set the language option?

1. An English audio contains scattered Spanish words in some sentences.
2. An English audio contains some Spanish paragraphs.
3. A multilingual audio and each language has one sentence.

I tried the default option and the specific option and the outputs are similar. No matter whether I set --language Spanish or not, it transcribes the Spanish parts clearly. Is it because my audio is not challenging enough?

In #49 there are discussions about how to keep the specific language.

[jongwook]

It's intended for monolingual audio inputs, and --language should specify the language used in the audio (English or otherwise). Whisper doesn't support code-switching inputs very well, and some third-party tools could be used to segment the audio into sentences and separately run Whisper to transcribe each.

[DiogoBerna]

Would you mind quote some of these third-party tools that could be used to segment the audio as you said? I was not able to find any :(

[glangford]

@DiogoVidevince See for example

https://github.com/facebookresearch/svoice

We provide a PyTorch implementation of our speaker voice separation research work. In Voice Separation with an Unknown Number of Multiple Speakers, we present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed.

and the associated paper

https://arxiv.org/abs/2003.01531