Trained model 600 epochs complete flop

[Eyalm321]

Hey guys,

I need your help with some tips for training
I fed the model around hour and half of single speaker voice about 2200 segments 1-5 seconds average
Did I over segment it with pyannote?

Should I fed it more data? Less?
How many epochs is average to get good result? Whats your process?

Seems like after 600 epochs its talking complete giberish

[Eyalm321]

I think I started too big, im going to go over dataset manually and get best samples 3-8 seconds and feed it 20 minutes, see what it does

[Eyalm321]

So after feeding it 20 minutes of selected audio about 300 segments , I got better results but still not a predictable voice tonation or clear voice

[erew123]

Hi @Eyalm321 You are best reading up on the Coqui forums about training, however, 600 epochs is a lot of training. That's approaching the kind of training you would use to train a new language.

If you are just training a voice in a language that the model supports e.g. English, you wont need a huge amount of epochs (the defaults should work). You can over-train a model. Its best to start low and you can always further train a model if needs be. There is no actual perfect recipe for training a model however and ask Coqui note It is hard to estimate the voice quality without asking the actual people.. To paraphrase the advice on the Coqui discussion forums, train the model a bit and see how it sounds, if it doesnt sound correct, train it a little more.

https://docs.coqui.ai/en/latest/faq.html#how-do-i-know-when-to-stop-training

The absolute most important thing is the quality of the training data and dataset. Higher quality audio and ensuring the eval and train csv files match the spoken audio, gives the best results.

[Eyalm321]

Would you have more of a narrator sound? (kinda flat sound) or a tv show for example would be ok (more emotion samples)

[erew123]

Thats really a personal preference. The main things are:

https://docs.coqui.ai/en/latest/what_makes_a_good_dataset.html

Mistake free.

Remove any wrong or broken files. Check annotations, compare transcript and audio length.

Noise free.

Background noise might lead your model to struggle, especially for a good alignment. Even if it learns the alignment, the final result is likely to be suboptimial.

Compatible tone and pitch among voice clips.

For instance, if you are using audiobook recordings for your project, it might have impersonations for different characters in the book. These differences between samples downgrade the model performance.

Good phoneme coverage.

Make sure that your dataset covers a good portion of the phonemes, di-phonemes, and in some languages tri-phonemes.

Naturalness of recordings.

For your model WISIAIL (What it sees is all it learns). Therefore, your dataset should accommodate all the attributes you want to hear from your model.

[Eyalm321]

Thanks alot! ill try and let you know how it works out, appreciate the work!