Preparing scientific audio text for whisper fine-tuning

[kojomensahonums]

I am currently working on training whisper for scientific and mathematical audio datasets. In preparing the ground truth text data, what is the best way to go about preparing it? How should equations be written down, should they be plainly put in text without symbols? How should SI units be written? Assuming there is a quadratic equation, example x^2-y^2=25, what's the best way to put this in text so whisper can follow through to transcribe? These are just few examples I am thinking through.

[Purfview]

should they be plainly put in text without symbols?

Yes, avoid any symbols.

[gongouveia]

@kojomensahonums Hello, I believe my tool (I am still adding features, and solving bugs) for creating synthetic audio datasets must be useful for your project, you can translate and edit audios to match your desired format.

[kojomensahonums]

What tool is that? Would like to have a look

[gongouveia]

Sorry, I forgot to add the link.
https://github.com/gongouveia/Whisper-Temple-Synthetic-ASR-Dataset-Generator I am glad you could look it and give some feedback if it is useful for your project.
Next feature I am adding is export the audiodataset to Huggingface or kaldi format.

[kojomensahonums]

Hi @gongouveia, I have tried using your tool but facing some challenges. The instructions are quite different from the files present. For example, the requirement.txt file mentioned is a req.yml file and I can't seem to run it. I don't know what it is with the yaml file. I changed the prefix to my virtual environment but I keep getting name errors. Also, the speech_gen.py file is not present. The instructions don't match with the files. Do help me out here.

[gongouveia]

@kojomensahonums sorry for the reply, I will get to you as soon as possible with some fix.

[kojomensahonums]

@gongouveia any progress so far?

[gongouveia]

@kojomensahonums Hello, I have been very busy. Yes the ReadMe is deprecated.
You can send me private message in contacts in my profile and I can work it our for you.

[kojomensahonums]

That would be great! How can I reach you?

[gongouveia]

@kojomensahonums I updated readme. the dependencies are in the req.yml file. and you can install it using conda for example.
Any other question you can reach on email. See end of repository for me contact.

Also updated the software images, because it has more things that previosuly stated

[kojomensahonums]

@gongouveia I have just tried your tool and it's awesome!

[gongouveia]

@kojomensahonums I am very glad you enjoyed the tool, please don't forget to support the project giving a star and contributing.
If you would like new features, open an issue with detailed description.