Tiny model going insane

[fillwithjoy1]

Decided to try and transcribe Panda - Desiigner which was a song that hard lyrics so hard to decode that Desiigner himself had to add into Genius (and led to Google allegedly stealing it and Genius fighting Google case).

except, well its gone insane even when i give it an initial prompt

Python Version: 3.11.7
Package Version: 20231117

I suspect this could be due to the song having a guy in the background part of the music making Grraaah and Yeah yeah yeah sounds???

[Purfview]

Did you tried it with the latest PR included -> #1838?
And try with this bugfix too -> #1903.

[fillwithjoy1]

idek, i just downloaded the latest version from python

[Purfview]

Then you don't have any of that.
You need to install Git: https://git-scm.com/downloads [all default, except in option with PATH select option at the bottom]

To have all latest merged PRs use:

python -m pip install --upgrade --no-deps --force-reinstall git+https://github.com/openai/whisper.git

To have unmerged #1903 PR [not sure if this is correct]:

python -m pip install --upgrade --no-deps --force-reinstall git+https://github.com/openai/whisper.git@refs/pull/1903/head

[Purfview]

Another alternative is to install from the downloaded ziped repo.

python -m pip install --upgrade --force-reinstall "whisper-main.zip" [ downloaded from https://github.com/openai/whisper ]

Or to have #1903 too:
python -m pip install --upgrade --force-reinstall "whisper-patch-1.zip" [ downloaded from https://github.com/Purfview/whisper/tree/patch-1 ]

[fillwithjoy1]

Seems like that fixed it. Thanks!

[Purfview]

Did you installed with #1903 ["whisper-patch-1.zip"] ?

[fillwithjoy1]

Yes I used whisper-patch-1

[Purfview]

Dunno if you used exactly this audio -> https://www.youtube.com/watch?v=4NJlUribp3c

I tested aac from it and on the first try I got almost exactly same hallucination with "come on, come on, come on...".
This very long repetition in one segment [in your second screenshot] is definitely because of bug fixed in #1903.

Then I run 6 times every "r135", "r136 [aka #1838]" and "r135+#1903"

1. r135:             5 results out of 6 had very long repetition per one segment.
2. r136 [aka #1838]: 6 results out of 6 had very long repetition per one segment.
3. r135+#1903:       1 result  out of 6 had very long repetition per one segment.

That "1 result" from the "r135+#1903" tests was a bit different:

[00:17.780 --> 00:26.680]  GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGHGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG firmware

Note:
Results from "r136 [aka #1838]" looked better than from "r135" because I guess it was fixing other hallucinations.

[fillwithjoy1]

It's been a mixed bag. The ones that had problems were fixed, but the ones that were fine started hallucinating. It seems to be that static in something like Mandela Catalogue doesn't signal to the model to stop transcribing, and so it keeps repeating the last output. At least that's what I'm getting Sent with Shortwave <https://www.shortwave.com?utm_medium=email&utm_content=signature&utm_source=am95c2luZ2guanMxQGdtYWlsLmNvbQ==>

…

On Fri Jan 19, 2024, 03:00 PM GMT, Purfview ***@***.***> wrote: Dunno if you used this exactly audio -> https://www.youtube.com/watch?v=4NJlUribp3c <https://www.youtube.com/watch?v=4NJlUribp3c> I tested aac from it and on the first try I got almost exactly same hallucination with "come on, come on, come on...". This very long repetition in one segment [in your second screenshot] is definitely because of bug fixed in #1903 <#1903>. Then I run 6 times every "r135", "r136 [aka #1838 <#1838>]" and "r135+#1903 <#1903>" 1. r135: 5 results out of 6 had very long repetition per one segment. 2. r136 [aka #1838]: 6 results out of 6 had very long repetition per one segment. 3. r135+#1903: 1 result out of 6 had very long repetition per one segment. That "1 result" from the "r135+#1903 <#1903>" tests was bit different: [00:17.780 --> 00:26.680] GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGHGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG firmware Note: Results from "r136 [aka #1838 <#1838>]" looked better than from "r135" because it was fixing other hallucinations. — Reply to this email directly, view it on GitHub <#1962 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHBFVTXUCJKBDSNVIBZLGULYPKDAPAVCNFSM6AAAAABB6NWST6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DCOBSGU3TC>. You are receiving this because you authored the thread.Message ID: ***@***.***>