Not getting the transcript

[nicolasguasca1]

Hello! I was starting to interact with this hook and I haven't been able to get the transcript yet. Im not sure how to enable the microphone to start listening. Any help will be much appreciated!

Here is what I have:

let captionRecord = true;
const [transcribing, setTranscribing] = useState(false)
const [clearTranscriptOnListen, setClearTranscriptOnListen] = useState(true)
const toggleTranscribing = () => setTranscribing(!transcribing)
const toggleClearTranscriptOnListen = () => setClearTranscriptOnListen(!clearTranscriptOnListen)
const support = useState<boolean>(SpeechRecognition.browserSupportsSpeechRecognition);
const {
    transcript,
    interimTranscript,
    finalTranscript,
    resetTranscript,
    listening,
  } = useSpeechRecognition({ transcribing, clearTranscriptOnListen })

const captioning = () => {
  if(!support){
    console.log("speech recognition not supported");
    captionRecord = false;
  } else{
    console.log("speech recognition supported");
  }
  console.log("captionRecord", captionRecord);
  if (!captionRecord) {
    console.log("captions not supported in this browser!");
  } else {
    console.log("Setting caption recognitions");
    // SpeechRecognition.startListening()
  }
}

useEffect(() => {
  if (interimTranscript !== '') {
    console.log('Got interim result:', interimTranscript)
  }
  if (finalTranscript !== '') {
    console.log('Got final result:', finalTranscript)
  }
}, [interimTranscript, finalTranscript]);

const startCaptioning = () => {
  if(!captionRecord){
    return
  }
  if(transcribing) {
    return;
  }
  setTranscribing(true);
  SpeechRecognition.startListening();
  captioning();
  console.log(interimTranscript);
}

const stopCaptioning = () => {
  if(!transcribing){
    return;
  }
  if(captionRecord){
    setTranscribing(false);
    SpeechRecognition.stopListening();
    console.log({transcript});
 }
}

and the the rendering component:

<div>
        {transcribing ? (
            <Button className={tw(`border-red-600 border-opacity-100 rounded px-4 py-2`)} onClick={stopCaptioning}>Stop Captioning</Button>
          ) : (
            <Button className={tw(`border bg-yellow-700 rounded px-4 text-white py-2`)} onClick={startCaptioning} >Start Captioning</Button>
          )}
      </div>
      <div>
       <span>{transcript}</span>

What am I missing?

[JamesBrill]

Hi @nicolasguasca1 I reproduced your code and it produced a transcript in Chrome. So your code is working as expecting. Note that, for the most part, only Chrome supports the Web Speech API, which is where browserSupportsSpeechRecognition comes in. Speaking of which, when browserSupportsSpeechRecognition is used via SpeechRecognition, it is a function rather than a boolean. So you will need to change your line from:

const support = useState<boolean>(SpeechRecognition.browserSupportsSpeechRecognition);

to

const support = useState<boolean>(SpeechRecognition.browserSupportsSpeechRecognition());

to detect browser support correctly. Alternatively, you can use the browserSupportsSpeechRecognition state from useSpeechRecognition instead - this will eventually replace SpeechRecognition.browserSupportsSpeechRecognition() in future. It's also possible this is producing a compile error in TypeScript as there is a mismatch between the function type and the boolean type you are passing as a type argument - applying the above change should fix that.

Some questions:

• Which browser and device are you using?
• Does your web app produce a transcript on Chrome?
• If not, does it produce a transcript if reducing it to the example from the README?

[nicolasguasca1]

Thank you so much for this James! After digging a bit into the code I made it work by adding setTranscribing to true. You can see the final piece below. I tried what you suggested with browserSupportsSpeechRecognition() and it now works. My question now is that as you can see I also put some code to notify if the microphone is listening. The console log throws false saying the mic is not listening but it is indeed doing it and transcribing it. I tried adding setListening to true before calling the SpeechRecognition.startListening method but it didn't work. I think that maybe it has to do with the fact that the entire speechrecognition method comes from a promise so I should add an await to that listening confirmation, am I right? Anyways, thank you for your feedback, this package is awesome!

let captionRecord = true;
const [transcribing, setTranscribing] = useState(false)
const [listening, setListening] = useState(false)
const [clearTranscriptOnListen, setClearTranscriptOnListen] = useState(true)
const support = useState<boolean>(SpeechRecognition.browserSupportsSpeechRecognition());
const {
    transcript,
    interimTranscript,
    finalTranscript,
    resetTranscript,
  } = useSpeechRecognition({ transcribing, clearTranscriptOnListen })
const captioning = () => {
  if(!support){
    console.log("Speech recognition not supported");
    captionRecord = false;
    return <span>Speech recognition is not supported in this browser</span>
  } else{
    console.log("Speech recognition supported");
  }
  console.log("captionRecord", captionRecord);
  if (!listening) {
    console.log("Microphone not listening!");
  } else {
    console.log("Active listening!");
  }
}
useEffect(() => {
  if (interimTranscript !== '') {
    console.log('Got interim result:', interimTranscript)
  }
  if (finalTranscript !== '') {
    console.log('Got final result:', finalTranscript)
  }
}, [interimTranscript, finalTranscript]);
const startCaptioning = () => {
  try {
    if(!captionRecord){
    return
    }
    if(transcribing) {
      return;
    }
    resetTranscript();
    setTranscribing(true);
    SpeechRecognition.startListening({ continuous: true });
    captioning();
  } catch (error) {
    alert('Captioning error')
  }
}
const stopCaptioning = () => {
  if(!transcribing){
    return;
  }
  if(captionRecord){
    resetTranscript();
    setTranscribing(false);
    SpeechRecognition.stopListening();
    console.log({transcript});
 }
}

Finally answering your questions:
Browser: Chrome Version 93.0.4577.15
OS: Mac OS Big Sur
It does the transcript.

[JamesBrill]

Hi @nicolasguasca1 Glad that helped! I notice in your new code, you have added your own listening state:

const [listening, setListening] = useState(false)

You do not need to manage listening state yourself - react-speech-recognition provides this for you. You just need to replace your state with one from useSpeechRecognition. i.e.

const {
    listening,
    transcript,
    interimTranscript,
    finalTranscript,
    resetTranscript,
} = useSpeechRecognition({ transcribing, clearTranscriptOnListen })

[jkshitij77]

I tried copying the same code as above (both the iterations of nicolas's code) but I still get an empty transcript. I am working with React and Electron trying to make an app that will run on MacOS. Any suggestions on what else to try?