Async server to handle multiple requests

[criminact]

This Server Handles Concurrency and it capable to serve multiple clients in parallel

[KoljaB]

Thanks a lot for putting in the work. Will test that soon, there are some issues that I see already:

1. speaking

    def play_text_to_speech(self, text):
        self.speaking = True
        try:
            self.stream.feed(text)
            logging.debug(f"Playing audio for text: {text}")
            print(f"Synthesizing: \"{text}\"")
            self.stream.play_async(on_audio_chunk=self.on_audio_chunk, muted=True)
        finally:
            self.speaking = False

self.speaking will instantly be False again, because feed and play_async are both nonblocking.

2. handling of request_handler.speaking

@app.get("/tts")
def tts(request: Request, text: str = Query(...)):
    with tts_lock:
        request_handler = TTSRequestHandler(current_engine)
        browser_request = is_browser_request(request)

        if play_text_to_speech_semaphore.acquire(blocking=False):
            try:
                if not request_handler.speaking:
                    threading.Thread(
                        target=request_handler.play_text_to_speech,
                        args=(text,),
                        daemon=True).start()
            finally:
                play_text_to_speech_semaphore.release()

        return StreamingResponse(
            request_handler.audio_chunk_generator(browser_request),
            media_type="audio/wav"
        )

Even if request_handler.speaking could be True - so in the case we are already speaking, we don't start a play thread and just do nothing here?

3. duplicate routes
   @app.get("/tts-text") route contains calls to play_text_to_speech and audio_chunk_generator and is not doing much more than @app.get("/tts").

Thx again, will test as soon as I find time...

[criminact]

@KoljaB

Here is what I have updated -

Pt1.
I have removed try block and kept self.speaking = False in

def on_audio_stream_stop(self):
        self.audio_queue.put(None)
        self.speaking = False

Pt2.
I have removed request_handler.speaking if else statement, since it should never go in else block anyway?

if play_text_to_speech_semaphore.acquire(blocking=False):
      try:
          threading.Thread(
              target=request_handler.play_text_to_speech,
              args=(text,),
              daemon=True).start()
      finally:
          play_text_to_speech_semaphore.release()

Pt3.
Removed the duplicate code

[KoljaB]

Thanks a lot, now it looks good!

[KoljaB]

Moved the async_server.py file to the example_fast_api folder.

[criminact]

Thanks. I was about to 😅