Asynchronous API support

[czue]

Have you considered adding support for Asychronous APIs for the models that support it? It would make deploying llm as part of a web application easier and more scalable.

Would you accept contributions that tried to add this functionality?

[simonw]

Yes, I absolutely want to do this.

I need it myself for Datasette plugins such as https://github.com/datasette/datasette-extract and https://github.com/datasette/datasette-enrichments-gpt

[simonw]

Dupe of:

• Ability to execute prompts against an asyncio (non-blocking) API #507