[Question]: Reading an Audio-Stream

[jacob7395]

I can't get any data from the AudioInStreamer, not sure if its a permissions issue for I'm missing something in to code. I also can't find any docs about reading audio just sending it.

Any help would be appreciated.

_client = new DiscordSocketClient(new DiscordSocketConfig()
{
    GatewayIntents = GatewayIntents.AllUnprivileged | GatewayIntents.MessageContent,
    AlwaysDownloadUsers = true,
});

Task ConnectToVoice(SocketVoiceChannel voiceChannel)
{
    if (voiceChannel == null)
        return Task.CompletedTask;

    _ = Task.Run(async () =>
    {
        try
        {
            Console.WriteLine($"Connecting to channel {voiceChannel.Id}");
            _audioClient = await voiceChannel.ConnectAsync();
            _audioClient.StreamCreated += AudioClient_StreamCreated;
            Console.WriteLine($"Connected to channel {voiceChannel.Id}");
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }
    });
    return Task.CompletedTask;
}

Task AudioClient_StreamCreated(ulong userId, AudioInStream stream)
{
    Task.Run(() =>
    {
        while (true)
        {
            stream.TryReadFrame(token, out RTPFrame frame);

            if (frame.Payload is { })
            {
                _waveWriter?.Write(frame.Payload);
            }
        }
        return Task.CompletedTask;
    });

    return Task.CompletedTask;
}

[stropheum]

I've been pulling my hair out since the start of the weekend tryign to solve this exact problem. I managed to get it so I can fire the audio received event once per connection, so if the user leaves the channel and rejoins it will fire once more.

I was told I could cache the AudioInStream in a concurrent dictionary and wait until it has available frames to read, but it never has >0 available frames and trying to call readasync hangs the thread and crashes the client.

At this point I don't know what to do. Trying to just leverage the API directly or maybe using a JS library and running the bot as a process and just reading its output as console text, i don't know if even those will work though. The solution is definitely out there somewhere though, because voice recognition bots exist

[jacob7395]

Same issues I've been seeing, can't see any documentation talking about audio in just stuff around audio out. Maybe someone can point us in the right direction until then, I'm going to try the same and look into using the Discord API.

AudioInStream has a packet count parameter, I put in a check that continued the loop if the count was zero and it never got past it.

[stropheum]

Same. I cached audio streams in a ConcurrentDictionary on voice activity and channel joined events and would poll asynchronously to see if the framecount ever changed, it never did

[jacob7395]

I managed to get something working using DiscordGo, for the most part gpt4 was able to tell me what to do. I had to learn go but it's something I've wanted to do for a while.

There is also an example on the repo for voice recording.

[stropheum]

I got a working example using netcord. not the most useful though because i'm trying to integrate into unity so i really need a native or netstandard version

[stropheum]

Hey I know you closed this but can we correspond a little bit in dms? I'd love to take a look at your example, maybe i could use your example with GoUnity to get something working

[jacob7395]

Happy to chat, dropped you an email with my discord name :)