SIPREC and return audio as response

[telmojsneves]

As I see I can use this to receive audio and process it.
Can I also use this to return a audio Stream?

For better understanding I have the following problem:
I want to have a "bot" that receives calls, processes the audio and return a new audio back using RTP streams.
This new audio generated is based on the audio received(Question / Answer Style)

[davehorton]

No, this would not be a repo that you would use for that. For one thing, SIPREC is pretty much a one-way protocol -- sending audio from a client to a SIPREC server, usually for the purpose of recording or doing speech analytics.

A different project that you might be interested in is this one.

It has the ability send audio over a websocket connection and receive audio back that can be played to the caller.

[telmojsneves]

So in that case we only need to use drachtio server, srf and drachtio-fsmrf(with mod_audio_fork) to receive and return audio? How can I test this? I am using Zoiper5. Also should be better to create a sip client for receiving the audio, process it and returning it?

Thank you

[davehorton]

I'm not completely sure what you want to do, but if you want to terminate an audio stream you also need Freeswitch, or another media server.

If you are wanting to use docker containers for this, and it sounds like you want to run it on your laptop, I would recommend installing docker-compose and then using a docker-compose.yaml file like this one.

Start up the docker containers by running

docker-compose -f docker-compose.yaml up -d

That file starts a drachtio container and a freeswitch container, using host networking. Then on your laptop you can run your node.js drachtio app as usual:

node app.js

and it should connect to both drachtio and freeswitch, since those are running in docker but using the host network. drachtio server should then be listening on 127.0.0.1:9022 and freeswitch should be listening on 127.0.0.1:8021.

You can then try sending a call in from your zoiper softphone and see if it hits the drachtio server, and triggers your app.

[telmojsneves]

I was already using a similar docker-compose file, and also freeswitch as a media server. Now what I am trying to do is receiving a wav from an external service and storing it. The file is being stored in the same place where drachtio-srf runs. So when I am running play nothing happens.

function run(ms) {
    srf.invite((req, res) => {
      ms.connectCaller(req, res)
        .then(({endpoint, dialog}) => {
          dialog.on('destroy', () => endpoint.destroy());
          var file = fs.createWriteStream("file.wav");
          await http.get("http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02.wav", function(response) {
            // the wav file is just for test purposes, in the future a service should be invoked to get the audio back
            response.pipe(file);
          });
          await ep.play('silence_stream://5000');
          await ep.play("file.wav");
        })
        .catch((err) => {
          console.log(err, 'Error connecting call to freeswitch');
        });
    });
  }

Should I share the volume folder containing the audio with the container? Or this is just the wrong way to do it?

[davehorton]

You reference ep as in ep.play but I never see it defined.

Perhaps you meant endpoint.play( ?

also, you will need to map whatever volume you write it to into the freeswitch container, yes.

And finally, freeswitch can do the http get for that file if you want, so theoretically you should be able to do

endpoint.play('http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02.wav')

[telmojsneves]

I am trying to play the url directly and no sound returns. Also the silence is being played, at least by the freeswitch logs. This is my code for the app.js file.

const Srf = require('drachtio-srf');
const srf = new Srf();
const Mrf = require('drachtio-fsmrf');
const mrf = new Mrf(srf);
const config = require('config');

const http = require('http');
const fs = require('fs');
const util = require('util');


srf.connect(config.get('drachtio'))
  .on('connect', (err, hp) => console.log(`connected to sip on ${hp}`))
  .on('error', (err) => console.log(err, 'Error connecting'));



mrf.connect({ address:'127.0.0.1',
  port: 9070,
  secret: 'ClueCon'})
  .then((ms) => run(ms));

srf.register((req, res) => {
  res.send(200);
});

function run(ms) {
    srf.invite((req, res) => {
      ms.connectCaller(req, res)
        .then(({endpoint, dialog}) => {
          dialog.on('destroy', () => endpoint.destroy());
          textToSpeech(endpoint,dialog);
        })
        .catch((err) => {
          console.log(err, 'Error connecting call to freeswitch');
        });
    });
  }

async function textToSpeech(endpoint, dialog) {
  var file = fs.createWriteStream("file.wav");
          
  await http.get("http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02.wav", function(response) {
    response.pipe(file);
  });
  await endpoint.play('silence_stream://5000');
  await endpoint.play("http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02.wav");
  dialog.destroy();
  endpoint.destroy();

}

What I am trying to do is a bot that receives a call (an RTP stream) process the audio, and return a new version of the audio using RTP also.

Also the register part is just for Zoiper be able to authenticate.

Thank you, and I am sorry, but I am still learning about drachtio and SIP protocol.

[davehorton]

can you post a link to a gist with the freeswitch log output (the entire log from start until your call is answered and you attempt to play the file)

[telmojsneves]

Here is the link for the logs during the call.

[davehorton]

I haven't tested on docker, but on a native server running drachtio server and freeswitch (configured to work with drachtio server), this app answers the call and plays your file

const Srf = require('drachtio-srf');
const srf = new Srf();
const Mrf = require('drachtio-fsmrf');
const mrf = new Mrf(srf);
const config = require('config');

srf.connect(config.get('drachtio'));
srf
  .on('connect', (err, hp) => {
    if (err) throw err;
    console.log(`connected to drachtio listening on ${hp}`);
  })
  .on('error', (err) => console.error(err));

srf.invite(async(req, res) => {
  try {
    const ms = await mrf.connect(config.get('freeswitch'));
    const {endpoint, dialog} = await ms.connectCaller(req, res);
    console.log('call connected');
    dialog.on('destroy', () => {
      console.log('call ended');
      endpoint.destroy();
    });
    await endpoint.play('http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02.wav');
    console.log('play done');
  } catch (err) {
    console.log(err);
  }
});