Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger #20

Merged
merged 8 commits into from
Dec 27, 2016
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions docs/API.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
## API Methods

Paramaters marked in **bold** are required

### Require sonus and a cloud speech recognizer in your project:
``` javascript
const Sonus = require('sonus')
const speech = require('@google-cloud/speech')({
projectId: 'streaming-speech-sample',
keyFilename: './keyfile.json'
})
```
For more information about Google Cloud Speech see: https://cloud.google.com/speech/
Note: don't forget to enable billing!

### Custom hotwords
You can train and download custom hotwords for sonus from https://snowboy.kitt.ai
In order to initialize Sonus you need to pass in 1 or more hotwords.
Each hotword supports the following proporties:
**`file`** - The path to your hotword model (either pmdl or umdl)
**`hotword`** - The string that represents your hotword (ex: "sonus")
`sensitivity` - (default `'0.5'`) If you are getting a lot of false positives or are having trouble detecting your hotword adjusting this value shoud help

**Example:** (to be passed into the sonus constructor)
``` javascript
const hotwords = [
{file: '/mymodel.pmdl', hotword: 'sonus'},
{file: 'snowboy.umdl', hotword: 'snowboy'}]
```

### Lenguages
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Languages

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks :)

Sonus lets you customize the lenguage for streaming speech recognition. For details on supported lenguages see the docs for your streaming speech recognizer

**Example:** (to be passed into the sonus constructor)
``` javascript
const lenguage = "en-US"
```

### Initialize Sonus
Sonus's initialization accepts two paramaters:
**`options`** - an options object that contains your hotwords, lenguage, etc
- **`hotwords`** - an array of recognizable hotwords
- `lenguage` - streaming lenguage recognition
- `dictionary` - [TODO] only supported by some streaming recognizers
**`speechRecognizer`** - the speech recognizer of your choice

**Example:**
``` javascript
const sonus = Sonus.init({ hotwords, language }, speech)
```

### Start recognition
Pass your initialized sonus object into `Sonus.start`
**Example:**
``` javascript
Sonus.start(sonus)
```

### Pause recognition
Pass your initialized sonus object into `Sonus.pause`.
Pausing recognition while streaming will not cancel the request, instead it will cause it to simulate the "end" of speech and return final results.
**Example:**
``` javascript
Sonus.pause(sonus)
```

### Resume recognition
Pass your initialized sonus object into `Sonus.resume`
**Example:**
``` javascript
Sonus.resume(sonus)
```

### Stop recognition
If you want to stop recognition enterly you can use `Sonus.stop`
**Example:**
``` javascript
Sonus.stop(sonus)
```
Note that after recognition is stopped it can not be started again without creating an enterly new sonus instance.

### Trigger keyword/hotword manually
You can manuall trigger a hotword by passing your initialized sonus object into `Sonus.trigger`
This will throw a `NOT_STARTED` exception if you have not started sonus when this is called.

**Example:**
``` javascript
Sonus.trigger(sonus)
```
sonus will be triggered with a hotword index of `0` and a hotword of `"triggered"`

While it's not officially supported, If you want to trigger a specific index/hotword you can call `trigger` directly on the initialized sonus object
**Example:**
``` javascript
sonus.trigger(0, 'hotword')
```

## Events
hotword
partial-result
final-result
error
36 changes: 36 additions & 0 deletions examples/trigger-example.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
'use strict'

const ROOT_DIR = __dirname + '/../'
const Sonus = require(ROOT_DIR + 'index.js')
const speech = require('@google-cloud/speech')({
projectId: 'streaming-speech-sample',
keyFilename: ROOT_DIR + 'keyfile.json'
})

const hotwords = [{ file: ROOT_DIR + 'resources/sonus.pmdl', hotword: 'sonus' }]
const language = "en-US"
const sonus = Sonus.init({ hotwords, language }, speech)

try{
Sonus.trigger(sonus)
} catch (e) {
console.log('Triggering Sonus before starting it will throw the following exception:', e)
}

Sonus.start(sonus)

sonus.on('hotword', (index, keyword) => console.log("!" + keyword))

sonus.on('partial-result', result => console.log("Partial", result))

sonus.on('error', (error) => console.log(error))

sonus.on('final-result', result => {
console.log("Final", result)
if (result.includes("stop")) {
Sonus.stop()
}
})

//Will use index 0 with a hotword of "triggered" and start streaming immedietly
Sonus.trigger(sonus)
21 changes: 19 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ const record = require('node-record-lpcm16')
const stream = require('stream')
const {Detector, Models} = require('snowboy')

const ERROR = {
NOT_STARTED : "NOT_STARTED"
}

const CloudSpeechRecognizer = {}
CloudSpeechRecognizer.init = recognizer => {
const csr = new stream.Writable()
Expand Down Expand Up @@ -56,6 +60,7 @@ Sonus.init = (options, recognizer) => {
sonus = new stream.Writable(),
csr = CloudSpeechRecognizer.init(recognizer)
sonus.mic = {}
sonus.started = false

// If we don't have any hotwords passed in, add the default global model
opts.hotwords = opts.hotwords || [1]
Expand All @@ -80,8 +85,7 @@ Sonus.init = (options, recognizer) => {

// When a hotword is detected pipe the audio stream to speech detection
detector.on('hotword', (index, hotword) => {
sonus.emit('hotword', index, hotword)
CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr)
sonus.trigger(index, hotword)
})

csr.on('error', error => sonus.emit('error', { streamingError: error }))
Expand All @@ -97,6 +101,16 @@ Sonus.init = (options, recognizer) => {
}
}
})

sonus.trigger = (index, hotword) => {
if(sonus.started){
sonus.emit('hotword', index || "0", hotword || "triggered")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you actually want string 0?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should fallback to zero, but I think in order to best support all scenarios users should be able to pass in an index and a hotword to the trigger function (to accurately simulate the triggering of a specific hotword in case they have hotwords that perform different actions).

Proposed API

Sonus.trigger(sonus)
// Triggers sonus instance with index 0 and hotword "triggered"

Sonus.trigger(sonus, 1, "sonus")
//Triggers sonus instance with index 0 and hotword "sonus"

High flexibility and easy to use, or is this crazy 🍌s?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You used a string

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - it's actually supposed to be a string... My proposal was wrong.

CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr)
} else {
throw ERROR.NOT_STARTED
}
}

return sonus
}

Expand All @@ -107,8 +121,11 @@ Sonus.start = sonus => {
})

sonus.mic.pipe(sonus.detector)
sonus.started = true
}

Sonus.trigger = sonus => sonus.trigger()

Sonus.pause = sonus => sonus.mic.pause()

Sonus.resume = sonus => sonus.mic.resume()
Expand Down