-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trigger #20
Trigger #20
Changes from 3 commits
c47cc19
de09a7b
1791bfc
ddcb5b6
8dcd68b
8cb562d
a5d9560
3535299
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
## API Methods | ||
|
||
Paramaters marked in **bold** are required | ||
|
||
### Require sonus and a cloud speech recognizer in your project: | ||
``` javascript | ||
const Sonus = require('sonus') | ||
const speech = require('@google-cloud/speech')({ | ||
projectId: 'streaming-speech-sample', | ||
keyFilename: './keyfile.json' | ||
}) | ||
``` | ||
For more information about Google Cloud Speech see: https://cloud.google.com/speech/ | ||
Note: don't forget to enable billing! | ||
|
||
### Custom hotwords | ||
You can train and download custom hotwords for sonus from https://snowboy.kitt.ai | ||
In order to initialize Sonus you need to pass in 1 or more hotwords. | ||
Each hotword supports the following proporties: | ||
**`file`** - The path to your hotword model (either pmdl or umdl) | ||
**`hotword`** - The string that represents your hotword (ex: "sonus") | ||
`sensitivity` - (default `'0.5'`) If you are getting a lot of false positives or are having trouble detecting your hotword adjusting this value shoud help | ||
|
||
**Example:** (to be passed into the sonus constructor) | ||
``` javascript | ||
const hotwords = [ | ||
{file: '/mymodel.pmdl', hotword: 'sonus'}, | ||
{file: 'snowboy.umdl', hotword: 'snowboy'}] | ||
``` | ||
|
||
### Lenguages | ||
Sonus lets you customize the lenguage for streaming speech recognition. For details on supported lenguages see the docs for your streaming speech recognizer | ||
|
||
**Example:** (to be passed into the sonus constructor) | ||
``` javascript | ||
const lenguage = "en-US" | ||
``` | ||
|
||
### Initialize Sonus | ||
Sonus's initialization accepts two paramaters: | ||
**`options`** - an options object that contains your hotwords, lenguage, etc | ||
- **`hotwords`** - an array of recognizable hotwords | ||
- `lenguage` - streaming lenguage recognition | ||
- `dictionary` - [TODO] only supported by some streaming recognizers | ||
**`speechRecognizer`** - the speech recognizer of your choice | ||
|
||
**Example:** | ||
``` javascript | ||
const sonus = Sonus.init({ hotwords, language }, speech) | ||
``` | ||
|
||
### Start recognition | ||
Pass your initialized sonus object into `Sonus.start` | ||
**Example:** | ||
``` javascript | ||
Sonus.start(sonus) | ||
``` | ||
|
||
### Pause recognition | ||
Pass your initialized sonus object into `Sonus.pause`. | ||
Pausing recognition while streaming will not cancel the request, instead it will cause it to simulate the "end" of speech and return final results. | ||
**Example:** | ||
``` javascript | ||
Sonus.pause(sonus) | ||
``` | ||
|
||
### Resume recognition | ||
Pass your initialized sonus object into `Sonus.resume` | ||
**Example:** | ||
``` javascript | ||
Sonus.resume(sonus) | ||
``` | ||
|
||
### Stop recognition | ||
If you want to stop recognition enterly you can use `Sonus.stop` | ||
**Example:** | ||
``` javascript | ||
Sonus.stop(sonus) | ||
``` | ||
Note that after recognition is stopped it can not be started again without creating an enterly new sonus instance. | ||
|
||
### Trigger keyword/hotword manually | ||
You can manuall trigger a hotword by passing your initialized sonus object into `Sonus.trigger` | ||
This will throw a `NOT_STARTED` exception if you have not started sonus when this is called. | ||
|
||
**Example:** | ||
``` javascript | ||
Sonus.trigger(sonus) | ||
``` | ||
sonus will be triggered with a hotword index of `0` and a hotword of `"triggered"` | ||
|
||
While it's not officially supported, If you want to trigger a specific index/hotword you can call `trigger` directly on the initialized sonus object | ||
**Example:** | ||
``` javascript | ||
sonus.trigger(0, 'hotword') | ||
``` | ||
|
||
## Events | ||
hotword | ||
partial-result | ||
final-result | ||
error |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
'use strict' | ||
|
||
const ROOT_DIR = __dirname + '/../' | ||
const Sonus = require(ROOT_DIR + 'index.js') | ||
const speech = require('@google-cloud/speech')({ | ||
projectId: 'streaming-speech-sample', | ||
keyFilename: ROOT_DIR + 'keyfile.json' | ||
}) | ||
|
||
const hotwords = [{ file: ROOT_DIR + 'resources/sonus.pmdl', hotword: 'sonus' }] | ||
const language = "en-US" | ||
const sonus = Sonus.init({ hotwords, language }, speech) | ||
|
||
try{ | ||
Sonus.trigger(sonus) | ||
} catch (e) { | ||
console.log('Triggering Sonus before starting it will throw the following exception:', e) | ||
} | ||
|
||
Sonus.start(sonus) | ||
|
||
sonus.on('hotword', (index, keyword) => console.log("!" + keyword)) | ||
|
||
sonus.on('partial-result', result => console.log("Partial", result)) | ||
|
||
sonus.on('error', (error) => console.log(error)) | ||
|
||
sonus.on('final-result', result => { | ||
console.log("Final", result) | ||
if (result.includes("stop")) { | ||
Sonus.stop() | ||
} | ||
}) | ||
|
||
//Will use index 0 with a hotword of "triggered" and start streaming immedietly | ||
Sonus.trigger(sonus) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,6 +4,10 @@ const record = require('node-record-lpcm16') | |
const stream = require('stream') | ||
const {Detector, Models} = require('snowboy') | ||
|
||
const ERROR = { | ||
NOT_STARTED : "NOT_STARTED" | ||
} | ||
|
||
const CloudSpeechRecognizer = {} | ||
CloudSpeechRecognizer.init = recognizer => { | ||
const csr = new stream.Writable() | ||
|
@@ -56,6 +60,7 @@ Sonus.init = (options, recognizer) => { | |
sonus = new stream.Writable(), | ||
csr = CloudSpeechRecognizer.init(recognizer) | ||
sonus.mic = {} | ||
sonus.started = false | ||
|
||
// If we don't have any hotwords passed in, add the default global model | ||
opts.hotwords = opts.hotwords || [1] | ||
|
@@ -80,8 +85,7 @@ Sonus.init = (options, recognizer) => { | |
|
||
// When a hotword is detected pipe the audio stream to speech detection | ||
detector.on('hotword', (index, hotword) => { | ||
sonus.emit('hotword', index, hotword) | ||
CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr) | ||
sonus.trigger(index, hotword) | ||
}) | ||
|
||
csr.on('error', error => sonus.emit('error', { streamingError: error })) | ||
|
@@ -97,6 +101,16 @@ Sonus.init = (options, recognizer) => { | |
} | ||
} | ||
}) | ||
|
||
sonus.trigger = (index, hotword) => { | ||
if(sonus.started){ | ||
sonus.emit('hotword', index || "0", hotword || "triggered") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Did you actually want string 0? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It should fallback to zero, but I think in order to best support all scenarios users should be able to pass in an index and a hotword to the trigger function (to accurately simulate the triggering of a specific hotword in case they have hotwords that perform different actions). Proposed API Sonus.trigger(sonus)
// Triggers sonus instance with index 0 and hotword "triggered"
Sonus.trigger(sonus, 1, "sonus")
//Triggers sonus instance with index 0 and hotword "sonus" High flexibility and easy to use, or is this crazy 🍌s? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ping! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You used a string There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah - it's actually supposed to be a string... My proposal was wrong. |
||
CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr) | ||
} else { | ||
throw ERROR.NOT_STARTED | ||
} | ||
} | ||
|
||
return sonus | ||
} | ||
|
||
|
@@ -107,8 +121,11 @@ Sonus.start = sonus => { | |
}) | ||
|
||
sonus.mic.pipe(sonus.detector) | ||
sonus.started = true | ||
} | ||
|
||
Sonus.trigger = sonus => sonus.trigger() | ||
|
||
Sonus.pause = sonus => sonus.mic.pause() | ||
|
||
Sonus.resume = sonus => sonus.mic.resume() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Languages
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks :)