We all lead busy lives, so it can be a challenge to find timely and relevant information. But what if you had a personal assistant to deliver this for you? In this lab, you will build your own cognitive cloud app powered by IBM Watson to deliver news and weather customized for your preferences and location. You will build your own Node.js application deployed to IBM Bluemix and use services from the Watson Alchemy News and the Weather Company. News articles are analyzed for category and sentiment and a real-time weather forecast is delivered for your location. You will interact with your assistant using Watson speech-to-text and text-to-speech services. In the process of doing all this, you will also learn more about Node-RED, Node.js and Bluemix, as well as some of the services that Bluemix has to offer.
Node-RED is an open-source visual tool, originated at IBM and now owned by the JS Foundation, that allows you to wire together IoT devices, API's and online services in interesting ways to build applications. A payload of data in JSON format flows from left to right and travels through a series of nodes that are connected by wires. Something triggers the flow -- could be an IoT device, or a twitter stream, for example, and then the nodes after it can analyze the data and decide on an action and finally an output node will send data somewhere (for example, to a database, or an alerting system). This is an overview of how Node-RED works -- You can learn more about Node-RED at nodered.org.
For the application you will be building today, your Node-RED flow will be triggered by a REST call issued by a microphone application that uses Watson to record and translate your voice to text. That text is then sent to your Node-RED instance via REST call. Your Node-RED instance will be a hosted application on Bluemix, IBM's PaaS. We will provide the microphone application, which acts like a client, and you will use Node-RED to build the backend that services the incoming text request and returns a voice response, which will then be played by the client application on your laptop's speaker. Your application will utilize news, weather and Watson text-to-speech services to create the response.
While Node-RED can run on many different hardware, including Raspberry Pi, in this lab, you will be working with a special version of Node-RED that is hosted on the cloud with Bluemix and has special nodes that can interface with Bluemix services including Watson services. If you are interested in learning more about how to build Node-RED applications that interface with Bluemix Watson services, this link has a variety of interesting starter-pack applications.
Bluemix is a cloud PaaS (Platform as a Service), which allows you to host your application online and bind it to a variety of SaaS service offerings from IBM including Watson analytic services. Learn more at http://www.ibm.com/bluemix and if you are new to Bluemix, you can create a trial account at http://www.bluemix.net/ (click "Sign Up").
Watson speech-to-text service converts the human voice into the written word using machine intelligence to analyze the grammar and language structure to generate an accurate transcription in real time. The transcription of the incoming audio is sent back continuously and corrected as more speech is heard. In our demo app, you will see confidence scores from Watson assigned to each word along with alternative words that it was considering.
You can try it out in your favorite language at this link -- please use the Chrome browser:
https://wow-mic.mybluemix.net/
Click "Record Audio" and wait for it to go red before you speak, then click it again to stop recording. Don't worry about the Node-RED server URL yet; you will use that later in this lab.
The Watson text-to-speech service produces natural-sounding speech from input text with appropriate cadence and intonation and also supports a variety of languages.
Learn more about Watson-powered STT and TTS at these links: http://www.ibm.com/watson/developercloud/speech-to-text.html https://www.ibm.com/watson/developercloud/text-to-speech.html
The Weather Company Data Service from Bluemix lets you integrate weather data from The Weather Company into your IBM Bluemix application for any specified geolocation. The service supports real-time as well we forecasts and historical weather data. The Weather Company was acquired by IBM in 2015 and is the world's largest private weather enterprise, delivering up to 26 billion forecasts daily. TWC powers top weather apps on all major mobile platforms, so you have likely used it before, "weather" you knew it or not.
The Alchemy Data News service indexes hundreds of thousands of articles and blogs every day, analyzes the text with machine learning algorithms, and makes those articles searchable across a variety of fields, including concept and sentiment. API documentation for the Alchemy News service is located here:
http://docs.alchemyapi.com/docs
Now let's build our application!
-
If you haven't already, create a Bluemix account at https://console.ng.bluemix.net/. You will be asked to select a unique name for your organization and space where you would like to host the application that you will build today.
-
Click on catalog to get various options of boilerplate code. Select Node-RED.
- Name the application and click on Create.
- Your application will start running. The app will have cloudant database by default to store node red metadata.
- Click on Overview tab to see the detailed view of your application.
- Click on Connections tab to add two more services.
Weather Company DataText to Speech
- Restage the application after adding each service.
- Click on the View App of your application. You will be taken to NODE-RED
- Click on this button to go into Node-red editor.
- Copy flows from this github repository and import in your Node-red editor like below.
- Import Main Flow
- Import Weather Flow
- Import News Flow
- Click on the "text to speech" node and select "US English" on language dropdown.
- The "name" field can be left blank. Finally, click on the each link node on the main tab.
- Connect to the proper receive node.
Outgoing weather should connect to Incoming weather andOutgoing news should connect to Incoming news
- Click on Deploy.
- We are using another Bluemix application to record our message and send it to our node-red app. Check the link below.
https://wow-mic.mybluemix.net/
- Note that if you have time / interest, you can also clone this code yourself and host it on Bluemix (you will also need to create and connect the Watson "speech to text" service to it): https://github.com/CDSLab/WOW-mic
To use the microphone client app, simply enter your Node-RED application link in the text box shown, as:
<--host name-->.mybluemix.net
Example: watsonkg.mybluemix.net Click on Record Audio to record your message eg.
What is the current weather in Chicago.
Stop recording and upload it to Node-RED by clicking on the "Upload to Node-RED" button.
If the URL you entered is wrong, an error message will pop up like below:
If everything works, you will receive a voice response (turn up your speakers).
Questions currently supported are:
- What is the [current] weather in (location) ?
- What is the weather forecast for (location) ?
- Give me [ positive | negative ] news about (topic for title search)
Now we will study each flow a little further to understand more fully how this application works.
Explanation: Text is passed in via a REST endpoint. Then we feed that into a function node which calls decodeURIComponent to remove the URL format of the text. That text is then fed into a switch statement which routes traffic to the news or weather flows based on if the input text contains "news" or "weather" keyword. If the text contains "read it", then the flow grabs the last requested news article from Cloudant and flows that into the text-to-speech node. If the text does not contain any recognized keywords, then pin 4 goes to a "format speech" node which formulates a catch-all response, "I don't understand what you meant".
Explanation: This flow is triggered when the user asks for the weather. The first thing that needs to be determined is the precise location that is being requested. The first node in this flow, "Construct URL", is a function node that gets ready to call the Google maps API. Open it up (double-click) and you will see some javascript that first parses the incoming text and scrapes the word(s) after "in" or "for" - this is assumed to be the location requested. A string URL is created and passed as msg.url that will ask Google for more information about that location, including the latitude and longitude coordinates. That URL is then fed into the http request node which returns the response from google. The response is a string manifestation of JSON, so the next node will convert that string into parseable JSON.
Next we have some error checking. The "Error check google API results" checks if zero results are returned from Google -- if so, a response is generated for the assistant to say that the location is not recognized. But if results are returned, then a function node parses the JSON to extract the latitude and longitude coordinates. Then the text transcription is parsed further to determine if "current" or "forecast" is requested, and that goes to the appropriate weather node which is pre-configured to return either the current weather or the forecast. The JSON response from the weather node (which calls the weather service) is then parsed and wrapped inside an English response which then goes to the "text to speech" node which returns a wav in the payload. Finally an audio html tag is added to the response and sent back with the wav as the http response, which allows the browser client to play the audio.
Explanation: This flow gets triggered when the user asks for news. In "construct msg.query", incoming text is parsed to build a query to pass to the Alchemy News service via the News node. Anything after "about" is set as the title search field, and if you want news articles with "positive" or "negative" sentiment, that is configured as well.
More options passed to the News node include: start=now-1d&end=now which requests news articles from the last 24 hours and rank=high which requests popular articles from reputable sources. Debug nodes are placed before and after the call to the News node so you can see the input and outputs to the News service in the Debug panel.
The response received from the News node is parsed and formatted and sent to the text-to-speech node and then returned as audio in the http response.
You will also notice another wire flowing out of the News node and going to Cloudant. That flow writes the text from the news article and persists it to a Cloudant database incase the says "Read it!".
If you so wish, you can try to modify the query parameters passed into the news node, for example, you could change start=node-1d to start=node-1h for fresher news updates. Or you could play with the sentiment thresholds. Or there are many other options you could use to filter and search for articles, the API documentation is located at this link:
http://docs.alchemyapi.com/docs
This is your application to modify and extend as you wish! We recommend that you experiment and start with small changes. For example, toggle the voice from male to female, or modify the text responses that are returned. Later maybe you could add a whole new category of question with a new flow and even hook it up to other Bluemix services. Your imagination is the limit!