Data + Science

1/11/2016
Tabitha - Voice Recognition and Voice Response for Tableau

Getting started with Tableau Javascript API

First thing is to understand the basics of the Tableau Javascript API. If you are not at all familiar with it, there is a Tableau Javascript Tutorial by Tableau Software to help get you started. This will walk you through the code step by step.

Requirements for Tabitha

  • A Tableau Public Visualization that you would like to interact with
  • Tableau Javascript API
  • annyang.js
  • responsivevoice.js
  • Chrome browser
  • Hosting capability, preferably secure hosting for microphone permission
  • Some basic HTML and Javascript knowledge to put it together (listed step by step below)
  • Some sort of text editor, for example Notepad++ (download here for free)

Step 1 - Build your Viz

Create your visualization and load it up to Tableau Public. In this demonstration we will show how to control the viz using both Highlight and Filter Actions. To make it easy, start with a single worksheet or a simple dashboard and pick a single dimension, for example a category or state, that you wish to highlight or filer.

In this tutorial I will be using the "Star Words" viz created by Matt Francis located here.

Step 2 - Create a basic HTML page

Start with some basic tags to create a generic HTML page.

 
<html>
	<title>Page Title</title>
<head>
</head>

<body>
<h1>Hello World.</h1>

<script type='text/javascript' src='https://public.tableau.com/javascripts/api/viz_v1.js'></script><div class='tableauPlaceholder' style='width: 1004px; height: 869px;'><noscript><a href='#'><img alt='StarWords ' src='https://public.tableau.com/static/images/St/Starwars/StarWords/1_rss.png' style='border: none' /></a></noscript><object class='tableauViz' width='1004' height='869' style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='Starwars/StarWords' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https://public.tableau.com/static/images/St/Starwars/StarWords/1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /><param name='showVizHome' value='no' /><param name='showTabs' value='y' /><param name='bootstrapWhenNotified' value='true' /></object></div>

</body>

</html>

Standard Tableau Embed code could be inserted after the "Hello Word" line and you would have an embedded viz on your page. Instead of doing this, we are going to load the viz a different way. This is the starting point for using the Tableau Javascript API.

In the example below we add:

  • A reference to the Tableau Javascript in the "head" tag.
  • An onload event in the "body" tag calling the "initializeViz" function.
  • A div id referencing the viz (we use "center" to align the viz to the center)
  • In the script section:
      • create some variables for the viz, workbook and worksheet
      • create the function for "initializeViz"
      • enter the URL for the viz you want to call
        var url = "https://public.tableau.com/views/my-workbook/my-view";
        TIP - Use the Twitter share button on any Tableau Public Viz to get the URL
      • set width and height to match the viz
        TIP - Use the Embed code from the Tableau Public viz to get width and height

 
<html>
<head>
	<script src="https://public.tableausoftware.com/javascripts/api/tableau-2.min.js " type="text/javascript"></script> 
</head>
<body onload=initializeViz();>

<h1>Hello World.</h1>
<br>
<br>
<center><div id='tableauViz'></div></center>

<script>
var viz, workbook, activeSheet, Worksheet, worksheet;
function initializeViz(){

var placeholderDiv = document.getElementById("tableauViz");

var url = "https://public.tableau.com/views/Starwars/StarWords";
var options = {
   width: "1004",   
   height: "869",
   hideTabs: true,
   hideToolbar: true,
  onFirstInteractive: function () {
  workbook = viz.getWorkbook();
  activeSheet = workbook.getActiveSheet();
}
};
  viz = new tableauSoftware.Viz(placeholderDiv, url, options);
}
</script>


</body>
</html>

Step 3

Next, you will need to download 3 Javascript files and place in the directory with the webpage (or whatever directory you wish to call them from in the HTML code).

Right click on these files and "Save As" to save them to your computer. Then upload to your hosting server in the appropriate directory.

Voice Response requires responsivevoice.js

• responsivevoice.js

Voice Recognition requires two files: annyang.js and annyang.min.js

• annyang.js
• annyang.min.js

Step 4

Next we add code for the various viz functions. In this Star Words viz, we will create 4 selection functions and a function to start over for this tutorial. There is a demo at the bottom of this page that includes all of the functions for Matt's viz.

Place this code in the script tag, right before the closing "/script" tag section in the above code. In this example, the field name is "Character" and the values in the field are the names of the characters, for example "DARTH VADER" and "YODA". The function uses activeSheet and the selectMarks based on those selections. Think of it as dragging the blue pill named "Character" over to the filter shelf and then selecting the name of the character you are filtering.


//Make Character selection
function selectVader()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'DARTH VADER', 'REPLACE'); 
	}

function selectYoda()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'YODA', 'REPLACE'); 
	}

function selectSolo()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'HAN SOLO', 'REPLACE'); 
	}	

function selectLuke()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'LUKE SKYWALKER', 'REPLACE'); 
	}	
	
//Start Viz Over
function startover()
	{

	viz.revertAllAsync();
	}

Step 5

In the step we create voice commands, add them to annyang and start the listening. Each voice command line is structured with 3 parts:

   1.) the words that Tabitha is listening for
   2.) the function to control the viz
   3.) the voice response back from Tabitha

Example:
'Select Darth Vader': function() { selectVader(); responsiveVoice.speak('selecting Darth Vader', 'UK English Male'); },

'Select Darth Vader' - these are the words Tabitha is listening for
selectVader() - the function we defined to control the viz to select Character = "DARTH VADOR"
responsiveVoice.speak('selecting Darth Vader', "UK English Female"); - this is the text that Tabitha speaks back and the voice type.

The default is a female voice, but notice in the code below we have the response for Darth Vader switched to a "UK English Male" voice. There are 160+ voice types available in the latest responsivevoice.js. See the Javascript documentation on their website for more information (here)


<script>
if (annyang) {
  // These are the voice commands in quotes followed by the function
  var commands = {
  	'Select Darth Vader': function() { selectVader(); responsiveVoice.speak('selecting Darth Vader', 'UK English Male'); },
 	'Select Han Solo': function() { selectSolo(); responsiveVoice.speak('selecting Han Solo'); },
 	'Select Yoda': function() { selectYoda(); responsiveVoice.speak('selecting Yoda', 'UK English Female'); },
 	'Select Luke Skywalker': function() { selectLuke(); responsiveVoice.speak('selecting Luke Skywalker'); },
	'start over': function() { startover(); responsiveVoice.speak('starting over'); }
 };
  
  // Add commands to annyang
  annyang.addCommands(commands);
  
  // Start listening.
  annyang.start();
}
</script>

Putting it all together

Here's what the final HTML looks like for this tutorial. Feel free to copy and paste from the box below and play with it. In addition, I completed all of the functions for "Star Words" and included a link below.


<html>
<head>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

	<title>Data + Science</title>
		<meta charset="utf-8">
		<meta http-equiv="X-UA-Compatible" content="IE=edge">
		<meta name="viewport" content="width=device-width, initial-scale=1.0">
		<meta name="description" content="Creating Tabitha">
		<meta name="author" content="Jeffrey Shaffer">

		<script src="annyang.min.js"></script> <!-- Voice Recognition -->
		<script src="responsivevoice.js"></script> <!-- Voice Response -->
		
		<script src="https://public.tableausoftware.com/javascripts/api/tableau-2.min.js " type="text/javascript"></script> 

</head>	

<body onload=initializeViz();>
<h1>"Star Words" viz by Matt Francis</h1>
Say: <i>"Select Darth Vader" and "start over"</i>
<br>
<br>
<center><div id='tableauViz'></div></center>

<script>
if (annyang) {
  // These are the voice commands in quotes followed by the function
  var commands = {
  	'Select Darth Vader': function() { selectVader(); responsiveVoice.speak('selecting Darth Vader', "UK English Male"); },
 	'Select Han Solo': function() { selectSolo(); responsiveVoice.speak('selecting Han Solo'); },
 	'Select Yoda': function() { selectYoda(); responsiveVoice.speak('selecting Yoda', "UK English Female"); },
 	'Select Luke Skywalker': function() { selectLuke(); responsiveVoice.speak('selecting Luke Skywalker'); },
	'start over': function() { startover(); responsiveVoice.speak('starting over'); }
 };
  
  // Add commands to annyang
  annyang.addCommands(commands);
  
  // Start listening.
  annyang.start();
}
</script>

<script>
var viz, workbook, activeSheet, Worksheet, worksheet;
function initializeViz(){

var placeholderDiv = document.getElementById("tableauViz");

var url = "https://public.tableau.com/views/Starwars/StarWords";
var options = {
   width: "1004",   
   height: "869",
   hideTabs: true,
   hideToolbar: true,
  onFirstInteractive: function () {
  workbook = viz.getWorkbook();
  activeSheet = workbook.getActiveSheet();
}
};
  viz = new tableauSoftware.Viz(placeholderDiv, url, options);
}

//Tabitha says hello
responsiveVoice.speak('Hi. My name is Tabitha. How may I help you?');

//Make Character selection
function selectVader()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'DARTH VADER', 'REPLACE'); 
	}

function selectYoda()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'YODA', 'REPLACE'); 
	}

function selectSolo()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'HAN SOLO', 'REPLACE'); 
	}	

function selectLuke()
	{
	activeSheet.getWorksheets()[1].selectMarksAsync('Character', 'LUKE SKYWALKER', 'REPLACE'); 
	}	
	
//Start Viz Over
function startover()
	{

	viz.revertAllAsync();
	}

</script>

</body>
</html>

A completed version of this tutorial with all of the commands added is available here.

Using Filters instead of Selecting Marks/Highlight

When using Filtering instead of Highlighting, use "applyFilterAsync" instead of "selectMarksAsync"

For Example, this code filters for the State="AL" instead of highlighting.

    function filterAlabama()
    {
    activeSheet.getWorksheets()[1].applyFilterAsync('State', 'AL', tableauSoftware.FilterUpdateType.REPLACE);
    }

This is the technique I used in my blog post from March of last year when I added voice to Kelly Martin's Birdstrike viz here.

I hope you have fun adding Tabitha to your visualizations. If you have any questions feel free to email me at Jeff@DataPlusScience.com

Jeffrey A. Shaffer

Follow on Twitter @HighVizAbility