About the Search Suggestion Explorer

The Search Suggestion Explorer is an example of an online tool built using the Google Suggest web service. Google Suggest is what lets Google Toolbar suggest search refinements as the user types into the search box. Although its primary purpose is search term autocompletion, the Google Suggest web service (GSWS) provides interesting search volume information that also makes it a useful keyword research tool. Using GSWS within your own applications — whether they're client- or server-side — is extremely simple, and this short guide gets you started.

Invoking Google Suggest

Since Google Suggest is exposed as a web service (in the loose sense), any programming environment that can make HTTP calls can invoke it and obtain the same data that the Google Toolbar receives. All you need to know is the URL format:

http://www.google.com/complete/search?hl=lang&type=true&qu=query

There are three values to substitute in this URL:

lang is the language for the results, a standard 2-letter language code. Right now only English results are supported, so en is the only choice.
type is the desired format for the results. At least three formats are supported: js for JavaScript, csv for comma-separate values, and xml for XML.
query is the string to complete.

You can try the GSWS directly from your web browser by forming the appropriate URL, as in the following examples.

CSV

This URL returns the suggestions for "eric" in CSV format:

http://www.google.com/complete/search?hl=en&csv=true&qu=eric

This is what you see:

"eric" "eric clapton","eric","ericsson","erica durance","erica campbell",
"eric prydz","eric bana","eric johnson","eric carle","eric clapton lyrics" 
"1,480,000 results","47,900,000 results","68,800,000 results",
"26,400 results","401,000 results","411,000 results","409,000 results",
"7,600,000 results","261,000 results","431,000 results" ""

XML

This URL returns suggestions for "eric clapton", but in XML format:

http://www.google.com/complete/search?hl=en&xml=true&qu=eric+clapton

Notice how any space characters in the search string are replaced with '+' characters. You can also replace them with the string "%20" instead — in other words, "eric+clapton" and "eric%20clapton" are identical when used in a URL like this. (This is known as URL encoding, by the way, and it's a way to pass characters that aren't normally allowed in a URL to the web server.)

The returned XML looks like this:

<?xml version="1.0"?>
<toplevel>
  <CompleteSuggestion>
    <suggestion data="eric clapton"/>
    <num_queries int="1480000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric"/>
    <num_queries int="47900000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="ericsson"/>
    <num_queries int="68800000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="erica durance"/>
    <num_queries int="26400"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="erica campbell"/>
    <num_queries int="401000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric prydz"/>
    <num_queries int="411000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric bana"/>
    <num_queries int="409000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric johnson"/>
    <num_queries int="7600000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric carle"/>
    <num_queries int="261000"/>
  </CompleteSuggestion>
  <CompleteSuggestion>
    <suggestion data="eric clapton lyrics"/>
    <num_queries int="431000"/>
  </CompleteSuggestion>
</toplevel>

JavaScript

This URL returns suggestions for "eric clapton" in JavaScript format:

http://www.google.com/complete/search?hl=en&js=true&qu=eric+clapton

The string that you see can be directly interpreted by a JavaScript application (think Ajax) via the eval() function:

sendRPCDone(frameElement, "eric clapton", new Array("eric clapton", 
             "eric clapton lyrics",
	     "eric clapton tabs", "eric clapton biography",
	     "eric clapton discography", "eric clapton cream",
	     "eric clapton guitar tabs", "eric clapton layla",
	     "eric clapton tears in heaven",
	     "eric clapton songs"),
	     new Array("1,480,000 results", "431,000 results",
	     "208,000 results", "120,000 results",
	     "96,400 results", "298,000 results",
	     "221,000 results", "144,000 results",
	     "200,000 results", "1,070,000 results"),
	     new Array(""));

HMTL

Here's another trick. If you drop the type specifier altogether and invoke the GSWS, you appear to get nothing:

http://www.google.com/complete/search?hl=en&qu=eric+clapton

But this isn't true. Keep the window open (or re-open it if you closed it) and use the "View source" menu item (or equivalent) to see the underlying HTML:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<script>
function bodyLoad() {
      if (parent == window) return;
        var frameElement = this.frameElement;
	  parent.sendRPCDone(frameElement, "eric clapton",
	     new Array("eric clapton", "eric clapton lyrics",
	     "eric clapton tabs", "eric clapton biography",
	     "eric clapton discography", "eric clapton cream",
	     "eric clapton guitar tabs", "eric clapton layla",
	     "eric clapton tears in heaven",
	     "eric clapton songs"),
	     new Array("1,480,000 results", "431,000 results",
	     "208,000 results", "120,000 results",
	     "96,400 results", "298,000 results",
	     "221,000 results", "144,000 results",
	     "200,000 results", "1,070,000 results"),
	     new Array(""));
}
</script></head><body onload='bodyLoad();'></body></html>

This JavaScript executes if and only if the HTML is loaded into a framed window. If you look closely, it's almost identical to the JavaScript code returned by js=true, except that it's wrapped with enough HTML to be directly invoked by the browser.

Invoking Google Suggest from Java

In standard Java, use the URLConnection class to invoke the service:

public InputStream invokeGoogleSuggest( String query ) throws IOException {
    String url = formGSURL( query ); // creates the URL
    URL u = new URL( url );
    URLConnection conn = u.openConnection();

    return conn.getInputStream();
}

The Java code to form the URL looks like this:

public String formGSURL( String query ) {
    String url = "http://www.google.com/complete/search?hl=en&xml=true&qu=";
    return url + java.net.URLEncoder.encode( query );
}

In Java ME (formerly known as J2ME) things are more complicated because of the encoding (there's no built-in encoder in MIDP), but the HttpConnection class makes it easy to get the connection:

public InputStream invokeGoogleSuggest( String query ) throws IOException {
    String url = .... // must encode the string here
    return ((HttpConnection) Connector.open( url )).openInputStream();
}

Of course, it would be better to do more error checking than what's shown here (none).

Parsing the Data

How you parse the data depends on which format you've decided to use and what programming language you're using. The XML format is the most portable, but most XML parsers are memory-hungry and that may be a deterrent. The Search Suggestion Explorer is written in PHP and parses the JavaScript format using regular expressions.

Note: Integers are formatted in a locale-sensitive fashion in the JavaScript, HTML and CSV formats, but not in the XML format.