Home > Freebase, Java > Google Freebase Client Library

Google Freebase Client Library

Google’s APIs Client Library for Java provides a way to utilize Google APIs using JSON and OAuth 2.0. One library provides access to Freebase and this post describes my experiences using it.

It quickly became clear that this library is a work in progress. It labels itself as beta but I’d describe it as alpha. The documentation is sparse, I couldn’t find good Java samples, and the library’s functionality is incomplete. Having said that, this library could be quite useful once completed.

Freebase’s HTTP API is well documented and you can find samples such as this that use Google’s HTTP client library but not the Freebase library.

public class FreebaseSearchExample {
 public static String API_KEY = "YOUR-API-KEY-GOES-HERE";
 public static void main(String[] args) {
  try {
    HttpTransport httpTransport = new NetHttpTransport();
    HttpRequestFactory requestFactory = httpTransport.createRequestFactory();
    JSONParser parser = new JSONParser();
    GenericUrl url = new GenericUrl("https://www.googleapis.com/freebase/v1/search");
    url.put("query", "Cee Lo Green");
    url.put("filter", "(all type:/music/artist created:\"The Lady Killer\")");
    url.put("limit", "10");
    url.put("indent", "true");
    url.put("key", FreebaseSearchExample.API_KEY);
    HttpRequest request = requestFactory.buildGetRequest(url);
    HttpResponse httpResponse = request.execute();
    JSONObject response = (JSONObject)parser.parse(httpResponse.parseAsString());
    JSONArray results = (JSONArray)response.get("result");
    for (Object result : results) {
      System.out.println(JsonPath.read(result,"$.name").toString());
    }
  } catch (Exception ex) {
    ex.printStackTrace();
  }
 }
}

All calls to Freebase are returned within a JSON envelop. One of the biggest decisions a programmer must make is how to unmarshal the reply. The example above stores the response in vanilla JSONArray object. This is all well and good but further use of the results requires sifting through arrays for the desired objects. Later I’ll show how to parse the response into a POJO.

The response from Freebase contains two parts. One, a JSON envelope that either represents a successful execution of the request or an error message. The examples I’ve seen don’t deal with error checking or handling. At this point I won’t either and leave that for another post. A successful response from MQL Read will look like this.

{
"result": [
{
"name": "Jack Abramoff",
"type": "/people/person"
},
{
"name": "Bob Ney",
"type": "/people/person"
},
{
"name": "David Safavian",
"type": "/people/person"
}
],
}

In this case “result” contains the data requested. Other API calls, such as Topic, return different name/value pairs. The envelop can also contain values such as cost, cursor, and status. The unmarshalling code needs to handle (either accept and/or ignore) all the values returned in the envelop.

The Freebase client library supports the following APIs.

Freebase API Freebase Client Class Description Functional
Text Freebase.Text Returns a blob of HTML data pertaining to a specified id. Yes
Topic Freebase.Topic Returns properties and meta-data about a topic at a specified id. Yes
Image Freebase.Image imageReturns the scaled/cropped image attached to a freebase node. No
MQL Read Freebase.Mqlread Query the Freebase database using the Metaweb query language (MQL). No
MQL Write Freebase.Mqlwrite Write to the Freebase database No

For some reason the library does not support Search or Reconciliation.

The Text API returns a result containing a blob (blurb) about the specified topic. For some reason it’s not described in the Freebase developer guide. One answer on stackoverflow describes it as …

The freebase text api (http://wiki.freebase.com/wiki/ApiText) gives you back a lot of the top blurb from the wikipedia page.

I think this is all the that Freebase (and DBPedia) store about the topic. To get the full text of the wikipedia article, you can go directly to wikipedia.

For example a call to the Text API using Albert Einstein’s mid produces the following.


Request:
https://www.googleapis.com/freebase/v1/text/m/0jcx

Response:
{ “result”: “Albert Einstein ( /ˈælbərt ˈaɪnstaɪn/; German: [ˈalbɐt ˈaɪnʃtaɪn] ( listen); 14 March 1879 – 18 April 1955) was a German theoretical physicist who developed the theory of general relativity, effecting a revolution in physics. For this achievement, Einstein is often regarded as the father of modern physics. Einstein is generally considered the most influential physicist of the 20th century. While best known for his mass–energy equivalence formula E = mc (which has been dubbed \”the world’s most famous equation\”), he received the 1921 Nobel Prize in Physics \”for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect\”. The latter was pivotal in establishing quantum theory within physics.\nNear the beginning of his career, Einstein thought that Newtonian mechanics was no longer enough to reconcile the laws of classical mechanics with the laws of the electromagnetic field. This led to the development of his special theory of relativity. He realized, however, that the principle of relativity could also be extended to gravitational fields, and with his subsequent theory of gravitation in 1916, he published a paper on the general theory” }

Here’s a sample that uses the Text client API

public class GetTest {
 public static void main(String[] args) {
  HttpTransport httpTransport = new NetHttpTransport();
  JsonFactory jsonFactory = new JacksonFactory();
  HttpRequestInitializer httpRequestInitializer = new HttpRequestInitializer() {
   @Override
   public void initialize(HttpRequest request) throws IOException {
   }
  };

  Freebase.Builder fbb = new  Freebase.Builder(httpTransport, jsonFactory,
          httpRequestInitializer);
  Freebase freebase = fbb.build();
  ArrayList list = new ArrayList();
  try {
   list.add("/m/0jcx");
   Freebase.Text.Get text = freebase.text().get(list);
   ContentserviceGet get = text.execute();
   System.out.println(get.toPrettyString());
  } catch (Exception e) {
  e.printStackTrace();
  }
 }
}

ContentserverGet is a simple Class class to receive the JSON envelop.

public final class ContentserviceGet extends GenericJson {

 @com.google.api.client.util.Key
 private String result;

 public String getResult() {
  return result;
 }

 public ContentserviceGet setResult(String result) {
  this.result = result;
  return this;
 }
}

@com.google.api.client.util.Key specifies that a field is a data key, optionally providing the data key name to use. The JSON parser uses this key to unmarshal the JSON “result” value into the result field. In this case things are pretty simple because we’re only dealing with a string.

The Topic API returns all the known facts for a given topic including images and text blurbs. It will return a lot of data. It’s like getting back everything but the kitchen sink. Clink on this link for an example. The request also supports a filter property to reduce the amount of information returned. A call to the Topic API returns two root level values (id and property). Property contains a map of all the associated properties. Here’s a sample that uses the Topic client API.

public class TopicTest {
 public static void main(String[] args) {
  HttpTransport httpTransport = new NetHttpTransport();
  JsonFactory jsonFactory = new JacksonFactory();
  HttpRequestInitializer httpRequestInitializer = new HttpRequestInitializer() {

   @Override
   public void initialize(HttpRequest request) throws IOException {
   }
  };

  Freebase.Builder fbb = new  Freebase.Builder(httpTransport, jsonFactory,
      httpRequestInitializer);
  Freebase freebase = fbb.build();
  ArrayList list = new ArrayList();
  try {
   list.add("/m/0jcx");
   Freebase.Topic.Lookup lookup = freebase.topic().lookup(list);
   TopicLookup topic = lookup.execute();
   System.out.println(topic.toPrettyString());
  } catch (Exception e) {
   e.printStackTrace();
  }
 }
}

In this case TopicLookup, which is a little more complex than ContentserverGet, receives the JSON envelop. If suffice to say that TopicLookup is filled with the map of all the properties returned and can be programmatically traversed as desired.

The MQL Read API provides access to the Freebase database using the Metaweb query language (MQL). For example, the following call will query for Albert Einstein’s children.


Request:
https://www.googleapis.com/freebase/v1/mqlread?query={"type":"/people/person","id":"/en/albert_einstein","children":[]}

Result:
{
“result”: {
“children”: [
“Eduard Einstein”,
“Hans Albert Einstein”,
“Lieserl Einstein”,
“Ilse Einstein”,
“Margot Einstein”
],
“type”: “/people/person”,
“id”: “/en/albert_einstein”
}
}

Freebase.Mqlread reminds me of all the empty cities in China. Everything needed is there but not used. It’s currently coded as following …

Mqlread extends FreebaseRequest<Void>

This results in the query being executed but nothing returned to the caller.

I wanted to try it out so I modified the source to use a generic type …

Mqlread<T> extends FreebaseRequest<T>

I also needed to pass along the generic’s Class when instantiating Mqlread.

public Mqlread mqlread(String query, Class<?> classType) throws java.io.IOException {
  Mqlread result = new Mqlread(query, classType);
  initialize(result);
  return result;
}

public class Mqlread extends FreebaseRequest {

...

  protected Mqlread(String query, Class classType) {
    super(Freebase.this, "GET", REST_PATH, null, classType);
    this.query = Preconditions.checkNotNull(query, "Required parameter query must be specified.");
  }

...

}

The following code uses the modified Freebase.Mqlread to fetch Albert Einstein’s children.

public class MQLReadTest {
	static private final String query =
			"{\"type\":\"/people/person\",\"id\":\"/en/albert_einstein\",\"children\":[]}";

	public static void main(String[] args) {
	    HttpTransport httpTransport = new NetHttpTransport();
	    JsonFactory jsonFactory = new JacksonFactory();
	    HttpRequestInitializer httpRequestInitializer = new HttpRequestInitializer() {

			@Override
			public void initialize(HttpRequest request) throws IOException {
			}
	    };

	    Freebase.Builder fbb = new  Freebase.Builder(httpTransport, jsonFactory, httpRequestInitializer);
	    Freebase freebase = fbb.build();
	    try {
	        Freebase.Mqlread mqlRead = freebase.mqlread(query, GenericJson.class);
	        GenericJson response = mqlRead.execute();
	        System.out.println(response.toPrettyString());
	    } catch (Exception e) {
	        e.printStackTrace();
	    }
	}
}

Using GenericJson as the generic causes the parser to create a map of the response returned by the query. This is similar to what occurs with Freebase.Topic. Getting back the results as a map is all fine and well but if I’m making the query for very specific reason I’d prefer that it be parsed into a POJO that lets me directly access the data I need. This can be done but it takes a little work.

I created the Person class to hold values from Freebase’s /people/person topic. I want the JSON parser to populate the children string array with the children values returned from the query.

public class Person {
 @com.google.api.client.util.Key
 private String[] children;

 public String[] getChildren() {
  return children;
 }

 public void setChildren(String[] children) {
  this.children = children;
 }
}

I created SingleMQLReadPersonResponse to hold envelop returned by the query. The result field is typed as Person so the JSON parser should populate it accordingly.


public class SingleMQLReadPersonResponse extends GenericJson {
 @com.google.api.client.util.Key
 private Person result;

 public Person getResult() {
 return result;
 }

 public void setResult(Person result) {
 this.result = result;
 }
 }

<span style="font-family: Consolas, Monaco, monospace; font-size: 12px; line-height: 18px;">

With the following code I can directly access the person’s children.


Freebase.Mqlread mqlRead = freebase.mqlread(query, SingleMQLReadPersonResponse.class);
 SingleMQLReadPersonResponse response = mqlRead.execute();
 System.out.println(response.toPrettyString());
 Person person = response.getResult();
 System.out.println(person);
 for (String child : person.getChildren()) {
 System.out.println(child);
 }

Here’s the output …

{
"result" : {
"children" : [ "Eduard Einstein", "Hans Albert Einstein", "Lieserl Einstein", "Ilse Einstein", "Margot Einstein" ]
}
}
org.bwgz.freebase.sample.MQLReadTest$Person@3aa59d
Eduard Einstein
Hans Albert Einstein
Lieserl Einstein
Ilse Einstein
Margot Einstein

So far so good. But I don’t want to create a response envelop class for every query I plan to make. I’d rather use Java generics to create a single class that can be reused. Something like this …


public static class SingleMQLReadResponse extends GenericJson {
 @com.google.api.client.util.Key
 private T result;

 public T getResult() {
 return result;
 }

 public void setResult(T result) {
 this.result = result;
 }
 }

Here’s the code that uses it.


Freebase.Mqlread<SingleMQLReadResponse> mqlRead = freebase.mqlread(query, SingleMQLReadResponse.class);
 SingleMQLReadResponse response = mqlRead.execute();
 System.out.println(response.toPrettyString());
 Person person = response.getResult();
 System.out.println(person);

But there a problem.  Here’s the output …

{
"result" : {
"id" : "/en/albert_einstein",
"type" : "/people/person",
"children" : [ "Eduard Einstein", "Hans Albert Einstein", "Lieserl Einstein", "Ilse Einstein", "Margot Einstein" ]
}
}
java.lang.ClassCastException: com.google.api.client.util.ArrayMap cannot be cast to org.bwgz.freebase.sample.MQLReadTest$Person
at org.bwgz.freebase.sample.MQLReadTest.SingleResponseTest2(MQLReadTest.java:101)
at org.bwgz.freebase.sample.MQLReadTest.main(MQLReadTest.java:123)

While SingleMQLReadResponse has been typed to Person the mqlread method is only passed SingleMQLReadResponse.class. Java does not allow SingleMQLReadResponse<T>.class. The parser cannot determine the result field’s class (type) and it falls back to com.google.api.client.util.ArrayMap. That in turn causes the cast exception.

Luckily there a way around this. The underlying code will use the class’s type (Type) when converting JSON to a Java object. If SingleMQLReadResponse.class is passed it’s parameter types are not included. This wasn’t a problem when SingleMQLReadPersonResponse.class was passed because the result field as explicitly typed to Person. The underlying code will accept a Type instead of a Class. If we can pass in a ParameterizedType the parser will have the information it needs to convert the JSON results to the specified Java object. There are two things we need:

  • Create a ParameterizedType representing the response class we are using.
  • Get the underlying code to use Type rather than Class.

Java contains the ParameterizedTypeImpl class which will produce a ParameterizedType object. Unfortunately it is protected and can’t be accessed. Rather than write my own I just grabbed the source code and included it in my project. The code isn’t complicated and I could have written my own but why bother when it’s freely available. The following code creates the Type object I need and takes create of the first thing I needed.

ParameterizedTypeImpl.make(SingleMQLReadResponse.class, new Type[] { Person.class }, null));

The execute method is located in AbstractGoogleClientRequest which is a deeply nested superclass of Mqlread. Here’s the code for execute.


public T execute() throws IOException {
 HttpResponse response = executeUnparsed();
 // TODO(yanivi): remove workaround when feature is implemented
 // workaround for http://code.google.com/p/google-http-java-client/issues/detail?id=110
 if (Void.class.equals(responseClass)) {
 response.ignore();
 return null;
 }
 return response.parseAs(responseClass);
 }

This code parses the HTTP response using the class (responseClass) which was passed through when Mqlread was constructed. HttpResponse.parse will accept either a Class or a Type. I don’t want to mess with AbstractGoogleClientRequest but I’ve already got my mitts into Freebase.Mqlread and there are changes I can make there.

First, create a second constructor with accepts a Type and saves it.

protected Mqlread(String query, Class responseClass) {
 super(Freebase.this, "GET", REST_PATH, null, responseClass);
 this.query = Preconditions.checkNotNull(query, "Required parameter query must be specified.");
 }

// Here's the second constructor.

protected Mqlread(String query, Class responseClass, Type responseType) {
 this(query, responseClass);
 this.responseType = responseType;
 }

I still have to pass along responseClass even though it won’t be used. If I don’t the underlying code will throw a validation exception.

Second, I override the execute method.

private T execute(Type type) throws IOException {
 HttpResponse response = executeUnparsed();

 return (T) response.parseAs(type);
 }

@Override
 public T execute() throws IOException {
 return (getResponseType() != null) ? execute(getResponseType()) : super.execute();
 }

Now we I run this code everything is parsed as expected and get the same results as with SingleMQLReadPersonResponse.

Type type = ParameterizedTypeImpl.make(SingleMQLReadResponse.class, new Type[] { Person.class }, null);

Freebase.Mqlread<SingleMQLReadResponse> mqlRead = freebase.mqlread(query, SingleMQLReadResponse.class, type);
 SingleMQLReadResponse response = mqlRead.execute();
 System.out.println(response.toPrettyString());
 Person person = response.getResult();
 System.out.println(person);
 for (String child : person.getChildren()) {
 System.out.println(child);
 }

{
"result" : {
"children" : [ "Eduard Einstein", "Hans Albert Einstein", "Lieserl Einstein", "Ilse Einstein", "Margot Einstein" ]
}
}
org.bwgz.freebase.sample.MQLReadTest$Person@b3826f
Eduard Einstein
Hans Albert Einstein
Lieserl Einstein
Ilse Einstein
Margot Einstein

Some final thoughts.

  • When Mqlread is completed classes like SingleMQLReadResponse will likely exist in com.google.api.services.freebase.model.
  • The Google Client API should support Type along with Class parsing. When that occurs Mqlread can more easily support parameterized types.
  • Error handling needs to be considered. If the response returns a JSON envelop with an error message then there’s a need to easily capture and use it.

The source code is available at GitHub.

Advertisements
Categories: Freebase, Java
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: