Home > Bruce's Posts > Working with Freebase – Part 1

Working with Freebase – Part 1

Freebase provides a remote read service API for accessing Freebase database using the Metaweb query language (MQL). Using an HTTP endpoint an application can send a MQL query and received a result set in the response. The result set is returned in the JSON (JavaScript Object Notation) format.

In my case I wanted to query the database for quotes. For example, return quotations from the database. The following MQL returns all quotations associated with a person in the database.

[{
"type": "/people/person",
"id": null,
"name": null,
"/people/person/quotations": [{
"type": "/media_common/quotation",
"id": null,
"name": null,
}]
}]

This produces a results similar to this:

{
"code":          "/api/status/ok",
"result": [
{
"/people/person/quotations": [
{
"id":   "/en/first_thing_we_do_lets_kill_all_the_lawyers",
"name": "First thing we do, let's kill all the lawyers.",
"type": "/media_common/quotation"
},
{
"id":   "/en/o_brave_new_world_that_has_such_people_int",
"name": "...O brave new world, That has such people in't!",
"type": "/media_common/quotation"
}
],
"id":   "/en/william_shakespeare",
"name": "William Shakespeare",
"type": "/people/person"
},
{
"/people/person/quotations": [
{
"id":   "/m/02hylj7",
"name": "...the shifty, hangdog look which announces that an Englishman is about to talk French.",
"type": "/media_common/quotation"
},
{
"id":   "/en/routine_is_the_death_to_heroism",
"name": "Routine is the death to heroism.",
"type": "/media_common/quotation"
}
],
"id":   "/en/p_g_wodehouse",
"name": "P. G. Wodehouse",
"type": "/people/person"
}
],
"status":        "200 OK",
"transaction_id": "cache;cache02.p01.sjc1:8101;2013-02-14T07:31:07Z;0064"
}

In addition to executing the query I want the code to follow an ORM (Object Relational Mapping) style pattern. The code should know how to build the query, execute it, and return the results as objects. I used JPA annotations as a model for creating the mappings.

I created two annotations:

  • FBEntity – Describes which Freebase resource type the class is associated with.
  • FBProperty – Describes which Freebase property the method or field is associated with.

Using those annotations I create the following classes to represent a person and gender.

Person.java

@FBEntity(type = "/people/person")
public class Person {

	@FBProperty(property_value = "/people/person")
	private String type;
	@FBProperty
	private String id;
	@FBProperty
	private String name;
	@FBProperty
	private Gender gender;

	public String getType() {
		return type;
	}
	public void setType(String type) {
		this.type = type;
	}
	public String getId() {
		return id;
	}
	public void setId(String id) {
		this.id = id;
	}
	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}
	public Gender getGender() {
		return gender;
	}
	public void setGender(Gender gender) {
		this.gender = gender;
	}
}

Gender.java

@FBEntity(type = "/people/gender")
public class Gender {

	@FBProperty(property_value = "/people/gender")
	private String type;
	@FBProperty
	private String id;
	@FBProperty
	private String name;

	public String getType() {
		return type;
	}
	public void setType(String type) {
		this.type = type;
	}
	public String getId() {
		return id;
	}

	public void setId(String id) {
		this.id = id;
	}
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}
}

A query builder class uses the annotations to create a MQL query. This is where all the heavy lifting occurs. The builder will parse the class and for each field/method with an annotation it will generate the necessary MQL. It accepts special directives such as limit which are associated with a class. These directives can be used to control the behavior of the query.  It also accepts an instance of the object being queried and use it’s values to refine the query.

MQLQueryBuilder.java

public class MQLQueryBuilder {
	static private final String OPEN_SQUARE = "[";
	static private final String CLOSE_SQUARE = "]";
	static private final String OPEN_BRACKET = "{";
	static private final String CLOSE_BRACKET = "}";
	static private final String EMPTY = "";
	static private final String SPACE = " ";
	static private final String QUOTE = "\"";
	static private final String COMMA = ",";
	static private final String NEW_LINE = "\n";

	static public int PROPERTY_COMPACT	= 0;
	static public int PROPERTY_PRETTY	= 1;

	private String space = EMPTY;
	private String new_line = EMPTY;

	public MQLQueryBuilder(int property) {
		if (property == PROPERTY_COMPACT) {
			space = EMPTY;
			new_line = EMPTY;
		}
		else if (property == PROPERTY_PRETTY) {
			space = SPACE;
			new_line = NEW_LINE;
		}
	}

	public MQLQueryBuilder() {
		this(PROPERTY_COMPACT);
	}

	private String indent(int tab) {
		StringBuffer buffer = new StringBuffer();
		for (int i = 0; i < tab * 2; i++) {
			buffer.append(space);
		}
		return buffer.toString();
	}

	private String quote(String string) {
		return new StringBuffer().append(QUOTE).append(string).append(QUOTE).toString();
	}

	private Object getValueFromObject(Class<?> clazz, Object object, Field field) {
		//System.out.printf("clazz: %s  object: %s  field: %s\n", clazz, object, field);
		Object value = null;
		String name = "get".concat(field.getName()).toLowerCase();

		Method method = null;

		for (Method m : clazz.getMethods()) {
			if (m.getName().toLowerCase().equals(name)) {
				try {
					method = clazz.getMethod(m.getName());
				} catch (NoSuchMethodException e) {
					e.printStackTrace();
				} catch (SecurityException e) {
					e.printStackTrace();
				}
				break;
			}
		}

		if (method != null) {
			try {
				value = method.invoke(object);
			} catch (IllegalAccessException e) {
				e.printStackTrace();
			} catch (IllegalArgumentException e) {
				e.printStackTrace();
			} catch (InvocationTargetException e) {
				e.printStackTrace();
			}
		}

		return value;
	}

	private String createQuery(Class<?> clazz, Map<Class<?>, MQLProperty[]> map, Object object, Stack<Class<?>> stack, int tab) {
		StringBuffer sb = new StringBuffer();
		String indent = indent(tab);

		if (clazz.isArray()) {
			sb.append(String.format("%s", OPEN_SQUARE));
			sb.append(createQuery(clazz.getComponentType(), map, object, stack, tab + 1));
			sb.append(String.format("%s", CLOSE_SQUARE));
		}
		else {
			FBEntity fbEntity = clazz.getAnnotation(FBEntity.class);

			if (fbEntity != null) {
				stack.push(clazz);
				boolean next = false;
				sb.append(String.format("%s%s", OPEN_BRACKET, new_line));

				for (Field field : clazz.getDeclaredFields()) {
					Class<?> type = field.getType();
					if (!stack.contains(type.isArray() ? type.getComponentType() : type)) {
						FBProperty fbProperty = field.getAnnotation(FBProperty.class);
						if (fbProperty != null) {
							String name = quote((fbProperty.property_name().length() != 0) ? fbProperty.property_name() : field.getName());
							String value = null;

							if (type.isArray()) {
								value = createQuery(type, map, object != null ? getValueFromObject(clazz, object, field) : null, stack, tab + 1);
							}
							else {
								fbEntity = type.getAnnotation(FBEntity.class);
								if (fbEntity != null) {
									value = createQuery(type, map, object != null ? getValueFromObject(clazz, object, field) : null, stack, tab + 1);
								}
								else {
									if (object != null) {
										Object o = getValueFromObject(clazz, object, field);
										if (o != null) {
											value = quote(o.toString());
										}
									}

									if (value == null) {
										value = fbProperty.property_value().length() != 0 ? quote(fbProperty.property_value()) : null;
									}
								}
							}

							sb.append(String.format("%s%s%s:%s%s", next ? (COMMA + new_line) : EMPTY, indent, name, space, value));
							next = true;
						}
					}
				}

				if (map != null) {
					MQLProperty[] directives = map.get(clazz);
					if (directives != null) {
						for (MQLProperty directive : directives) {
							String name = quote(directive.getName());
							String value = directive.getValue() != null ? directive.getValue() instanceof String ? quote(directive.getValue().toString()) : directive.getValue().toString() : null;
							sb.append(String.format("%s%s%s:%s%s", next ? (COMMA + new_line) : EMPTY, indent, name, space, value));
							next = true;
						}
					}
				}

				sb.append(String.format("%s%s%s", new_line, indent, CLOSE_BRACKET));
				stack.pop();
			}
		}

		return sb.toString();
	}

	public String createQuery(Class<?> clazz, Map<Class<?>, MQLProperty[]> map, Object object) {
		return createQuery(clazz, map, object, new Stack<Class<?>>(), 0);
	}

	public String createQuery(Class<?> clazz, Map<Class<?>, MQLProperty[]> map) {
		return createQuery(clazz, map, null);
	}

	public String createQuery(Class<?> clazz) {
		return createQuery(clazz, null);
	}
}

The following test program will produces two queries. One will return all persons and the other only males. The limit directive limits the query to just three results. Passing an instance of Person with the Gender set to “Male” will refine the results to only males.

PersonTestOne.java

public class PersonTestOne {

	public static void main(String[] args) {
		Map<Class<?>, MQLProperty[]> map = new HashMap<Class<?>, MQLProperty[]>();

		MQLProperty[] directives = new MQLProperty[] {
				new MQLProperty("limit", new Integer(3))
		};
		map.put(Person.class, directives);

		MQLQueryBuilder builder = new MQLQueryBuilder(
				MQLQueryBuilder.PROPERTY_PRETTY);
		String mql = builder.createQuery(Person[].class, map);
		System.out.println(mql);

		Gender gender = new Gender();
		gender.setName("Male");
		Person person = new Person();
		person.setGender(gender);
		mql = builder.createQuery(Person[].class, map, person);
		System.out.println(mql);
	}
}

Running the program generates the following MQL.

[{
 "type": "/people/person",
 "id": null,
 "name": null,
 "gender": {
 "type": "/people/gender",
   "id": null,
   "name": null
   },
 "limit": 3
 }]

[{
 "type": "/people/person",
 "id": null,
 "name": null,
 "gender": {
 "type": "/people/gender",
   "id": null,
   "name": "Male"
   },
 "limit": 3
 }]

The next step is to execute the query in a manner that populates an array of objects with the results.

Advertisements
Categories: Bruce's Posts
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: