茄子直播

Working with his student, Professor Parham Aarabi developed a search tool that quantifies relationships between people even when they are not tagged in a photo (photo by Johnny Guatto)

New algorithm finds you, even in an untagged photo

A new algorithm designed at the University of Toronto has the power to profoundly change the way we find photos among the billions on social media sites such as Facebook and Flickr. 

And, this month, the United States Patent and Trademark Office will issue a patent on this technology.

Developed by Parham Aarabi, a professor in The Edward S. Rogers Sr. Department of Electrical & Computer Engineering, and his former master鈥檚 student Ron Appel, the search tool uses tag locations to quantify relationships between individuals, even those not tagged in any given photo.

Imagine you and your mother are pictured together, building a sandcastle at the beach. You鈥檙e both tagged in the photo quite close together. In the next photo, you and your father are eating watermelon. You鈥檙e both tagged.

Thanks to your close 鈥榯agging鈥 relationship with both your mother in the first picture and your father in the second, the algorithm can determine that a relationship exists between those two and quantify how strong it may be. In a third photo, you fly a kite with both parents, but only your mother is tagged. Given the strength of your 鈥榯agging鈥 relationship with your parents, when you search for photos of your father the algorithm can return the untagged photo because of the very high likelihood he鈥檚 pictured.

鈥淭wo things are happening: we understand relationships, and we can search images better,鈥 says Professor Aarabi.

The nimble algorithm, called relational social image search, achieves high reliability without using computationally-intensive object-recognition or facial-recognition software.

鈥淚f you want to search a trillion photos, normally that takes at least a trillion operations. It鈥檚 based on the number of photos you have,鈥 says Aarabi. 鈥淔acebook has almost half a trillion photos, but a billion users 鈥 it鈥檚 almost a 500 order of magnitude difference. Our algorithm is simply based on the number of tags, not on the number of photos, which makes it more efficient to search than standard approaches.鈥

Work on this project began in 2005 in Aarabi鈥檚 Mobile Applications Lab, Canada鈥檚 first lab space for mobile application development.

Currently the algorithm鈥檚 interface is primarily for research, but Aarabi aims to see it incorporated on the back end of large image databases or social networks.

鈥淚 envision the interface would be exactly like you use Facebook search 鈥 for users, nothing would change. They would just get better results,鈥 says Aarabi.

While testing the algorithm, Aarabi and Appel discovered an unforeseen application: a new way to generate maps. They tagged a few photographs of buildings around the University of Toronto and ran them through the system with a bunch of untagged campus photos.

鈥淭he result we got was of almost a pseudo-map of the campus from all these photos we had taken, which was very interesting,鈥 says Aarabi.

This work received support from the National Science and Engineering Research Council of Canada. It will be presented at the IEEE International Symposium on Multimedia December 10.

Marit Mitchell is a writer with the Faculty of Applied Science & Engineering

The Bulletin Brief logo

Subscribe to The Bulletin Brief